Very poor Performance with fast SSD
Again and again I hear and read that Capture One Pro needs a fast SSD or benefits greatly from it. But that's probably not even half the truth. With a 20 times faster SSD I cannot quite double Capture One performance at the moment.
I have come quite far in rebuilding a totally crashed and unrepairable catalog. In the course of this work, I was once again annoyed by Capture One's shockingly slow read and write accesses. The catalog contains about 50,000 images, which is quite little from the point of view of a photographer who really has to work with it. I therefore decided to bring forward a long-planned step and afford a really fast SSD with 7.68TB. NVMe for U.2, a Samsung OEM Datacenter SSD PM9A3 7.68TB.
Well, the SSD is fast; I tested it again extensively after the first results under Capture One, which I did not want to believe at first. But Capture One pretends that a fast HDD is still at work. Both reads and writes are - again tested - slower than my HDDs can read or write. Considerably slower, in fact. When writing previews (size 1440px), C1 manages a maximum of about 400MB/sec read (average about120, when writing a maximum of only 9.5MB/sec at all. I'm so shocked. That's not possible, is it?!
That Capture One for Windows (V16.1) so not even somehow benefit from the speed of the SSD (read ~5000MB/sec real, write ~3000 real, both measured at OS level with Powershell) is clear. I/Os of the new SSD are also tremendously faster hardware-wise than any HDD I know. Now I already know that many small files.... But that doesn't really explain the huge performance drop to me.
Where is my mistake?
Kind regards Ernst
-
If you have a horse-drawn carriage with several horses, the speed is limited by the lamest horse. It doesn't help much if you swap one of the horses with a speed-king.
I know my comment doesn't help you directly but that's the crux with performance where many steps are involved which are executed by different actors like cpu, gpu, and architecture, ram, disks, operating system and drivers, a lot of factors. Usually, when trying to improve the performance of a complex process it is best to identify exactly which steps and ressources take the most time and work at their limit and thus are bottle-necks. But sometimes it is just how the software, here C1, is implemented.
1 -
Thank you for your answer. Let's have a look at my horses:
Asrock 570 Creator, Ryzen 3800X, 64GB RAM, Nvidia GeForce RTX 3070, Windows 10 and Software installations (including settings etc.) on aSeagate SSD NVMe Firecuda, all data on Samsung PM9A3. The only lame horses are my harddisks (WD Red Pro) for Backups - but they are all external connected with Thunderbolt ONLY WHEN NEEDED. And there is a Card Reader für CFexpress connected with thunderbolt (normally without card in). All devices except keyboard, mouse and LAN are connected by thunderbolt or WLAN (printers).
No software installed that is not needed, no tests on this machine.
What could I have forgotten?
0 -
My last question was a serious one. I sometimes forget something or do not see it as a problem.
0 -
John wrote:
I've heard it said that there's barely a difference between an NVME SSD and a SATA SSD even though the transfer speed they are capable of is dramatically different (I've never verified that myself)
I did. I did not experience a major difference so I currently store my catalog, sessions and all images on a SATA SSD.
I couldn't agree more to what John said in his post.
Ernst, your specs look very decent, the only thing I notice is that you are running an AMD CPU whereas C1 recommends an Intel CPU. Whatever that means, maybe nothing.
Recommended system requirements
- Intel CPU with 4+ cores
1 -
John wrote:
I don't think there's necessarily any mistake here. Seeing a video of how you normally use the program would provide a little more point of comparison to other fast computers.
If you like, I can give you a video in a corner of my homepage (www.weinzettl.info, where you can find all contact infos. If you can provide me your mail address I could send you the url for a video). But when starting Capture One with "All images" you will see nothing interesting for a couple of minutes till Capture One has build all the groups, projects and albums and has gathered all information it needs to start work.
If I start with another (smaller) album the delay is not the problem. But as I must access "All images" the first time after start to find a special group of images I have to wait for the same time as starting with "All images". And normally I have to start a search from "All images", the task mostly is like "get all images that contain 'Sonja' and are made in 'street' environment". Or similar.
It would be nice if both Adobe and C1 gave better guidance on where to invest in the hardware to best optimize speed, but alas neither provides much useful info at all.
I agree of course. But technical service has at least told me long time ago that uising a very fast SSD for images and datas would "help a lot". Now I use one of the most performant SSDs and it does not help more than a bit.
I've heard it said that there's barely a difference between an NVME SSD and a SATA SSD even though the transfer speed they are capable of is dramatically different (I've never verified that myself).
I have verified. The difference is dramatically - except with Capture One. I have defined the type of SSD above and it is one of the fastest SSDs in real work load. The PC feels now like real time computing no matter if is in the ten year old Indesign with illustrated travel book (>300 pages, >500 images) or in CRM. And it did change a lot while backuping in Capture One (no surprise).
So far from what I've read, it seems that the fastest CPU you can get is definitely worth it.
My CPU was the second fastest at the point I build the PC. And it is a very fast till now, even when there are some CPUs more performant at the moment. CPU should stand the test. ;)
For disks, your experience and what others have reported is that C1 and LR just rarely maximize the transfer rate of your disk.
Yes, this I have found within my tests. Capture One does not even use half speed of my former used harddisk.
if you have a motherboard that is PCIE5 NVME capable
My motherboard is PCIe4 as the SSD and the U.2 adapter are too. Should be enough.
I would think that if you have a fast card reader, then the copy part of import should significantly benefit from a disk with fast write times as will backing up the catalog.
There are fast card readers, all connected via thunderbolt, normally is only one online. And yes, backups benefits from new SSD.
Thank you very much for your very detailled answer!
Kind regards Ernst
0 -
BeO wrote:
the only thing I notice is that you are running an AMD CPU whereas C1 recommends an Intel CPU
I do not know exactly if this could affect performance in that degree, but my CPU was the second fastest on the market at the time I configured my PC. Okay, Threadripper .. but this was really oversized at that moment.
It seems that I have to be happy with the enormous boost I got for all other software.
Kind regards Ernst
0 -
John Friend wrote:
It sounds like this is an issue at the intersection of how many images you have in your catalog and how you're using the catalog and not so much an issue with the regular image processing of a particular image
It is definitely not an issue of image processing. For image processing I would need not more than the speed of a very old harddisk. It is a matter to administrate images. You may say, this is not a strengh of Capture One - and you are right. BUT if a software manufacturer offers a software for workflow including metadatas, albums and more, I suspect it should work. If manufacturer does not provide metadata, albums, groups and project, possibility to filter images and more, it need not to work - but no one would purchase it.
Therefore I thought - til yesterday - its a matter of "brute force" to bring Capture One to perform at least reasonably. And now I proved it to be a false assumption. Not happy, not even amused about this.
you probably won't like what you see there as larger catalogs are simply not a strength of C1
This is what I know. But on the other hand I have to administrate overall about 800.000 images from different sources. This would be a large catalog. I have to split, yes. But to handle eighty catalogs if I have to pick out 200 images for one customer is not the way it works. Even when Capture One introduced catalogs and told us not to use it really. For this reason I accept some restrictions regarding performance. But what I get (the last time going worse from version to version) is waitimng minutes from step to step.
I really wonder why Capture One has introduced catalogs with such "funny" restriction and aiming at professional photographers at the same time.
Okay I have seen I can do nothing against.
Choosing a different DAM (that is more optimized for large numbers of images and the types of cataloging things you're doing) and using C1 only for editing, not for organization would be another option.
I would have done but I have not only to pick out the images - what would be an easy work with a DAM. I must davelope EVERY SINGLE IMAGE then to get them all to one color impression and same colotspace, even when they come from different color spaces, are made with different light (daylight, cool, warm, LED, neon...) to make them fit for what customer need (web, offset, wallpaper, showroom etc.).
Couldn't I develope incoming image for all theese purposes at the beginning? Hardly, aside from the fact that they com in with different photo sessions in differenz years. No chance. And I didn't find a DAM working with Capture One as a "plugin". I would to have open image by image with Capture One...
Strange conditions, maybe, not many users will have them. And Capture One does not do anything. No other software did the color job as good as Capture One. But I have to go into testing if this handicap is the same as five years before. Overall the performance handicap proves to be too costly for me.
Thanks again, John!
Kind regards, Ernst
0 -
I have to administrate overall about 800.000 images from different sources. This would be a large catalog. I have to split, yes. But to handle eighty catalogs if I have to pick out 200 images for one customer is not the way it works.
...
And I didn't find a DAM working with Capture One as a "plugin". I would to have open image by image with Capture One...
Well, I think there is a way. But the workflow is a little more complex than just working with catalogs. And it is not perfect, but way faster. I did't test it with 800k images though.
Here's the highlevel approach/roundtrip (I can discuss it a little more in case you have interest):
- Photo Mechanic Plus as your catalog for searching
- Once you found the 200 images in PMPLus which you want to edit in C1 you drag&drop them from PMPlus to a C1 session album.
- Once you edited the 200 images in the C1 session you export new jpgs in the same directory as the raw files as the visual representation of the raw files in PMPlus
- You do all the edits, including metadata and keywording in C1 (I found that PMPlus and C1 are not fully compatible for a two-way synchronization.
- That approach requires that you start with all your adjustments are located in the subfolder CaptureOne/Settingsxxx in each of your image folders, just as if the adjustements were created by a C1 session.
This start state you can achieve even if you have only catalogs but it requires some manual effort (export originals "including adjustments" of each folder in the catalog to a temporary space, and moving that exported subfolder (CaptureOne) to the original image folder location. The effort depends on the number of image folders you have for your 800k images in your C1 catalogs.
1 -
btw, not using C1 catalogs for 800k images means no more "rebuilding a totally crashed and unrepairable catalog".
1 -
BeO wrote:
btw, not using C1 catalogs for 800k images means no more "rebuilding a totally crashed and unrepairable catalog".
This was first time in ten years. Quite new situation.
Btw: I just finished rebuilding as close as possible. Means there are not all images (about fifty to hundred) processed as they where before catalog crashed. But theese were delivered before, so I can leave them as they are till they are needed next time. And this catalog is stable surprising till now.
Kind regards Ernst
0 -
BeO wrote:
Well, I think there is a way. But the workflow is a little more complex than just working with catalogs. And it is not perfect, but way faster. I did't test it with 800k images though.
Here's the highlevel approach/roundtrip (I can discuss it a little more in case you have interest):
I am interested.
The way looks a little bit complicated and will generate thousands over thousands of C1 sidecars all over my SSD. And at the moment it looks so as if it will be very challenging to get all adjustments which are done til now into session folders so I could start.
Another thing that makes me heasaches: transfer of metadatas between the different platforms. It needed two years to organize, import and tag all the images...
The effort depends on the number of image folders you have for your 800k images in your C1 catalogs.
There are overall not quite hundredthousend image folders. Not all of them hold originals, it is a tricky system of archieve. ;) Working fine till now.
I will try to test Lightroom within the next two month too. If it can give me a useful way to manage the different colorspaces, it could be an alternative. I do not like Adobe, I hate subscription, and I know, I will loose something but get at least a working database. No problem with - let's say - 200.000 images in one catalog (more I do not know (they use SQLite too when I remember right, but the db-access is programmed ways better).
Kind regards Ernst
0 -
Yes, you would have many C1 sidecar files in CaptureOne subfolders for the preview and thumbnail cache and the settings files, but also xmp for metadata exchange with PMPlus for raw files. I love sidecar files because that means this data is not buried in some catalog file, but I understand that's not everyone's perspective.
To get to the state where you have all the subfolders in place is indeed next to impossible if you have almost 100k image folders. Here's a feature request which deals with it, if you like feel free to upvote and comment:
The metadata: I do prefer the metadata editing in C1 over PMPLus and for the sake of security I would only edit them in C1 and do the sync into the direction to PMPlus, which means never edit metadata in PMPlus and to be on the safe side don't use AutoSync in C1 (there is Load and Full Sync but not Write), rather manually sync metadata for the changed images in the session.
Keep this approach in mind as an additional option, should your Adobe testing not be satisfying. You definitely should do a lot of testing before really commiting to this approach.
Best regards
BeO1 -
BeO wrote:
Keep this approach in mind as an additional option, should your Adobe testing not be satisfying. You definitely should do a lot of testing before really commiting to this approach.
I am behind now because of crashed and restored catalog. when I have cleared the backlog I will have a closer look on Photo Mechanic Plus. Just to see if it could solve my problems. As I read your posts I could sync via XMP?
Best regards
Ernst0 -
Yes you can sync the metadata incl. rating, color tag and keywords via xmp for your raw files. There is a problem with hierarchical keywords though, C1 and PM write them differently to the xmp file, but it should be ok if you use C1 to edit and PM to read them (one-way only).
For jpgs and tiffs PM can read the metadata from an xmp too, if I remember correctly, but it doesn't write to it, and after reading the xmp I remember PM deletes them. Which is not a problem if you go for one-way sync only (C1->PM).
You can configure PM to combine imagename.raw and a imagename.jpg to one visible item only, with the jpg being shown as the thumbnail / preview, which means C1 adjustments can be seen in PM. The raw and the jpg need to be in the same directory and have the same name.
You would not see variants (unless you want to create them as additional jpgs with a different name, so no link between the raw and jpg in PM).
When you drap&drop such an image item to a C1 session album the raw file path will be passed to C1 (not the jpg) if I remember correctly, which is good. Until v22 you see all variants of the raw in the C1 album, so nothing is lost here, not tested with v23 though.Btw. you can test the drag&drop action by searching with Windows Explorer for images and dragdrop them to a C1 session album. A CaptureOne subfolder will be created in each image folder but of course with no adjustments other than your defaults settings. Of course, if there is already a CaptureOne subfolder with .cos files / .comask files, then these adjustment files will be used.
For your information, I did not buy into PMPlus because my hope was for a two-way sync between the two applications, which doesn't work if using hierarchical keywords, and I did not like the user interface for metadata editing in PMPlus.
And with a rather low number of images the performance of my C1 catalog is sufficient, so my approach is C1 catalog to search for images, then drag&drop them to a C1 session to edit them, remove and synchronize (import) them back to my C1 catalog.
But I like the catalog function of PMPlus, and would I have too many images like you I would have swapped my C1 catalog with a PMPlus catalog (after more extensive testing, especially for jpg, dng and tiff formats, I focussed more on testing for raw formats).
Kind regards
BeO1 -
SFA wrote:
Joining on late here but a couple of general observations.
Never too late. ;) Thanks for help.
SSD speed measurements are great for marketing and for consumers to gain some impression of their capabilities together with which sort of application a particular disk is aimed at. This is also true for spinning drives.
That's not new for me, I have developed computer hardware for years. Long ago but the rules didn't change much. ;) What I expect is what I got. Nearly all other software I use got the boost. Just Capture One (and a second not often used product) does nearly not. And I was shocked HOW Capture One does nearly not when working with catalogs.
Continued work, especially working through images sequentially, is generally smooth and responsive
Same here. But this is not what I do most.
My final observation is that I strongly suspect that internal components within a system, stuff that is not in the headline specs for processors and memory and disk speeds, can significantly affect the performance delivered compared to the performance on paper. As can things like backup strategies or system level file indexing and so on, all useful features in situations where they may be required but potential resource brakes applied to outright performance.
One potential activity of that nature in C1 might be Synchronisation of external EXIF/IPTC data but there might be many more - virus-checking software for example.
Backups are startet manually only. In this case Capture One and all other "normal" software apps are closed. The only virus-checking software that is on my pc is Microsofts Defender. Nothing else here. And no software installations that are not needed at least once a week. At the moment there is no use of XMP or non-CO sidecars. This will have to change because of preperation when migrating to other software. But at the moment this cannot be a process decreasing software performance.
What I can say today: Performance with Windows version while working with "All images" decreased with increasing keywording. A matter of (programming access to) database as it seems. I could not see same issue on Mac - but I cannot change to because of other software.
BTW: I cannot find a serious hardware bottleneck, neither Microsoft did find. Most recent (some minutes ago) Windows Assessment of my machine:
CPUScore : 9,3
D3DScore : 9,9
DiskScore : 8,8
GraphicsScore : 9,9
MemoryScore : 9,3
TimeTaken : MostRecentAssessment
WinSATAssessmentState : 1
WinSPRLevel : 8,8
PSComputerName : WS10It is not the most performant PC ever but should be enough for most software...
Kind regards Ernst
0 -
BeO wrote:
Yes you can sync the metadata incl. rating, color tag and keywords via xmp for your raw files. ...
A lot of hints. Thank you! It will last some time to check all out...
Kind regards Ernst
0 -
I've had wildly varied performance, regardless of whether or not I'm working off my SSD (Samsung 990 Pro Gen 4) or my 7200 RPM Mech Drives.
I have a 7950X, 64 GB DDR5 and an Nvidia 4090 and it's pretty random. Sometimes making adjustments to an image will take over 1 second for the change to take effect on the screen.
Very strange
1
Post ist für Kommentare geschlossen.
Kommentare
17 Kommentare