Large Catalog Limitations
Hello! I am working on transferring my archive of about 250,000 photos from a large lightroom catalog (taken from 2009-2021) to smaller capture one catalogs. I was planning on separating the catalogs into groups of 5 years. With about 135,000 photos being the maximum number in a grouping.
What are the size limitations of a Capture One catalog? Would 135,000 be too many photos for a given catalog? I would like to know if there will be significant performance hits and if there are any constraints I should be aware of.
Thanks in advance!
-
C1 is not known for a strong catalog. However, I am afraid there is not definite anwser. Some complain about a catalog with only a few ten thousand images, but I have also seen users in the forum with 100k+ who seem to be satisfied. Keep that in mind if someone replies here. Probably the only way to figure out is by testing yourself.
2 -
i only have 55k in my catalog. sometimes it works-some days it constantly crashes. its far from good. whats even worse is the support…
1 -
I have waited for years to finally be able to replace an Aperture library with a Capture One catalog. Somehow, Aperture managed to organise and display hundreds of thousands of pictures without a hitch but all of my many attempts to transfer my library to Capture One, which was the software I felt the most comfortable with, ended up in catalogs too slow to use or the whole Mac Pro crashing. I spent ten years trying to nurse an ageing platform while at the same time trying almost every DAM and photo editing software available without ever finding the simplicity and convenience of Aperture. Finally, I took advantage of the release of the new M2 Mac Pro to try again and to my good surprise, it worked. I now have a 285,000 images Capture One catalog which is fast and responsive and on which I can use up-to-date editing tools. I did it both by importing all the RAW images straight of the disk, which took about 30 hours, and by converting my old Aperture library with Avalanche, which took about 4 hours. Both new catalogs work fine and while trying to scroll or search through the whole library is not very smooth, everything else is. Now, I did have to throw everything and the kitchen sink at it: I am using a Mac Pro M2 192GB 8TB 76 cores with a Sonnet 4x8 NVMe card in Raid 0 (12,000MBs R/W). I also used a small(ish) preview image size (1920px) to keep the catalog a reasonable size (less than 400GB). The small previews make no practical difference since it is so fast to load and process the originals and, so far, it just works.
0 -
135,000 images in a catalogue would require quite a lot of patience.
I've had problems with Capture One's handling of catalogues more or less since I started using Capture One. It rarely crashes, but constantly becomes unresponsive – when switching tool tabs, selecting folders/albums, editing keywords, etc.
I just split my main catalogue (containing about 64k images) into four different catalogues in order to alleviate the issue, and while this has reduced unresponsiveness to some degree, at least Capture One recovers slightly faster now, it hasn't eliminated it.
(This is with a MacBook Pro M1 Max 64GB.)
Have a look at this improvement request.
0 -
I have a 65k+ catalog that works well on my iMac Pro and Intel MacBook Pro. I imported the original 20k from from Aperture (library + referenced files) when Apple stopped supporting Aperture. At the time, Aperture was a lot faster than Capture One 8 and the referenced files were a lot smaller.
I have also noticed that matching preview size to display resolution makes little difference in the time it takes Capture One to display a full resolution image. This may be because the preview size almost never matches the window size. However, memory is becoming cheaper and cheaper, so this is not a huge issue.
I am pleased to see Gregory's M2 Mac Pro results and hope these will be similar on the Mac Pro M2 studio Ultra. The size of the catalog has no effect on the sluggishness of the sliders and time to show a fully resolved image (now, often a second or two) as Capture One adds bells and whistles. I hope adding computer power can overcome Capture One's lack of multi-core multi-threaded code.
0 -
Here, upgrading to M1 Max from Intel didn't make much of a difference.
0 -
I think what made the most difference was the Sonnet 4x8 NVMe card. When I tried this with earlier computers, memory use was off the roof and an obvious bottleneck, but not this time and, indeed, there is little use made of all the cores available so even a base Studio should work. The one significant improvement was storage I/O, both of the catalog, on the 6000 MBs built-in SSD and the Raw files on the 12000 MBs Sonnet. You can get 8TB on a Studio, which is enough for both catalog and files and probably fast enough. Now that all the previews have been processed in the background, the program is perfectly responsive with no lag unless you are trying to put the whole catalog on screen for a search which takes 20 to 30 seconds but does work. I am sure Apple's patents have now expired and it is time someone ripped off the incredibly smooth, elegant (no sidecars) and powerful file management of Aperture if Apple won't give a new life to one of their best "pro" applications.
0 -
How long does it take for Capture One to bring up search results on your system? For instance when pasting a file name in the search field?
0 -
One last important trick: my collection is organised in daily folders and has been imported as such so while I occasionally do a whole catalog search for an animal or a place (I'm a wildlife photographer and being able to find all my Cnemidophorus vanzoi when needed is paramount..), I normally work with a daily folder on screen with usually no more than 1,000 images, or a selects collection of a few hundred images.
0 -
Opening the catalog translated from Aperture (283,442 images with all my annotations), takes 1 minute 30 seconds. Searching for keyword "vanzoi" returns 30 images in 1 minute 24 seconds but, interestingly, clicking on the keyword button takes only 5 seconds, for the same result.
0 -
Currently my largest catalogue contains 28k+ images. When I search for a file name, it takes 20 seconds for Capture One to bring up the result, that is, it brings up the result faster than that, but then immediately becomes unresponsive again and only fully recovers after 20 seconds.
When I click on a keyword in the Filters tool (same catalogue), Capture One becomes unresponsive for 15 seconds before bringing up the result.
0 -
I'm just north of 140k images in my archive catalog. Bog standard MBP M1 16/500. Catalog lives on the internal. All photos are stored in folders by year on various external SSDs. They are the fastest SSDs I can buy and I use tested/certified data cables. I have no problems. Opening images from xxxx year is as simple as plugging in the correct external. Every now and then images will show up as offline even after I plug in the appropriate SSD. Right-clicking and selecting Locate solves the problem. I don't know what kind being blessed me but I have very little by way of lag and that generally subsides within minutes of opening the correct year folder. I get a syncing metadata action box but that's all. I'd hazard the guess that no individual year folder has more than 16k files.
By way of process, I keep a current year folder in a separate, active catalog. Following each year end, I do my final dusting and cleaning and then import that year catalog into my archive catalog. The only time I access that archive catalog is when/if I need to do some type overarching search which, admittedly, is rare. The very vast majority of my 'looking back' is confined to the current year and prior year which are each contained in stand-alone catalogs.
Those of you with catalog gremlins please keep your hands off my laptop? :)
0 -
I've tried everything I could think of to solve Capture One's unresponsiveness issues, and I've been in contact with support about it for a long time. Their conclusion was that the cause is Capture One's lack of optimization for larger catalogues, and that the only thing to do was to split up larger catalogues into smaller catalogues. Also, I was told that this isn't something that will be fixed any time soon.
0 -
Hi Brian Jordan
All photos are stored in folders by year on various external SSDs.
...
I'd hazard the guess that no individual year folder has more than 16k files
You have only one folder per year? No sub folders?
0 -
Parent for the year. Subs for month and exported photos and tiffs for print. The don’t segregate exported images by month. Instead I split them into subs by format. Everything gets keywords so I can find an export or print file easily. I haven’t come up with a better way to store them.
So:
2023
—-01
—-02
—-03
….
—-12
—-Export
—-PrintEach month will also include any job or client specific folders if I decide to add them to my archive. I only do this if I think they’re somehow potentially relevant going forward. The Jones wedding will never be relevant to my catalog going forward so they won’t be added. If the Jones want photos down the road, images are named Job - yyyymmdd - original image name. If the client can give me that, I can go to the year archive drive and open the session.
Of note, I make heavy use of a standardized keyword list and smart albums. I am also very rigid about image names.0 -
Thanks Brian Jordan,
You have a folder structure which is probably not that much different to the structure of others (you have sub folders), this seems not to be the reason that the gremlins have passed by :-)
As you have several external SSDs, if you access that archive catalog for some type overarching search, which is rare, C1 is searching on the catalog which is internally stored but some images will be offline (unless you mount all SSDs at the same time). Do you experience that the performance and responsiveness is better if many images or drives are offline? What is the highest number of images being online at the same time (no. of images on the actually mounted drives).
Sorry for the many questions, I guess I'm trying to find out why you are satisfied, probably because you use your big archive catalog rarely and mainly work with two catalogs each covering one year. What's the highest no. of images in a years's catalog?
Thanks,
BeO0 -
BeO, I connect drives through a CalDigit Thunderbolt connector. 4TB Samsung T7 drives. I can have up to 4 connected at a time with no problems. If a search returns an image that lives on a drive currently disconnected, it shows an OFFLINE tag top center. Generally connecting the drive is enough but sometimes I will have to right click and select Locate. So far that's my only gripe and it's a minor one (and likely due to the way I manage files). It's a rare day I need to pull an image beyond the last 3 years and those are all stored on my current working drive and at most 2 archive drives. Those are connected basically always. My largest year has roughly 30k files. Some early years have only a few hundred.
0 -
In my experience, none of the following factors make much difference in regard to performance (not to say that they make no difference at all, just that it's a relatively insignificant difference):
• number of keywords in a catalogue
* number of albums/smart albums in a catalogue
* whether images in a catalogue are online or offline
* what sort of external hard drive image files are stored on (HDD/SSD)
* Intel / Apple Silicon (Mac).
0 -
Hi Gregory Guida,
Searching for keyword "vanzoi" returns 30 images in 1 minute 24 seconds but, interestingly, clicking on the keyword button takes only 5 seconds, for the same result.
When "Searching for keyword "vanzoi" means you put this string into the search box, then what happens is that a lot more image metadata is looked at, not only the keywords, this explains why it takes so much longer.
0 -
Hi Thomas Kyhn,
If I remember correctly you did a lot of tests in the past, importing catalog into a new one, splitting and much more. Did you ever try to create a completely new catalog and import all images including adjustments (which you export before)?
0 -
Hi BeO,
Yes, I've tried importing my old catalogue into a newly created catalogue, which made no difference. I think I also tried exporting the whole catalogue and then importing the resulting catalogue into a newly created catalogue.
Recently, I split up my main catalogue into four parts (exporting three parts as separate catalogues, and deleting the images contained in these three parts from the old catalogue). With these four catalogues, Capture One is about as responsive/unresponsive as you should expect, that is, it becomes unresponsive regularly, just like before, but recovers faster.
0 -
Hi Thomas Kyhn,
Maybe there is something inherently wrong in your catalog from an earlier version, bug, or thunderstroke, which is inherited by all your catalog exports/imports. I'm not saying that it is, but if you are as annoyed from the unresponsiveness as I think you are then it might be worth a try to export all images with adjustments and reimport them into a new catalog, and recreate the albums and smart albums manually (and gradually) from scratch in that new catalog.
0 -
Hi BeO,
Do you mean export all images individually with adjustments? How would you do that (I'm not home, so I don't have access to Capture One right now)? And would that include metadata?
0 -
Hi Thomas Kyhn
Last question first: Yes, it would include the metadata, keywords etc.
How would you do that?
I did this once, when I switched to the approach of using both a catalog for searching and one (dummy) session for editing. And the reason is that I trust individual sidecar files more than a catalog. But this is another topic to discuss.
Sessions store their settings (adjustments incl. metadata) in the CaptureOne/Settingsxxx subfolder, in .cos files. Exporting the images including adjustments from a catalog creates this subfolder and the individual .cos files (it also exports or copies the layer masks, read ".comask files").
The easy way is to select All images and export them. Downside is that all images end up in the same folder. Tokens can be used to create the subfolders but whether or not the tokens satisfy your needs remains to be seen. For testing purposes this should be viable.
(When I did this I wanted to keep my folder hierarchy and folder names and the use of tokens did not allow me that, if I remember correctly, or at least the export could not be executed on the existing folders without appending a _1 for each image file, doubling the images and ending up with imagename_1.cos files, hence I opted to export all images of each catalog folder separately to a dummy location and manually moved only the CaptureOne subfolder to the original image folder).
1 -
Export Originals with
Format: Token "Image Name"
Sub Folder: Token "Image Folder Name"but for the folder hierarchy above the image folder I haven't found anything adequate.
0 -
save as template and you get a empty catalog with all smart albums, etc. create a new catalog using this template and import all images without moving them. done. right?
0 -
Hi BeO,
Thanks for explanation. I'll have a look at this when I'm home again.
If I was to do this, it would probably have be folder by folder in order to keep the folder structure.
0 -
In my case it would probably be a good idea to avoid keeping anything from current catalogues. I don't use that many albums anyway, so it should be reasonably quick to set up new catalogues from scratch.
0 -
If your folders which hold the images have all distinct folder names (so, not JPG for example, but a date) then it is probably relatively easy to export them all and only re-create the folder structure above by manually creating parent folders and moving the image folders beneath, with Explorer/Finder. (Before you import them into the new catalog). If they are not unique, or you have images in parent folders, then manually one by one is the way to go.
But for testing only (whether or not it speeds up your catalog) you can certainly use a token image year/year-month or so, which then is automatic.
0 -
you just pick the root folder and say search all subfolders. c1will search all and leave the structure as it is. you should end up with the exactly same catalog- just fresh and probably without hickups.
me personally worked with a main catalog where i add each passed year. it was only approx 60k pictures.
all worked until it didnt and the catalog started to crash.
ended up splitting the catalog again.
oh- did i mentioned the laughable support? these guys are a joke. great software if it works- a shame when it doesnt...
heres one pro tip how to make the support get back to you if they went silent.
post one nasty comment on IG- within 2 hours they will tell you they just pm you and they are sorry to hear you are not happy.
i mena not that this was from great help- but at least they answered.... lol
0
Post is closed for comments.
Comments
43 comments