Skip to main content

⚠️ Please note that this topic or post has been archived. The information contained here may no longer be accurate or up-to-date. ⚠️

Improve catalog loading performance?

Comments

29 comments

  • Paul Steunebrink
    When you import an image into the catalog, Capture One generates a preview for that image, stores it into the catalog, and uses that preview as a quick reference instead of the raw file until you print, export or process.

    Maybe you have recently imported a bunch of images and Capture One has not yet finished generating all previews. Until that is finished, performance can be slow.

    Another issue could be that the preview size is set to low. You can set this in Preferences, Image tab. You might want to check that and set it to a size at least the number of pixels of your main display, long edge.
    0
  • NNN637158872702285724
    Thanks for the reply. I did just import a bunch of images into the catalog (about 1800 raw files) but I just let it sit there and do its thing until it was done. It took quite a while, around 2-3 hours as far as I recall.

    But this here happens as I open Capture One. Every time. It starts with image 1 in my catalog, loads that (with disk activity), then image 2, loads that, ... etc. It's very obviously accessing the HDD for each raw file, and it's doing something more intensive than just checking the file size or other minor metadata. It doesn't take 3 hours, it takes perhaps 10 minutes before the catalog is completely loaded and ready to use inside Capture One.

    Not sure about the preview size. What would happen if the preview size was too small in the catalog? Would that explain the activity on my HDD?

    - Michael
    0
  • Ian Leslie
    I'm going to guess you are working with the "All Images" collection. Let me know if I have that correct.

    I do not have an answer to what C1 is doing. I am convinced that whatever it is C1 should not be doing it again and again. I have an ongoing issue with support - in the old system with v12. I just upgraded to v20 and the issue is still the same. I will wait for the support team to climb out from under their pile before engaging them again.

    My catalogue is 60 000+ images (I keep forgetting to check the current count). Both the catalogue and the files are on a fairly fast HDD.

    Here is what I see:
    When working with the "All Images" collection, if I try to filter (in my example I filter four and five star images with the keyword Great Blue Heron - which pulls back twenty something images) it takes 10 - 15 minutes to complete the search. That whole time the system is churning the HD.

    Here is what I think:
    I don't know for sure what it is doing but it is totally inappropriate - the information for that search should be in the catalogue SQL database. A query to get the list of files should be seconds. Actually rendering the images should take the longest time and I think we can all agree that rendering 20 images should not take 10 minutes.

    What I have done - with the advice of support
    * Turned off two way syncing of metadata for images. The theory was that with that one C1 felt it needed to update the metadata information in the database for each image. This had no effect.
    * Adjusted the preview size to better match the actual size of my view area. The theory was that C1 felt it needed to regenerate preview for all mages. They thought I had too small a size but I actually had too big a size. This had no effect.
    * Regenerated the GPU information (I forget the name). I have no idea how this would help but sure I'll try anything. This had no effect.

    Support's final advice was to move everything to SSD drives. I advised them that was an insane thing to suggest - it will not stop constant file access and while it might make things faster (obviously) it will not get to the bottom of their bug / design problem - how is accessing every file everytime ever the right thing even if it's done on a fast drive?
    0
  • Ian Leslie
    Ooops I forgot to talk about the workarounds I have been using - this may help you.

    The problem seems to stem from the raw number of images being handled by the filter pane in the app. If I create smart albums that filter a much smaller number of images then filtering with the filter pane performs OK.

    I don't have issues with working with images in my library folders (I think my largest directory is 1000 images but most are under 200).

    I don't have issues working with smart albums that bring back a subset of the whole catalogue. So, I have created smart albums for each year. Usually when I am looking for something I have an idea of a year or two that it is likely in.

    I am currently experimenting with smart albums based on keywords - all people pics, all inset pics, all bird pics that type of thing. I have not had time to finish that work to see if that helps too.

    Try creating smart albums or even regular albums and use those to work with your images - see if that helps.
    0
  • NNN637158872702285724
    Thanks for taking time to respond, Ian.

    Clearly there is something going on here which forces Capture One to visit every image in the collection. It could be that it just wants to verify that the image is there or that it has not been altered. Both of these tests could perform really badly on a HDD depending on how it's done. Even without access to the source code, I would think that it should be fairly trivial for a developer to determine what type of access is going on, but there may not be an easy fix which could explain why the problem has not been addressed yet.

    At the very least the user should be able to set an option which disables these types of checks. For example, if the user knows for a fact that these images do not ever change, then there would be no reason for Capture One to verify anything at all, it could rely entirely on what metadata was initially collected and persisted in the catalog.

    I will try working with smart albums and libraries and see if that helps. I'm new to Capture One so there are still many features I am not aware of, and which can probably remove or at least mitigate the problem somewhat.

    Anyway, thanks for your response and let's both hope that Phase One figures this out.

    -Michael
    0
  • David Bleeker
    Since C1 20.02 Catalogues are absolutely terrible, loading takes ages. And catalogues only show a few folders. Terrible! Now looking how to go back to 20.0

    Not Happy!

    David
    0
  • Samoreen
    IanL wrote:

    Here is what I think:
    I don't know for sure what it is doing but it is totally inappropriate - the information for that search should be in the catalogue SQL database. A query to get the list of files should be seconds. Actually rendering the images should take the longest time and I think we can all agree that rendering 20 images should not take 10 minutes.


    I totally agree. Please find below the links to a few (IMHO) related threads :

    [The Capture One forum has migrated to a new platform, as a result all links to Capture One related postsstopped working and have been removed]
    [The Capture One forum has migrated to a new platform, as a result all links to Capture One related postsstopped working and have been removed]
    [The Capture One forum has migrated to a new platform, as a result all links to Capture One related postsstopped working and have been removed]

    I think that the C1 20 catalog is totally broken. Such an intensive disk activity is certainly not necessary. Many operations appears to be uselessly repeated just because the necessary/searched for information is not stored in the catalog database.

    I'm now using C1 from Lightroom using the scripts I suggested here :
    [The Capture One forum has migrated to a new platform, as a result all links to Capture One related postsstopped working and have been removed] .
    0
  • Ian Leslie
    DavidBleeker wrote:
    Since C1 20.02 Catalogues are absolutely terrible, loading takes ages. And catalogues only show a few folders. Terrible! Now looking how to go back to 20.0


    Sounds ugly. I am not experiencing problems that you describe. As frustrating as it is I think you need to log a support request to help trouble shoot your totally different problem. Perhaps validating your catalogue and letting it fix any errors it might find would be something you can do in the mean time.

    The issue described by the OP are present in v11, v12 and v20. Going back to 20.0 from 20.02 will have no effect on the issue the OP is experiencing.
    0
  • Thomas Tveit Rosenlund
    IanL wrote:

    My catalogue is 60 000+ images (I keep forgetting to check the current count). Both the catalogue and the files are on a fairly fast HDD.

    Here is what I see:
    When working with the "All Images" collection, if I try to filter (in my example I filter four and five star images with the keyword Great Blue Heron - which pulls back twenty something images) it takes 10 - 15 minutes to complete the search. That whole time the system is churning the HD.


    Good to know I am not alone With this problem. I have about the same size of my catalog and experience much worse load time if I ever try to choose all images, the program will sit for minutes just loading the catalog. Thats before I do any actual searching or filtering. It's certainly a horrible design. They must have done something wrong With their database design to achieve the performance I am seeing.

    In addition the Program is accessing the disc, I don't know why as my computer should be able to hold the entire database in memory if needs be. Do it really have to load every preview from the entire catalog as well? Or worse, Access every raw file?
    0
  • Ian Leslie
    ThomasTR wrote:
    They must have done something wrong With their database design to achieve the performance I am seeing.

    In addition the Program is accessing the disc, I don't know why as my computer should be able to hold the entire database in memory if needs be. Do it really have to load every preview from the entire catalog as well? Or worse, Access every raw file?


    Yep, I have no idea. The good news my tests with smart albums using keyword filtering worked. I can now pick a year and then filter based on other criteria or pick a high level keyword (for me birds, animals, insects, events) also work to mitigate the problem.

    It seems like they are applying some level of filtering (the smart album criteria) and then reaching out and touching every file that matches and that is what we are waiting for. It still just a guess.
    0
  • Bud James

    I've been using C1P for about 3 years. My Lightroom catalog had about 150k images. This choked C1P on the import. I ended up having to split my LR catalog into several smaller C1P catalogs to keep the catalog under 20k images. C1P does not like large catalogs, period. I'm running a iMacPro with 64GB RAM, 2TB internal SSD and Thunderbolt 2 connected OWC Thunderblade 8TB SSD. The read/write speeds of the internal SSD averages 2,700-2,900 MB/S while the external SSD averages 2,100-2,300 MB/S. Even with this setup, C1P is sluggish once the catalog exceeds about 20k image.

    Regards, Bud James
    Please check out my fine art and travel photography at www.budjames.photography or on Instagram at www.instagram.com/budjamesphoto

    0
  • Lars Sundin

    I'm sick and tired of the slow import function in CO20. During import it only uses a small portion of my computer performance. Can't find any bottleneck there. One import lasted 75 minutes in CO and it took LR just 8 minutes. What's going on CO?

    My intention was to migrate from LR to CO because I really like the way CO handles RAW files. But since the poor performance on my relatively fast computer made me think twice. My main DAM will be LR and for now I will use CO for special projects only (if at all).

    I hate to admit it, but I was betting on the wrong horse.

    0
  • KEN DICKSON

    I am having similar issues with CO20 in that it was taking ages to load folders of files into my catalogue, the more I loaded the slower the machine got to the extent that the programme itself would fail. Check on CPU and RAM (24gb remaining) and only 7% CPU so nothing untoward there. I filed an observation with support and all they told me was that it could be down to the fact that Apple no longer support the NVIDIA GPU that is installed in my iMac.

    This is rather strange as I have checked that Metal is supported for the graphics card installed.

    I deleted an old version of CO13 from my machine in case there were any conflicts and reinstalled the software, all was a lot better but eventually slowed down to a point where my machine froze and the software wouldn't even respond to a "force quit" - the only way to close my machine down was to go to the terminal and "Kill" the Capture One Process.

    Having reset everything for a third time I then discovered a work around - either use Sessions or create multiple Catalogues, as soon as I did that I was able to down load all my files (2300) in under 25 Minutes - the software now seems stable, but I am not committing to purchase until I can be convinced of it's integrity. It is a great pity that you should have to go through this learning process as the software itself does a spectacular job in processing,

    0
  • David Car

    I'm getting nervous now! I just subscribed to CO20 and am noticing it takes ages to load my catalogue. I've only imported 5,600 images from my Lightroom catalogue and it chokes trying to load the thumbnail image previews. I have the preview generation set to my long screen resolution size. I've "rebuilt" these thumbnails a few times, but every time I load up CO20 it opens some quickly and about half don't, and it takes a long time to fill in. I'm running on a newer Macbook Pro with 16GB RAM and 2TB Samsung SSD. It's not a problem with LR. It's not file specific as it does this with jpg, tif and NEF files. I really do prefer CO over LR and am hoping to make the jump, but if I can't work in it, not sure if I'll be ready to make the switch. I hope they are listening and will let us know they acknowledge this and are working on a fix.

    0
  • KEN DICKSON

    David - since my post I have made some changes in that I had been storing my working files on a NAS drive and to be fair the transfer rate from the drive to my computer was very slow - this appears to have been the issue.  I have now transferred all my working files to an external SSD and reloaded the catalogue from that drive - it consists of 2500 x 25 mb NEF files (Nikon Raw) - uploading took 8 minutes from start to finish and the software is now behaving as it should do. Processing images within the software are being carried out as quick as Lightroom and for the moment my fears have been put aside. I think it's true to say that they prefer files of a single type i.e. JPG or NEF but it gets a bit fussy when formats are mixed for some reason. 

    I must say that I am in a better position now with the software - but have gone through the issues you have described which seem for the moment to be resolved.

    It was rather disappointing to have received advice that the issue could have been due to the graphics card of my machine which is NVIDIA as there would have been no other solution than updating my equipment to run the programme. I was in a way prepared if somewhat reluctant to do that and I am pleased that I have appeared to have found a solution before I actually placed an order, - However your equipment should be well within the specification of what is required - I would file an issue with customer support if I were you just to make them formally aware that you are having this issue just in case there is actually a problem with the structure of the Catalogue Data Base.

    What you also might like to try is to load you files into "Sessions" as opposed to loading everything into one catalogue.

    Good luck...keep us posted.

     

     

    0
  • David Car

    Ken,

    Thanks for the info. I too have some of the files on a NAS, which is interesting, because I have waaay more files still on the NAS and Lightroom accesses more from the same location no problem. I don't want to pay the cost to transfer all my files to an SSD. I have the NAS backing up to iDrive so it's my storage and backup solution. That option won't work for me. 

    I got CO20 because I like the Sessions option. I don't want to archive all my files in Sessions. That is why I use the Catalogue. I can see everything and keyword and search for images I want. I'm needing some now so I'll have to go back to LR to find them as I can't get CO to fully load the 5,600 of some 15-20,000 that I have. Also, with Sessions, I'd have to duplicate everything as I'd have to put copies in the Session Capture folder, if I still want to retain a Master Catalogue. I suppose I could export everything in LR to separate Session folders and retain the Master LR Catalogue, but I don't relish the idea of breaking up and exporting 20k files to other locations. Right now the files are all in the same place for LR and CO... in their own catalogues. I do a Single Edit Session to do single edits for small jobs, and another Master Session I use to browse all my photos.

    Looks like I'm in the same boat as another poster, will have to use LR for my cataloguing and storage, and see if I can work with CO for the editing. Will have to see how that goes.

    0
  • KEN DICKSON

    Good Luck David - I have no problem using a NAS drive with either PS or LR as I think Adobe handles the files differently, but if you check your transfer speed (I used Blackmagic speed test) you will probably see the same as me that the transfer speed to CO is far too slow to be workable. Once I had transferred everything from the NAS onto the SSD then everything changed and I could use CO without a problem.I have now ordered a second SSD and will transfer the OS-X onto this disk to improve things further.

    I hope you are able to find a solution as CO seems to offer some very interesting rendering options and files look a lot more "alive" in CO.

    Cheers for now.

    0
  • Corey Surette

    Hello,

    CO 20 kept autoloading meta data and previews every time I opened it up. I use a 2017 13" MacBook Pro that has a 128GB SSD. I was using a 1TB Seagate Ultra Touch portable HDD for all my images and catalogues that I use, but I felt it was slower than it should be. I recently picked up a 500GB Samsung T5 portable SSD for $100 CAD on sale to work off of and now use the HDD as just a backup. Even though performance was noticeably better it still kept autoloading meta data and previews for the ~8500 images I have and did it quite slowly. I decided to put the catalogue on my Mac instead of the Samsung SSD and the autoloading/syncing stopped and CO now works the way it should on my machine.

    I don't recommend using any HDD for anything other than just storing files. If you are going to access files on it regularly to work with them you are going to have a slow time, even more so the larger the capacity is. Hope this helps in someway or another and can clarify anything if anyone needs.

    Cheers! 

    1
  • Jeff Leak

    I am having similar issues with Capture One catalog.  Here are some details of my situation:

    • Coming from Lightroom Classic
    • Windows 10 Desktop Tower
    • Intel i9 9700K
    • 32GB RAM
    • Three SSD Samsung Drives
    • 4TB LAcie Porche Drive
    • Nvidia RTX 2060
    • Fujifilm System:  X-T30 & XF10

    My computer is a boss ... that is certainly not the issue.  Lightroom just smokes through.  I have two Fujifilm cameras with about 2,000 images total.  I like to keep thing tidy, so my hard-drives have folders for each camera, their RAW, and the JPEG finals.  I wanted to keep the X-T30 & XF10 files separate, so there are two catalogs.  

    Open the X-T30 catalog on "all files" and everything is there and no immediate load issues.  The problems come when importing new images.  Maybe I took 60 new RAW files with the X-T30 ... I go to import, find them on the SSD, then it bogs down trying re-load every photo again.  I check-box the "no duplicates" and then it has to re-scan every single file to compare them to the new 60 I am trying to import - then it will find that 1,900 (or whatever number) have already been imported, and then bring in the new 60.

    That is a non-starter ... it makes just logical sense, like others have said, that is should check the database, and fly through this little import.  It does that every single time on either catalog.  It is like it doesn't remember something about all of the previous images and has to compare - though they are all still in the same spot as last time.

    My solution: 

    1. place your images from your camera to your RAW folder on the HD as you probably already do.
    2. Open the catalog you want to use - notice all previous photos are there and no issues.
    3. Navigate to the Windows 10 folder and highlight the new RAW files you just copied from the camera.
    4. Drag those into Capture One - drag and drop - or even at the Import dialog. Drag & Drop.
    5. Do not let Capture One look for them or it bogs down.
    6. Capture One will create the Date for Recents and boom - those new photos are right there in the strip ready to go.

    You would think that that would somehow not be as good as the Capture One Import, choose folder, destination, etc., but it works just the same and I have yet to have an organizational issue or speed issue.  All keyword tags, searches, everything works.

    This is frustrating because I want it to work like Lightroom, which is to say, logical:  it knows the last file it imported, and simply can use the Import function and it sees only the new files - simple.  That is the way it should be but sorry to say I cannot get Capture One 20 to work that way.  ONLY with the drag & drop method, do I have no speed issues.  It works always ... I am not sure why this is the case, but everyone else's comments are very familiar, but I am not sure about a few things - like how you are importing or if at that one you have the issues.

    Nevertheless, there is something fundamentally wrong with this version of Capture One, just by these similar comments alone.  I doubt the Support Team at Capture One will ever read any of these posts.

    They need to dix the indexing of the catalog for the Import function.  I think that may be the link amongst us.

     

     

     

    1
  • Martin Cichon

    this update makes the app simple unusable!!!! thats it.... waited an h to load the session, it starts loading the root folder, going deeper in the structure.... it seems on ever tree depth to reload from root:) crazy!!! this is a fatal error!! not used to it on pro?? software.... and the last update was 45days ago... it had to take few days!! still no solution, even a catolg downgreader app for the 12v is not there.... sorry to say but:: the soft is a nice marketing addition to the hardware, nothing else... performance sluggish for too long time.... i guess i quit the subscription and some work with lightroom, or other app....

    0
  • BeO
    Top Commenter

    Session? Catalog downgrader app? What are you using, session or catalog? A session is for shoots, maybe 1k images as a ballpark. What is C1 doing for you in this hour or days you mention, opening a session you imported images into previously, or, importing images and creating previews? You seem to be a very impatient person when learning a new complex but gorgeous (for the most time) software but have enough time to spread your frustration into the world? God luck with LR or other software.

    edit: And I don't mean this last sentence sarcastically, C1 is definitely not for everyone, the learning curve is steeper and there are more pitfalls. LR or other software might be simpler and more hassle free.

    0
  • Jeff Leak

    Martin - I would give Capture One some more time ... it really is the better solution for serious Fujifilm users.  I didn't used to (want to) think it is better than Lightroom, but it certainly is.  If you are able to edit your images and navigate the software pretty well (practice makes perfect - shot anything then practice on those first) then I suggest that you try my solution posted above if you are using the Catalog system.  

    Like BeO said, Sessions is mostly for tethering a camera, which most of us are not doing.  And if we are coming from LR, then we are used to Catalogs.  The Import function does need some work - I'm not exactly sure what the deal is - it may be that the Import function is for bringing in photos directly from the camera or SD card.  I am using photos from the hard-drive directory, and you may be as well.  Everything works just perfectly with the drag-and-drop method.  That could possibly be the way they envisioned using the Catalog.  Just grab your photos from Windows and drop them anywhere in C1 ... the Import window will open with those selected photos.  It will create a point with the Date you brought them in (left Folder icon to see the structure).

    After using this drag-and-drop method, I am quite happy with the performance.  It is a little faster than the LR import I think ... this method is pretty fast.  I hope this helps.

    0
  • Thomas Tveit Rosenlund

    @Corey Surette solutions seems quite interesting. I am contemplating moving my catalog to my main drive in Windows (C:\) to see if there are any perfomance benefits to be had. In Lightroom I got better performance if the catalog was actually on a separate SSD, but using CO loading the whole catalog with a (seems like) rescanning av all images an loading metadata takes forever so I might have to try this trick. Have anyone else tried this and experienced the same performance benefit as @Corey?

    0
  • BeO
    Top Commenter

    Jeff,

    if you create sub folders on disk so that the no. of files in each folder is reasonable, the import dialog opens in reasonable time, just don't have "duplicates" checked. Having sub folders has also the advantage that you can easier locate an image e.g."show in library", which is sometimes needed for some C1 functionality which won't work if you have selected an image in a smart album. And having some at least minimal folder structure on disk might help with backups or if you want to use another editor / raw developer at a certain point in time. But your method seems to be working too.

    0
  • Jeff Leak

    @BeO thanks for the tip!  I noticed that after checking the "no duplicates" box it would re-index everything ... which seems that something isn't quite right - but I no longer have to worry with that using the drag-and-drop.  I have my photos on an internal Samsung 1TB SSD (I have four SSDs) and everything is in folders specific to a camera.  So the X-T30 has a folder and the XF10 has a folder ... inside of that are dates by year - then just about 2,000 photos inside the year.  I definitely have a folder structure and do the same for Sony, Nikon and Panasonic cameras.  I use SyncBack program to backup the entire structures to an external LaCIE 4TB drive.  I will look tomorrow at using the import button again and see how that works without checking the "duplicates" box.  Thanks for the reply!  Need more active people in these forums.

    0
  • BeO
    Top Commenter

    Jeff,

    You then have to manually select the images in the import dialog (similar effort as you would have in explorer/finder when using drag&drop). Not a problem for me since I have broken down my year folders into months.

    Alternatively you can also try to go to the library tool in C1, select the folder, context menu: Synchronize.

    I don't know how long it takes the dialog to pop up if you are having 2000 images in the folder though. The importer (if you choose this option) only shows you the new images then.

     

     

    0
  • BeO
    Top Commenter

    Thomas,

    The faster the drive (and the connection to it) where the catalog and the raw files are stored, obviously, the better. If space is a consideration, you can put the catalog on C: (should this be faster) and the raw files on external SSD.

    0
  • Jeff Leak

    BeO - thanks for the screen shot ... I don't think I have noticed mine looking quite like that.  Yes, I manually drag the files over into C1.  I have not come across the Synchronize option - I will definitely be looking for this tomorrow!  This might solve my indexing every-time I use the Import button.

    0
  • Permanently deleted user

    The gist of the issue is the cataloging function in the Windows version of Capture One is broken. I stopped using Catalogs a couple of months ago and am fairly happy with the result.

    Here is what I did,

    • Create a session on the fastest drive in your computer. I've got two SSD's my windows box, so that was easy.
    • Export your files, folder by folder using the export function. Make certain and set it to save all the metadata and such. I exported all of mine to a folder on my NAS box. It helps if you use metadata to create folder and file names, otherwise you will have to fat finger it... Sorry.
    • Once you have all your images exported, close your catalog and open your session.
    • The key here is don't import anything into the session. Browse to the folder desired. The first time you do this, C1 will create previews in the folder of the images of the folder.

    The good part about doing it this way, is you can backup all your previews and edits. This is major as I was starting to run into serious catalog instability issues and had to regress to a backup catalog two within a couple of weeks.

    The initial preview generation oftentimes takes a few minutes, but once done, it's done.

    Unfortunately C1 doesn't save much in XMP files, but Exiftool reads the C1 cop files, so I'm working on a standalone tool, which uses Exiftool to syncronize settings between cop files and XMP files. That way, I can still use LightRoom as a DAM

    0

Please sign in to leave a comment.