Catalog bugs and performance problems
I recently made a commitment to C1 and purchased it. I started putting new work into a catalog and left my LR catalog in place. I went on a trip and shot 1300 or so pics that went into about 9 sessions. Most of the work to cull and edit was done during the trip. The session folders were on an external SSD so it was easy to import images with adjustments into the new catalog back home. I'd used the location metadata field for the subject matter--easy to copy across all of the images in a folder. Using tokens, it was easy to use the import to generate a hierarchy of folders that fit the structure of the referenced image catalog. I thought I'd found it.
I decided to transfer the LR catalog to C1. Copied everything from the main machine internal drive to an external ssd. Relocating in LR--well I looked away from the screen for a second and didn't know it had even completed. I later had to do the same thing in C1: more than 1/2 of an hour. Did a little bit of clean up on the LR catalog: creating static albums out of smart albums and deleting a few unnecessary albums. I should have purged many keywords, but I didn't realize what a problem that would be. Now I have a nice LR catalog pointing at referenced images on an ssd.
I began to import the catalog into Capture One. I didn't expect perfect fidelity. Ratings and crops were enough--anything more is a bonus. At first, the process went really fast. Images were ticking off at a rate of 6-8 a second. It looked like it would take only around 3 hours. Well, then it started bogging down. Memory use of C1 is about 24G on an 8G machine. Of course, C1 doesn't get all 8 to itself. It probably only gets 4-6G physical memory. So that is 5 to 4 times overcommit of virtual memory. What the C1 code thinks it is doing in memory it is actually doing my swapping from disk to memory with millions, yes millions of page faults. And then there is the process of finding keywords in a newly imported image and verifying against all of the existing keywords. By the end the process was taking 2 seconds per image. In the end it took over 14 hours. I know; I did it twice.
Problems:
Some images result in 4 variants per image. Huh? I checked the LR catalog to see if I had virtual copies for those images? Yes. Turns out you can quickly filter for virtual copies in LR but you cannot filter for non-primary variants in C1. So, I had 1289 virtual copies (not counting the primary) in LR. These were tripled to be nearly 3900 in C1. Since there is no search, I had to visually scan for the multiple variants icon, select the non-primary variants, and delete them. Painful. This is obviously a bug.
Horrible bug in relocate. So, the reason I did this twice in C1 is because it didn't work the first time. What I thought I could do was just import the Lightroom catalog and then relocate to the copied folders on the SSD. Ha Ha. Now, C1 developers will not consider this a bug but it is. Folders in a C1 catalog are just another collection--there is nothing special about them. Seems clever, but folders are special: they are the physical organization. Here is the problem. Let's say you pick a year/month folder that contains say 6 sub-folders. You "locate..." the year/month folder. It is found and moves under its appropriate parent folder. But the 6 sub-folders DO NOT. This is not a recursive process! Never mind it costs me some work to go through and locate... all 1200 or so folders individually. The bug is that the referential integrity of the database has been violated--it's in an inconsistent state--a big NO-NO. A parent folder on physical volume A shows that it has children sub-folders on physical drive B. C1 Devs: tell me that this is a valid consequence of implementing folders as collections. Big bug. So, part way through I stopped. Relocated the ENTIRE LR catalog just via the single top level--the whole relocate was recursive. Then, I imported this LR catalog for a second pass.
Now I finally have a C1 folder with about 3900 too many variants. Let's poke around a bit and see if everything came over more or less ok. The good news is that basic metadata comes over fine: IPTC, exif, ratings, keywords. Cropping. Awesome. Exposure and other basic edits--not clear. I can deal. But, there is a problem: I want to look at what's in 2009 09 / foo. Oops, where is it. The folder hierarchy is in essentially random order. This is a bug.
Lets sort it. Sort by name. On the SSD it takes over 40 minutes. Seriously? There are 140 2nd level folders (one below the root). And roughly 8-12 subfolders each on average. Something like 1200 to 1680 folders. 40 minutes to sort? Seriously sorting a folder list to display in the UI is NOT a physical task. Nothing is moved on disk. No OS directory is being changed. It is just a list to aid command dispatch in the UI of the product. But, because folders are collections nearly every image's metadata has to be touched to update the collections the image is member of. Poor performance based on poor architectural decision.
here is the bug: sort it once. Do some viewing. Import something new. The display order in the library tab goes random again. 40 minutes to sort it. This is a just a bug.
Add new folders (Add new folder... command). The folders don't go into the sorted order. They go at the bottom. Bad design decision. Guys: sorting in the UI is a display trick; it's not physical. You are sorting a tiny amount of data in memory, putting it into your data structures for UI display, re-displaying part of the UI. Those locations in the library tab reference a data structure which does contain the actual folder (or collection) so that the command can trigger a query and then display the results. But, just generating a sorted list is trivial.
Deleting keywords is incredibly slow. I should have cleaned up in LR first. But, the problem here is that when a keyword is deleted, any use of it (reference by an image) has to be removed from said images. With proper way 2 way join tables this should be very fast. It could take over 10 minutes for the C1 UI to be active again. I did go back to LR and do the clean up just to see how long it takes and I deleted some of the messy keywords. Instant.
Sync'ing a folder containing images is not recursive. I can't tell whether it is or is not in LR. But, in C1 there is a checkbox for "known folders". Of course, the known folders might be out of order. What's clear is that checking the box does NOT sync the subfolders. The immediate subfolders do come in, but not their children or images. Call this a UI error. The label for the check box is a bit misleading. It would be nice if the checkbox made the command fully recursive.
Applying to new keywords to an image took 5 to 10 minutes.
Deselect all would leave the solid white outline around select images even though the deselect all command became grayed out (as if it had been done). This is a consequence of slowness--the UI was not being refreshed. Better to address the architectural causes of slowness than to fix the minor display bug.
Multiple selecting images (command click) can take over 5 seconds per image.
"Show in library" for an image does NOT show the folder when the starting point is the "all images" collection. Well, that is when you need this command--out of a sea of images you want to find where it is--if it is anywhere (it turns out an image can be removed from its folder collection and still remain in the all images collection--this is another referential integrity bug). It turns out that "show in finder" works and works as fast as you'd expect--not quite instant.
Show/hide folder hierarchy command occasionally became inoperable. Not clear why. Changing command tabs did not "clear" it. Quitting and restarting C1 did get it back to normal.
A few bugs, but the big problem is poor--desparately poor performance--cause by poor schema design of the database, poor catalog architecture, and nearly non-existent memory management. Thrashing to disk with overcommitted virtual memory use will kill any app. Virtual memory is not magic: it is writing over-committed memory to disk and then retrieving it later. When this is used for less frequent actions (context switching between apps; loading a new file, etc) it's handy and not too agonizing. But, when it happens for highly frequent activities then an app grinds to a near halt. This is what happens to C1. I am sure the Devs will say, just get more RAM. Sure. The listed system requirements are 4G. That is surely unacceptable. At 8G with a Core I7 3.2gz processor I would not expect grinding slowness. Indeed, LR does all of the equivalent database-related and ingestion operations MUCH faster (it does do some things more slowly than C1). I would certainly expect my 8G machine to be slower than a 24G machine. But, slower, not blocked.
These things happened. They are facts. Sorry to all of the apologists. There are problems. I hope I proved my bonafides with my intro. I invested a lot in learning C1. I had a very successful travel shoot experience using sessions, granted with only 1300 images. Importing the session images (via import image, NOT import session) with adjustments and generating a nice automatic folder structure as part of an existing catalog was flawless. The quality of the raw conversions was excellent. The streamlined, very predictable steps to use editing commands to achieve desired image results was VERY PLEASING--better results and a better process than LR. Highlights: spot editing with so many operations enabled, especially curves. Highlights and shadows controls work marvelously. Many people love color editing--didn't need for the travel shoot but I've used subsequently. Great. What is special about Capture One is that as manufacturers (well, designers) of a medium format digital back, these guys understand how a scene and its lighting stimulate an image sensor, generate electrical measurements, which are then interpreted and processed into an image. Since that rendering process involves converting direct physical measurements from the sensor into image data, they also really understand how to ALTER that process with intentional edits--and produce impressive results. I respect this and enjoy it. This is what captivates users (I think) about Capture One. Unfortunately, the tawdry details of software engineering--memory management, data structures, algorithms, coding decisions seem to be the step child of development. So, we are left with the impressive image processing capabilities embedded in a poorly performing outer package.
In the race to compete with Adobe and the opportunity to acquire disgruntled former Aperture users perhaps there has been some rampant "featuritis". It's all understandable. I'd like to suggest perhaps some breathing room for software quality. The product is sufficiently competitive with LR. I don't see many people begging for website generation or auto-printbook generation. (Nothing wrong with these things; sure some people like this stuff, but they are infrequently used and unlikely to be competitively essential.) A moratorium on new features for 12 to 18 months would be beneficial. Focus only on new cameras (gotta do it), bugs, and architectural retooling for reliability and performance. The latter is difficult and risky work. Admitting the need for this would not be a marketing negative. It would earn trust and gain more users. The 12 to 18 months should not be done in one big bite. Better to have several releases with some work requiring a long effort and other work that can be released incrementally.
I am going to try to figure out how to get C1 working with Photo Supreme. There are some problems: crops don't show up. The process is quick, but less than ideal. I want to get to a global catalog solution. Not everyone agrees. I love using sessions for the initial pass of the work. It is reasonable for me to not have a single catalog--global really just means big--say all the images of family over the years. Other catalogs can cover other subject matters. I do not what all my images split across 200 or more sessions--I want searching across a larger span to work well. This is what I mean by "global": pretty big, which ought to mean 50,000 to 100,000 images per catalog with good performance, no hangs, no multi-minute UI hangs. I think it's possible.
I decided to transfer the LR catalog to C1. Copied everything from the main machine internal drive to an external ssd. Relocating in LR--well I looked away from the screen for a second and didn't know it had even completed. I later had to do the same thing in C1: more than 1/2 of an hour. Did a little bit of clean up on the LR catalog: creating static albums out of smart albums and deleting a few unnecessary albums. I should have purged many keywords, but I didn't realize what a problem that would be. Now I have a nice LR catalog pointing at referenced images on an ssd.
I began to import the catalog into Capture One. I didn't expect perfect fidelity. Ratings and crops were enough--anything more is a bonus. At first, the process went really fast. Images were ticking off at a rate of 6-8 a second. It looked like it would take only around 3 hours. Well, then it started bogging down. Memory use of C1 is about 24G on an 8G machine. Of course, C1 doesn't get all 8 to itself. It probably only gets 4-6G physical memory. So that is 5 to 4 times overcommit of virtual memory. What the C1 code thinks it is doing in memory it is actually doing my swapping from disk to memory with millions, yes millions of page faults. And then there is the process of finding keywords in a newly imported image and verifying against all of the existing keywords. By the end the process was taking 2 seconds per image. In the end it took over 14 hours. I know; I did it twice.
Problems:
Some images result in 4 variants per image. Huh? I checked the LR catalog to see if I had virtual copies for those images? Yes. Turns out you can quickly filter for virtual copies in LR but you cannot filter for non-primary variants in C1. So, I had 1289 virtual copies (not counting the primary) in LR. These were tripled to be nearly 3900 in C1. Since there is no search, I had to visually scan for the multiple variants icon, select the non-primary variants, and delete them. Painful. This is obviously a bug.
Horrible bug in relocate. So, the reason I did this twice in C1 is because it didn't work the first time. What I thought I could do was just import the Lightroom catalog and then relocate to the copied folders on the SSD. Ha Ha. Now, C1 developers will not consider this a bug but it is. Folders in a C1 catalog are just another collection--there is nothing special about them. Seems clever, but folders are special: they are the physical organization. Here is the problem. Let's say you pick a year/month folder that contains say 6 sub-folders. You "locate..." the year/month folder. It is found and moves under its appropriate parent folder. But the 6 sub-folders DO NOT. This is not a recursive process! Never mind it costs me some work to go through and locate... all 1200 or so folders individually. The bug is that the referential integrity of the database has been violated--it's in an inconsistent state--a big NO-NO. A parent folder on physical volume A shows that it has children sub-folders on physical drive B. C1 Devs: tell me that this is a valid consequence of implementing folders as collections. Big bug. So, part way through I stopped. Relocated the ENTIRE LR catalog just via the single top level--the whole relocate was recursive. Then, I imported this LR catalog for a second pass.
Now I finally have a C1 folder with about 3900 too many variants. Let's poke around a bit and see if everything came over more or less ok. The good news is that basic metadata comes over fine: IPTC, exif, ratings, keywords. Cropping. Awesome. Exposure and other basic edits--not clear. I can deal. But, there is a problem: I want to look at what's in 2009 09 / foo. Oops, where is it. The folder hierarchy is in essentially random order. This is a bug.
Lets sort it. Sort by name. On the SSD it takes over 40 minutes. Seriously? There are 140 2nd level folders (one below the root). And roughly 8-12 subfolders each on average. Something like 1200 to 1680 folders. 40 minutes to sort? Seriously sorting a folder list to display in the UI is NOT a physical task. Nothing is moved on disk. No OS directory is being changed. It is just a list to aid command dispatch in the UI of the product. But, because folders are collections nearly every image's metadata has to be touched to update the collections the image is member of. Poor performance based on poor architectural decision.
here is the bug: sort it once. Do some viewing. Import something new. The display order in the library tab goes random again. 40 minutes to sort it. This is a just a bug.
Add new folders (Add new folder... command). The folders don't go into the sorted order. They go at the bottom. Bad design decision. Guys: sorting in the UI is a display trick; it's not physical. You are sorting a tiny amount of data in memory, putting it into your data structures for UI display, re-displaying part of the UI. Those locations in the library tab reference a data structure which does contain the actual folder (or collection) so that the command can trigger a query and then display the results. But, just generating a sorted list is trivial.
Deleting keywords is incredibly slow. I should have cleaned up in LR first. But, the problem here is that when a keyword is deleted, any use of it (reference by an image) has to be removed from said images. With proper way 2 way join tables this should be very fast. It could take over 10 minutes for the C1 UI to be active again. I did go back to LR and do the clean up just to see how long it takes and I deleted some of the messy keywords. Instant.
Sync'ing a folder containing images is not recursive. I can't tell whether it is or is not in LR. But, in C1 there is a checkbox for "known folders". Of course, the known folders might be out of order. What's clear is that checking the box does NOT sync the subfolders. The immediate subfolders do come in, but not their children or images. Call this a UI error. The label for the check box is a bit misleading. It would be nice if the checkbox made the command fully recursive.
Applying to new keywords to an image took 5 to 10 minutes.
Deselect all would leave the solid white outline around select images even though the deselect all command became grayed out (as if it had been done). This is a consequence of slowness--the UI was not being refreshed. Better to address the architectural causes of slowness than to fix the minor display bug.
Multiple selecting images (command click) can take over 5 seconds per image.
"Show in library" for an image does NOT show the folder when the starting point is the "all images" collection. Well, that is when you need this command--out of a sea of images you want to find where it is--if it is anywhere (it turns out an image can be removed from its folder collection and still remain in the all images collection--this is another referential integrity bug). It turns out that "show in finder" works and works as fast as you'd expect--not quite instant.
Show/hide folder hierarchy command occasionally became inoperable. Not clear why. Changing command tabs did not "clear" it. Quitting and restarting C1 did get it back to normal.
A few bugs, but the big problem is poor--desparately poor performance--cause by poor schema design of the database, poor catalog architecture, and nearly non-existent memory management. Thrashing to disk with overcommitted virtual memory use will kill any app. Virtual memory is not magic: it is writing over-committed memory to disk and then retrieving it later. When this is used for less frequent actions (context switching between apps; loading a new file, etc) it's handy and not too agonizing. But, when it happens for highly frequent activities then an app grinds to a near halt. This is what happens to C1. I am sure the Devs will say, just get more RAM. Sure. The listed system requirements are 4G. That is surely unacceptable. At 8G with a Core I7 3.2gz processor I would not expect grinding slowness. Indeed, LR does all of the equivalent database-related and ingestion operations MUCH faster (it does do some things more slowly than C1). I would certainly expect my 8G machine to be slower than a 24G machine. But, slower, not blocked.
These things happened. They are facts. Sorry to all of the apologists. There are problems. I hope I proved my bonafides with my intro. I invested a lot in learning C1. I had a very successful travel shoot experience using sessions, granted with only 1300 images. Importing the session images (via import image, NOT import session) with adjustments and generating a nice automatic folder structure as part of an existing catalog was flawless. The quality of the raw conversions was excellent. The streamlined, very predictable steps to use editing commands to achieve desired image results was VERY PLEASING--better results and a better process than LR. Highlights: spot editing with so many operations enabled, especially curves. Highlights and shadows controls work marvelously. Many people love color editing--didn't need for the travel shoot but I've used subsequently. Great. What is special about Capture One is that as manufacturers (well, designers) of a medium format digital back, these guys understand how a scene and its lighting stimulate an image sensor, generate electrical measurements, which are then interpreted and processed into an image. Since that rendering process involves converting direct physical measurements from the sensor into image data, they also really understand how to ALTER that process with intentional edits--and produce impressive results. I respect this and enjoy it. This is what captivates users (I think) about Capture One. Unfortunately, the tawdry details of software engineering--memory management, data structures, algorithms, coding decisions seem to be the step child of development. So, we are left with the impressive image processing capabilities embedded in a poorly performing outer package.
In the race to compete with Adobe and the opportunity to acquire disgruntled former Aperture users perhaps there has been some rampant "featuritis". It's all understandable. I'd like to suggest perhaps some breathing room for software quality. The product is sufficiently competitive with LR. I don't see many people begging for website generation or auto-printbook generation. (Nothing wrong with these things; sure some people like this stuff, but they are infrequently used and unlikely to be competitively essential.) A moratorium on new features for 12 to 18 months would be beneficial. Focus only on new cameras (gotta do it), bugs, and architectural retooling for reliability and performance. The latter is difficult and risky work. Admitting the need for this would not be a marketing negative. It would earn trust and gain more users. The 12 to 18 months should not be done in one big bite. Better to have several releases with some work requiring a long effort and other work that can be released incrementally.
I am going to try to figure out how to get C1 working with Photo Supreme. There are some problems: crops don't show up. The process is quick, but less than ideal. I want to get to a global catalog solution. Not everyone agrees. I love using sessions for the initial pass of the work. It is reasonable for me to not have a single catalog--global really just means big--say all the images of family over the years. Other catalogs can cover other subject matters. I do not what all my images split across 200 or more sessions--I want searching across a larger span to work well. This is what I mean by "global": pretty big, which ought to mean 50,000 to 100,000 images per catalog with good performance, no hangs, no multi-minute UI hangs. I think it's possible.
0
-
I have been through a similar journey with COP 8, I spent countless scarce hours debuggings SW when I should have been shooting, editting or sorting.
I'm watching this forum to see if COP 9 becomes a performance/quality improvement. Until then I'm not buying.0 -
You have captured most of the current BIG issues with C1P. The catalog is a mess and although the random sort order and slow / non-persistent reordering issue is a known bug (which P1 are reportedly working on) the other issues are due to a very poorly implemented catalog system which really needs redesigning from the ground up. Don't ever loose sight that this is a 'premium' piece of software - it is pretty expensive (at least, it is to me) - I just don't think we're getting value for our money at the moment and it often feels like we're part of a beta test...
What you won't see is any apologies or commitment to deal with the problems from P1...0 -
I could not agree more. I complained about the completely random display of the subfolder structure already in version 8. And the performance of catalogue activities with large catalogues is just abysmal. I mentioned this in another blog entry: A few days ago I tried to import 16 images on a camera SD into my main catalogue - it took more than 14 minutes! C1 obviously is incapable to deal efficiently with large catalogues. This requires a major overhaul of the cataloguing software. 0 -
[quote="Martin24" wrote:
I could not agree more. I complained about the completely random display of the subfolder structure already in version 8.
What do you mean? My folder structure in C1 looks fine, the same as in Lightroom.
1 - Raw Files
------2010
------2012
------2013
------2014
------2015
------2016
------------2016-01
----------------------2016-01-01
----------------------2016-01-02
----------------------2016-01-03
2 - Derivatives
And yes, I put the 1 and 2 to order the folders in that particular order, even in Lr.0 -
[quote="lewis" wrote:
[snip]
Amen to that!
Please, PhaseOne, you're image processing is top notch. Now get the rest of the program on par!
Cheers,
Peter.0 -
I think I am going back to lightroom. I don't have time for this stuff. I just bought C1P and already having issues. I need to be shooting, not trying to figure out complicated buggy software. 0 -
Wholeheartedly agree with lewisl's post. Doesn't seem like anyone's unsatisfied with the image quality C1 produces but the DAM part is abysmal. I would be in favor of stopping feature development in exchange for fixing what's really broken. 0 -
[quote="dredlew" wrote:
Wholeheartedly agree with lewisl's post. Doesn't seem like anyone's unsatisfied with the image quality C1 produces but the DAM part is abysmal. I would be in favor of stopping feature development in exchange for fixing what's really broken.
+10000 -
[quote="Martin24" wrote:
A few days ago I tried to import 16 images on a camera SD into my main catalogue - it took more than 14 minutes! C1 obviously is incapable to deal efficiently with large catalogues.
An import of 16 images should not take 14 minutes. I would say on my system this would take about 5 seconds. Something is wrong on your machine.0 -
[quote="EnderWiggins" wrote:
[quote="Martin24" wrote:
A few days ago I tried to import 16 images on a camera SD into my main catalogue - it took more than 14 minutes! C1 obviously is incapable to deal efficiently with large catalogues.
An import of 16 images should not take 14 minutes. I would say on my system this would take about 5 seconds. Something is wrong on your machine.
5 seconds for 16 images? Wow, you must have some pretty beefy hardware and minuscule images... 😊
For sake of comparison, could you tell us a bit more? It seems this is not the only report of slow imports. Maybe we can get some ideas of what works fast and slow?
To all interested: the next time you import images, time it (and don't start working on them until the machine is idle again). Then report the following.
- computer/RAM
- the number of image imported
- the average size of an images,
- image type (RAW/JPG, camera brand/type)
- the duration of import (from clicking the import button until the last image is imported, but not waiting for all image previews to be ready)
- the duration until all previews are ready (starting when you click the import button until nothing more happens)
- the card reader and port (USB 2 or 3, thunderbolt, ...),
- the type and connection of the disk you're storing these images on,
- catalog or session
- the number of images in the database
- the type and disk holding the catalog/session database
- preset being applied during the import
- other?
I have a card waiting to be imported and I'll report back as soon as it's finished.
Cheers,
Peter.0 -
OK, I'm shutting up. And I'm glad that I can say that!
- computer/RAM
Mac Pro 15" mid 2010, 8GB RAM, NVIDIA GeForce GT 330M 512MB
- the number of image imported
316
- the average size of an images,
33MB
- image type (RAW/JPG, camera brand/type)
RAW Fuji X-T1
- the duration of import (from clicking the import button until the last image is imported, but not waiting for all image previews to be ready)
17 minutes (estimate was also 17, well done!)
- the duration until all previews are ready (starting when you click the import button until nothing more happens)
26 minutes
- the card reader and port (USB 2 or 3, thunderbolt, ...),
Hama, USB2
- the type and connection of the disk you're storing these images on,
WD Elements, USB2
- catalog or session
Catalog
- the number of images in the database
2115
- the type and disk holding the catalog/session database
SSD, internal
- preset being applied during the import
None, no auto-adjust, no description, only copyright
- other
Activity viewer showed "auto-loaded sets metadata" which took as long as building the previews.
So, average time to import 1 image: 3.2 seconds (call me impressed)
Average time including building a preview: 4.9 seconds.
Now back to the job!
Cheers,
Peter.0 -
Ok, maybe I was a little too optimistic 😲 , but here's what I just timed:
4 RAW files imported in roughly 8-9 seconds. Previews were available immediately, meaning after the thumbnail was in my catalog, I could doubleclick on it and had a 100% image without any delay.
Mac Pro 2010, 6-core Westmere, 16 Gb RAM, ATI Radeon 5870HD
RAW files by Canon 7D, between 25 and 27 Mb each
USB 3.0 card reader by Hama (3.0 USB interface via PCI slot)
CF Card, Sandisk 32 Gb Extreme Pro
RAW's stored on internal SATA drive
Catalog stored on internal SSD drive, currently holds 30.000 images
No preset, no auto adjust, no backup, no naming, recent imports only0 -
The 14 minutes to import 16 images has nothing to do with my equipment (top-of the line MacBook Pro, 16GB memory, 1TB SSD etc.). It has to do with the catalogue. The catalogue I wanted to import to, contains about 20k images. When I import the same images into a new, or small C1 catalogue, the import is decently fast, around 10seconds. But should not C1 operate well catalogues of more than 20k images? 0 -
[quote="Martin24" wrote:
The catalogue I wanted to import to, contains about 20k images. When I import the same images into a new, or small C1 catalogue, the import is decently fast, around 10seconds. But should not C1 operate well catalogues of more than 20k images?
Yes, and it does, as I just proved with my posting above. I have 30.000 images in my catalog and no issues with importing. So it's a problem with your catalog.
Have you checked the database integrity using the "File>Verify Catalog or Session" function?
Are the import settings the same if you import into a new catalog (presets, auto adjust etc.)?0 -
yes, I did do "verify catalogue" - was fast and did not report any errors. But just loading the catalogue with 22k images takes 1.5 minutes - is this normal?
I guess I have to split the catalogue now into chunks and start from scratch.0 -
[quote="Martin24" wrote:
But just loading the catalogue with 22k images takes 1.5 minutes - is this normal?
Absolutely not.
I would indeed look into setting up a new one by copying the projects/folders piece by piece into a new blank catalog file and check the performance after every step.
Are your RAW files managed or referenced in the catalog?0 -
Could you please share the underlying folder structure for your catalogues? 0 -
[quote="EnderWiggins" wrote:
I would indeed look into setting up a new one by copying the projects/folders piece by piece into a new blank catalog file and check the performance after every step.
Are your RAW files managed or referenced in the catalog?[quote="NN259560UL" wrote:
Could you please share the underlying folder structure for your catalogues?
I only use referenced images. The slow catalogue contains my former Aperture library (imported as Aperture library), where the images reside on an external disk (in year/month/day folder-subfolder structure), and it contains a second folder structure (year/month/day/images) with images that I imported directly into C1 after the switch from Aperture to C1 (about one year's worth of images). The latter reside on my main SSD still.
Could there be a problem in that my main SSD is encrypted? I use Apple's FileVault. I have not noticed any performance loss in other programs, though.0 -
[quote="Martin24" wrote:
where the images reside on an external disk (in year/month/day folder-subfolder structure)
This is the same folder structure as I have, my catalogue contains only 3,000 images and I have not fully transition my Lr catalogue into C1 yet.[quote="Martin24" wrote:
I only use referenced images.
What do you mean when you say "referenced"? Is this Aperture speech? My understanding, and don't mind somebody correcting me if I am wrong, is that in C1, like in Lr, all catalogues are referenced, insofar as they contain a link, a reference, to the hard drive location of the file.[quote="Martin24" wrote:
Could there be a problem in that my main SSD is encrypted? I use Apple's FileVault. I have not noticed any performance loss in other programs, though.
Encryption is going to slow things down for 2 reasons:
1) the file needs to be decrypted each time a program access the file
2) the catalogue needs to be encrypted each time and adjustment is made or metadata is altered
How big are the files?
Is this a common thread among people experiencing performance problem?0 -
[quote="NN259560UL" wrote:
[quote="Martin24" wrote:
I only use referenced images.
What do you mean when you say "referenced"? Is this Aperture speech? My understanding, and don't mind somebody correcting me if I am wrong, is that in C1, like in Lr, all catalogues are referenced, insofar as they contain a link, a reference, to the hard drive location of the file.
With "referenced" I mean in Aperture language that the images are not inside the catalogue.[quote="NN259560UL" wrote:
[quote="Martin24" wrote:
Could there be a problem in that my main SSD is encrypted? I use Apple's FileVault. I have not noticed any performance loss in other programs, though.
Encryption is going to slow things down for 2 reasons:
1) the file needs to be decrypted each time a program access the file
2) the catalogue needs to be encrypted each time and adjustment is made or metadata is altered
How big are the files?
Is this a common thread among people experiencing performance problem?
The images imported from aperture are about 255GB an external SSD disk. The images directly imported into C1 are about 40GB (these are the ones on the encrypted disk). It is indeed possible that the encryption slows the performance down. However, I have not seen any performance hits in any of my other programs. Would be curious if this is indeed an issue and if others have a similar experience.0 -
[quote="Martin24" wrote:
With "referenced" I mean in Aperture language that the images are not inside the catalogue.
My understanding is that in C1, like in Lr, all images in a catalogue are referenced, the concept of referenced images does not exist in C1 or Lr, all images are referenced.[quote="Martin24" wrote:
The images imported from aperture are about 255GB an external SSD disk. The images directly imported into C1 are about 40GB (these are the ones on the encrypted disk). It is indeed possible that the encryption slows the performance down. However, I have not seen any performance hits in any of my other programs. Would be curious if this is indeed an issue and if others have a similar experience.
I am assuming that you do not mean that each file is 40GB; so just to re-phrase the question, roughly what is the average size of each file?0 -
[quote="NN259560UL" wrote:
[quote="Martin24" wrote:
With "referenced" I mean in Aperture language that the images are not inside the catalogue.
My understanding is that in C1, like in Lr, all images in a catalogue are referenced, the concept of referenced images does not exist in C1 or Lr, all images are referenced.
In C1 you can use "referenced" (outside catalog) or "managed" (inside catalog) images.0 -
[quote="NN259560UL" wrote:
I am assuming that you do not mean that each file is 40GB; so just to re-phrase the question, roughly what is the average size of each file?
Most of them are raw files with about 20 MB each.0 -
[quote="Martin24" wrote:
[quote="NN259560UL" wrote:
I am assuming that you do not mean that each file is 40GB; so just to re-phrase the question, roughly what is the average size of each file?
Most of them are raw files with about 20 MB each.
They are not massive. But, personally, I would not have my catalogue and my RAW images in an encrypted drive. All the catalogue files need to be decrypted for use and encrypted to commit changes to disk, all the RAW files need to be decrypted for rendering and processing. I am sure C1 has no control over the encryption and decryption process, this is likely to be a task handled by the operating system.
I would be interesting to know if the same performance problems occur with an unencrypted disk and if a good portion of those reporting performance issues do also have an encrypted drive.0 -
[quote="NN259560UL" wrote:
They are not massive. But, personally, I would not have my catalogue and my RAW images in an encrypted drive. All the catalogue files need to be decrypted for use and encrypted to commit changes to disk, all the RAW files need to be decrypted for rendering and processing. I am sure C1 has no control over the encryption and decryption process, this is likely to be a task handled by the operating system.
I would be interesting to know if the same performance problems occur with an unencrypted disk and if a good portion of those reporting performance issues do also have an encrypted drive.
I managed now to extract from my main catalogue the recently directly imported images and made a new clean catalogue with them - images residing on an external disk, new catalogue on my main encrypted notebook disk. This new catalogue works perfectly and fast; but true, it has only 1500 images, and these sit now on a non-encrypted external disk. However, I played previously with some smaller catalogues for testing purposes by creating them on my encrypted main disk (including the images), and had no obvious issues. I still think that the main culprit for the bad performance of my main catalogue are the size and internal structure that was generated by the import from Aperture.0 -
One question that was often asked a year or so back but rarely now ....
Do people run any admin on their Aperture catalogs to tidy things up before passing them to the C1 import routine to convert to C1?
It might be interesting to work out whether or not that makes a significant difference to operational speed. I think the earlier questions were raised about a general inability to load the catalogs at all but identifying whether there might also be an influence on performance matters could be useful.
Just thinking out loud as an external observer.
Grant0 -
What do you mean by "run any admin" ? 0 -
[quote="John Doe" wrote:
What do you mean by "run any admin" ?
As I recall from the discussions last year Aperture has some tools to "tidy up" the catalog contents by way of "housekeeping" to help maintain performance and efficiency and, presumably, have some effect on size.
Sending a newly "cleaned" file with all internal tables and so on freshly tidied up would make for a better (and faster) conversion. Maybe also a better post conversion performance?
Grant0 -
That's actually a good point by Grant. My 30.000 image catalog also consists of about 27.000 which came from Aperture using the Importer tool and I have no issues.
But I did two things before I ran the Importer:
a) clean up the Library in Aperture by repairing permissions and the database structure. You can do this by holding down OPT+CMD when clicking on the Aperture icon.
b) import my Aperture library not all at once, but piece by piece. I exported every major folder out of Aperture into a new Mini-Library (File>Export>Items as new Library), which I then imported into C1. Doing it all at once led to massive problems, the Importer seems to be really having trouble with that. My Mini-Libraries were just 2.000-5.000 images each.
Don't trust C1's Aperture Importer too much, it may be the cause for many problems people have.0 -
[quote="EnderWiggins" wrote:
That's actually a good point by Grant. My 30.000 image catalog also consists of about 27.000 which came from Aperture using the Importer tool and I have no issues.
But I did two things before I ran the Importer:
a) clean up the Library in Aperture by repairing permissions and the database structure. You can do this by holding down OPT+CMD when clicking on the Aperture icon.
b) import my Aperture library not all at once, but piece by piece. I exported every major folder out of Aperture into a new Mini-Library (File>Export>Items as new Library), which I then imported into C1. Doing it all at once led to massive problems, the Importer seems to be really having trouble with that. My Mini-Libraries were just 2.000-5.000 images each.
Don't trust C1's Aperture Importer too much, it may be the cause for many problems people have.
Excellent - thanks for this tip! I'll try this.0
Post is closed for comments.
Comments
49 comments