Performance
Is there a recommended setup to get C1 to perform well with decent sized catalogs? I still mainly use Aperture, but C1 for finishing some client work because I like the RAW rendering better. I've tried several times over several years to switch to C1 completely for simplicity's sake, but it's just way too slow and beach balls are everywhere. Recently I imported just one Aperture library with about 30k pictures. I moved the originals inside C1 catalog, and all of this was on a very very fast multi-SSD RAID. It's a 2016 MBP with plenty of memory and whatever the top GPU was at the time. It's still basically unusable for searching/filtering.
Is there any way to deal with large catalogs that I missed in my searches? Is this something the C1 team cares about, or is it designed to be low volume software and I'm just never going to be able to use it how I need to? I don't need it to be Photo Mechanic fast, but it's still vastly slower than Aperture which has been dead for 5 years...
Is there any way to deal with large catalogs that I missed in my searches? Is this something the C1 team cares about, or is it designed to be low volume software and I'm just never going to be able to use it how I need to? I don't need it to be Photo Mechanic fast, but it's still vastly slower than Aperture which has been dead for 5 years...
0
-
Hi Matt,
I have a MBP Retina 15" late 2013, and I get an answer from a filter request in about 3 to 5 seconds on all my 22k images.
Cheers,
Robert0 -
It seems you catalog is on an external system.
An alternative could be to have a catalog with referenced images (=stored outside catalog). Place you catalog on internal drive, and images on external drive.0 -
[quote="Paul_Steunebrink" wrote:
It seems you catalog is on an external system.
An alternative could be to have a catalog with referenced images (=stored outside catalog). Place you catalog on internal drive, and images on external drive.
I tried this first.0 -
I have found that complex Metada in a large collection of Images can slow the Filter tool by a huge amount.
There was time when I was storing the original file name for each image in one of the IPTC Metadata fields. This results in a different value for every image file.
Removing Filter Tool from the Tool tab, or removing the Metadata, resulted in a significant speedup.
Filter tool indexes every Metadata Field, those that are shown, and those that are not. Indexing more than 10000 images, with a unique value for each images is not only of no value to the user, but takes a very long time. The number of string comparison operations theoretically grows as N*log(N), where N is the number of images. For 10000 images this is theoretically 6.7x as long as for 1000 images, and for 30,000 images 22x as long.
If the developer takes a straightforward approach, then the number of comparison operations grows as N squared (10,000 is 100X worse than 1000). For practical SW faced with data storage and CPU utilisation limits, the scaling is probably somewhere between these two values.
I observed that when Filter tool is on your active (visible) tool tab, it performs this indexing action just about every time you click a key or make a mouse click. On my newer iMac it was a 20s long beachball everytime I made a mouseclick in my All Images Collection of 16000 images. On the older iMac it was over a minute. That was pretty much unuseable and I came close to dumping Capture One at that point.
I ended up making a custom tool tab just for Filter Tool (and removed it from the Library Tool Tab), and this helped a lot. Since then I have decided that I don't need that Metadata, and scrubbed it.0 -
[quote="Eric Nepean" wrote:
I have found that complex Metada in a large collection of Images can slow the Filter tool by a huge amount.
There was time when I was storing the original file name for each image in one of the IPTC Metadata fields. This results in a different value for every image file.
Removing Filter Tool from the Tool tab, or removing the Metadata, resulted in a significant speedup.
Filter tool indexes every Metadata Field, those that are shown, and those that are not. Indexing more than 10000 images, with a unique value for each images is not only of no value to the user, but takes a very long time. The number of string comparison operations theoretically grows as N*log(N), where N is the number of images. For 10000 images this is theoretically 6.7x as long as for 1000 images, and for 30,000 images 22x as long.
If the developer takes a straightforward approach, then the number of comparison operations grows as N squared (10,000 is 100X worse than 1000). For practical SW faced with data storage and CPU utilisation limits, the scaling is probably somewhere between these two values.
I observed that when Filter tool is on your active (visible) tool tab, it performs this indexing action just about every time you click a key or make a mouse click. On my newer iMac it was a 20s long beachball everytime I made a mouseclick in my All Images Collection of 16000 images. On the older iMac it was over a minute. That was pretty much unuseable and I came close to dumping Capture One at that point.
I ended up making a custom tool tab just for Filter Tool (and removed it from the Library Tool Tab), and this helped a lot. Since then I have decided that I don't need that Metadata, and scrubbed it.
This is a very helpful answer, I appreciate it. Unfortunately I have very extensive metadata, and getting rid of it would defeat the purpose (actually being able to search and export the relevant pictures) of a DAM.
I'd be embarrassed if I made something that was more limited (in any way) than five-year-old dead software.0 -
[quote="mattcohen" wrote:
I'd be embarrassed if I made something that was more limited (in any way) than five-year-old dead software.
I get your point but is that not what Apple did when replacing its now dead software?
If the mighty Apple can do that to its users why on earth would you expect comparatively little Phase One to fill the gap left by Apple's development that used Apple's own development tools and, presumably, was optimised for them?
And if they did, would they have to develop on a completely different set of products for the Windows market?
Grant0 -
[quote="SFA" wrote:
[quote="mattcohen" wrote:
I'd be embarrassed if I made something that was more limited (in any way) than five-year-old dead software.
I get your point but is that not what Apple did when replacing its now dead software?
If the mighty Apple can do that to its users why on earth would you expect comparatively little Phase One to fill the gap left by Apple's development that used Apple's own development tools and, presumably, was optimised for them?
And if they did, would they have to develop on a completely different set of products for the Windows market?
Grant
Maybe you misunderstood, maybe you didn't. Today I can search through an Aperture library of 120,000 pictures with multiple keywords and full IPTC with zero lag. With the same setup, C1 chokes on 30,000, I wouldn't be surprised if it choked on 10,000. Aperture hasn't been developed really since 2013, and C1 has had ~5 versions since. Yes, I absolutely expect that problem to have been solved completely in the intervening years. This is basic DAM functionality.0 -
FileMaker Pro—which is also an Apple product (FM is a wholly owned Apple subsidiary)—is also almost instantaneous with large databases (it's expressly a database program after all). It's been Mac and Windows for something like 20-25 years, since version 2 (they're now on v17). I suspect their internal approach (as well as Aperture's) differs significantly from C1. If I remember correctly, FMP maintains internal indices that track occurrences of words with pointers the records that contain them, so when one searches for items the database only has to parse the appropriate index(es) rather than every field in each record. OTOH, I get the feeling that C1 doesn't do this sort of thing, instead parsing each field in each "record" in the current collection. That means that whereas FMP (and, I assume Aperture) only have to parse relatively small tables, C1 is left parsing the entire metadata table—a MUCH more laborious process as pointed out above.
I saw the tip about moving the Filter tool to its own tab several versions ago (v9 or 10). That sped navigating among images up hugely.
When searching for images, C1's filters start working as soon as you start typing. The "found results" are narrowed with each successive typed character. Generally I only need to type the first 3-4 letters in a word I'm looking for and C1's got a reasonable list. The bad news is that often the list of found images includes a lot that make me scratch my head in wonder—wondering why in the world they showed up in the found items. C1's usually accurate for small numbers of found items in a large collection, but for me falls down when there are a lot of found items.
For example (fictional examples for illustration only), I photograph a lot of birds in the wild. When I search for "description contains bald eagle", C1 will find bald eagles AND others that have descriptions like "golden eagle"—and some that have neither "bald" nor "eagle" in the Description field. Or "description contains great blue heron", C1 will return the desired images PLUS birds that have descriptions like "house finch", "egret"—and don't contain the words "great", "blue", or "heron" at all. I have no idea why this happens, but it's something I live with.
I wish there was a way to tell C1 not to start filtering until the user hits the Return key or something like that. But there isn't. Developing databases is hard work and probably pretty labor intensive (and maybe boring). I can understand why Phase One wants to concentrate on features related to processing, but it sure would be nice if they'd give a bit more love to the DAM aspect.0 -
[quote="Nature Isme" wrote:
When searching for images, C1's filters start working as soon as you start typing. The "found results" are narrowed with each successive typed character. Generally I only need to type the first 3-4 letters in a word I'm looking for and C1's got a reasonable list. The bad news is that often the list of found images includes a lot that make me scratch my head in wonder—wondering why in the world they showed up in the found items. C1's usually accurate for small numbers of found items in a large collection, but for me falls down when there are a lot of found items.
For example (fictional examples for illustration only), I photograph a lot of birds in the wild. When I search for "description contains bald eagle", C1 will find bald eagles AND others that have descriptions like "golden eagle"—and some that have neither "bald" nor "eagle" in the Description field. Or "description contains great blue heron", C1 will return the desired images PLUS birds that have descriptions like "house finch", "egret"—and don't contain the words "great", "blue", or "heron" at all. I have no idea why this happens, but it's something I live with.
I have noticed this as well. And it's not something I can live with.0 -
[quote="mattcohen" wrote:
[quote="SFA" wrote:
[quote="mattcohen" wrote:
I'd be embarrassed if I made something that was more limited (in any way) than five-year-old dead software.
I get your point but is that not what Apple did when replacing its now dead software?
If the mighty Apple can do that to its users why on earth would you expect comparatively little Phase One to fill the gap left by Apple's development that used Apple's own development tools and, presumably, was optimised for them?
And if they did, would they have to develop on a completely different set of products for the Windows market?
Grant
Maybe you misunderstood, maybe you didn't. Today I can search through an Aperture library of 120,000 pictures with multiple keywords and full IPTC with zero lag. With the same setup, C1 chokes on 30,000, I wouldn't be surprised if it choked on 10,000. Aperture hasn't been developed really since 2013, and C1 has had ~5 versions since. Yes, I absolutely expect that problem to have been solved completely in the intervening years. This is basic DAM functionality.
If I understand things correctly Apple replaced Aperture with Photos for OS X.
How does Photos compare to Aperture?
Remember that until recently Phase also offered Media Pro.
So if Apple don't see a long term need for an Aperture type product (or don't wish to compete with other applications available) and Phase don't see a future for Media Pro ( a comprehensive DAM product in its own right as I understand it) then maybe the DAM market is either too small or too specialised or too well populated with free application or something to make it worth the development investment?
Affinity seems to have become quite popular as an alternative to PhotoShop. No DAM functionality that I can find in Affinity. Nor Keywords. Nor much additional metadata.
Lightroom forced the use of a catalogue (at version one - maybe still?) for whatever reasons they had at the time.
I didn't like the catalogue approach at the time.
What else does Adobe offer for cataloguing other than LightRoom?
Canon always provided separate products for RAW conversion and cataloguing. I didn't get into either of them very much.
Other products may well be trying to offer both catalogues and RAW conversion/editing in a single integrated application and UI. Can they do it without performance issues on one side of the application or the other?
The point about the Apple in house development is that they only had to create a product for Apple users, could make use of all of their internal knowledge of their self developed tools and totally integrate to the OS or other useful features from their suites of products in way that third party developers may never be able to do.
The same goes for MS and Windows.
In terms of business strategy they can also to exit a particular market at any time they choose to and both of them do frequently. Some Microsoft applications have extremely short lifespans an no realistic end of life portability.
So if a major developer does create a super performing product based on their own architecture you would expect it to be good. And marketable. To drop it rather suddenly suggests they don't really see a future for that market in that form - or at least not a market they are interested in .
Alternatives development tools and market options may be more compromised.
Grant0 -
Then again, Grant, if the DAM side of things is not really important, then why do so many people use Lightroom? I think we mostly agree on this forum that C1 offers superior RAW tools and conversions, so that can’t be it... And a lot of relative newcomers on the market are busy developing and integrating a DAM solution in their RAW development products.
Personally, the lack of a professional cataloguing feature and Phase completely ignoring complaints about it are my big frustrations with C1. I like the product enough to work around it, with a multitude of 5k to 7k image catalogs, but it’s 2019. They could do a lot better.0 -
searching large groups of pictures is a solved problem, i'm not asking for something exotic like compression that gives me negative file sizes. just search on one year's pictures (hopefully more) not leading to beachballs. many apps can do this with no lag. as mentioned, other photo apps are moving toward DAMs, Luminar most recently, Photo Mechanic and other to come. yet C1 updates and charges every single year and still provides DAM functionality that isn't adequate for any professional size catalog. they just re-skinned the whole thing, while functionality is sacrificed year after year. 0 -
Reading some comments on here from time to time there do seem to be some users who are happy with the performance.
But then reading comments about LightRoom (not specifically about catalogues) some find it really fast and some find it really slow.
It's difficult to know what to say.
As for LightRoom and catalogues - does it have an option?
Do so many people use it because it has a catalogue and they know they really want to use a catalogue and make full use of the search facilities ....?
Or because they have been told the LightRoom is the way to go and they find PhotoShop and its derivatives unfathomable for their needs?
I do wonder if we are in some sort of Catch 22 with databases.
Some business software I use is, for high volume users with very large data sets, somewhat compromised on performance for desktop use but, if installed with an MS Server configuration as its workhorse database, much faster.
Also much more expensive and requiring more significant attention to manage the systems on which that sort of application runs.
The developers tell me that, when fine tuning the settings for the system, they have to make a balanced choice between UI level responsiveness so that users do not perceive long waits to do anything on screen, and then handling large numbers of records when performing analysis instructions. They could make both extremely fast by tweaking the settings - but not at the same time.
The application will allow for manipulation and analysis of millions of records reasonably quickly but not extremely fast and the UI is very responsive so long as the working data set does not have more than about 30k - 40k records. That's the point at which the setup parameters have been tuned for on the basis that most users will tend to work interactively with smaller datasets. If they are running a larger task they expect it to take a while and plan their time accordingly.
Better still they persuade their employer that the automated options in the higher versions of the product - running on a server - are great productivity enhancers. That's the sort of option that would not be available or practical for most of us.
Grant
I don't know the answer but it's a question that might be interesting to have answered.
Grant0 -
[quote="Emile" wrote:
Then again, Grant, if the DAM side of things is not really important, then why do so many people use Lightroom? I think we mostly agree on this forum that C1 offers superior RAW tools and conversions, so that can’t be it... And a lot of relative newcomers on the market are busy developing and integrating a DAM solution in their RAW development products.
Personally, the lack of a professional cataloguing feature and Phase completely ignoring complaints about it are my big frustrations with C1. I like the product enough to work around it, with a multitude of 5k to 7k image catalogs, but it’s 2019. They could do a lot better.
Lightroom is what's placed on people's lap so they just use it, also a Mac. People think this is the right way to do photography since they are given it or listened to someone and thus see everyone else do it out in the wild.
I took a Photo 101 class in college and we were taught to use Lightroom and all the computers in the room were iMac. I must have been the only one in class that ditched Lightroom for Bridge afterward than I found Capture One.
Catalog is a tacked on feature put in the middle of C1's current lifespan. It's evident that catalog is like Phase One's middle child.0 -
Yes, indeed Adobe products seem to be the default products for a majority of photographers, there are "must use this" choices for operating systems, cameras, social media platforms and a lot of things. Frankly, I rarely encounter photographers who have heard of Phase One or Capture One, and I think that's a big problem Capture One faces: it's hard to gain market share if nobody knows about your product, regardless of how good it is. At this point Adobe probably doesn't need to advertise (much) because their huge user base and media do it for them. I wonder how Capture One gets advertised other than to Phase One, Sony, and now Fuji users who get versions included with their cameras. But that's a different question than the big DAM comments I see in this and other threads.
I don't think that with Aperture it's fair to say things like "Apple didn't see a market for DAM (etc.), so they dropped it". Apple's revenue is huge, and at this point some pundits even wonder whether Apple will continue with computers: they're now seen more as a device and services vendor. For them, the relatively minor revenue stream from Aperture probably wasn't enough to hold their interest. They've discontinued a lot of popular, market leading programs over the years. Remember MacDraw, MacPaint, MacDraft, and so on? How about their big thrust into desktop publishing? Apple seems to build markets and then desert them when they're no longer of interest—for whatever reason.
As far as DAM goes, given that Capture One, Lightroom, Luminar, OnOne and probably others programs have DAMs built in, I'd suspect that there isn't sufficient demand for standalone DAM products anymore to justify continuing them. Further, given the number of vendors who seem interested in incorporating DAM into their post-processing programs, I'm inclined to think that at this point DAM is expected to be part of a good pp program—table stakes as it were. And as time goes on, I can't see Phase One being able to either continue with a so-so DAM module or discontinue it. If they really want Capture One to continue appealing to professionals and serious amateurs who need to be able to quickly find images that meet specific criteria, then it's important to have a useful catalog/search capability.
So regardless of what Apple did to Aperture years ago (which it seems brought Phase One a number of customers), and regardless of whatever else Apple or other vendors have done, Phase One made a decision (imo the right decision) to incorporate DAM into Capture One (I for one wouldn't have considered it without DAM), and just as they continue to improve C1's processing capabilities, I hope they listen to customers and improve the DAM capabilities beyond their current "just usable" state.0 -
[quote="mattcohen" wrote:
This is a very helpful answer, I appreciate it. Unfortunately I have very extensive metadata, and getting rid of it would defeat the purpose (actually being able to search and export the relevant pictures) of a DAM.
I'd be embarrassed if I made something that was more limited (in any way) than five-year-old dead software.
I wouldn't like it either, but I don't work for Phase One and I don't have responsibility for this product.
You have a few options if you think its the metadata filter tool problem:
1) Move Filter Tool to another tool tab. Use it only in collections of 5,000 -10,000 images or less.
2) Review your Metadata. The expensive part of your Metadata is not necessarily the part that's causing the problem. The kind of Metadata that seems to cause issues is longer strings that are different for every image.
3) Raise a support ticket and see if you can make any progress there. It would be nice if you are successful, but past history is not encouraging.
4) Find a better tool.
I've complained (and others have to) about performance issues for a long time. About 2 years ago, at the beginnning of Capture One 10, there were some considerable performance improvements. Since then other aspects have sped up some, but not the Filter Tool issue. I have a choice of staying with this product warts and all, or of finding another product, (likely with a different set of warts).
A relatively low impact solution would be a way to disable indexing of some Metadata fields. An index of a Metadata field with 15000 unique values provides no real value to the user, it just burns a lot of CPU and human time.0 -
[quote="Eric Nepean" wrote:
A relatively low impact solution would be a way to disable indexing of some Metadata fields.
Using a tool such as DB Browser for SQLite (Windows) makes it possible, among other things, to create and drop indices to your hearts content. Drop the wrong indices, and retrieval performance will suffer.[quote="Eric Nepean" wrote:
An index of a Metadata field with 15000 unique values provides no real value to the user,
Not universally true.[quote="Eric Nepean" wrote:
it just burns a lot of CPU and human time.
Indices in general speed up retrievals by noticeable amounts, saving human time. They only cost at definition time, the time it takes to write like "CREATE INDEX ZKEYWORD_ZLEFT_INDEX ON ZKEYWORD (ZLEFT)", and at Insert and Deletes (not noticeable to users)
MLI0 -
[quote="SFA" wrote:
Reading some comments on here from time to time there do seem to be some users who are happy with the performance.
Indeed that is true, but I believe the differences in performance originates in different usage patterns, more specifically in the use of keywording, which because of the database design I believe comes with severe penalties in Capture One.
My analysis re. the keywording performance has been moved to:
MLI0 -
Eric, every time I search the forum for help regarding the Mac version of CO, your name pops up. I cannot find a way of contacting you privately - perhaps this function is disabled in the forum. I thought I'd add you as a friend, but there seems to be a problem with that function: I think your name is in the forum member list two times. I just wanted to thank you for all the work you're doing answering questions in the forum. If you're interested in contacting me, I can provide my email address. Cheers, Francis. 0 -
[quote="SFA" wrote:
If I understand things correctly Apple replaced Aperture with Photos for OS X.
How does Photos compare to Aperture?
Apple replaced both Aperture, its professional-level photography software, as well as iPhoto, its consumer-oriented photography software, with Photos. Compared with Aperture, Photos was more limited; compared with iPhoto, it was more advanced. I used iPhoto only sparingly and used Aperture regularly, but my impression was that Photos was closer to iPhoto than Aperture. Photos was supposed to improve its editing capabilities through its plugin architecture, but I've been largely unimpressed with it... and was somewhat surprised to see Capture One, which is very strong for photo editing, go the route of adding a plugin framework as well.
I stopped using Aperture a few years ago (no support for my current camera) and was limping along with Photos, so I can't speak exactly to the performance difference between Aperture and Capture One. But it's definitely surprising how Aperture's performance and features haven't yet been totally overshadowed by modern software.0 -
People speaking about Apple ditching Aperture to Photos as an excuse for PhaseOne not improving CaptureOne are missing the point.
Apple first ditched Aperture because they wanted people to use Service-based (think iCloud) programs, that promote the use of both Apple-branded computer and smartphones. That increases revenue and loyalty.
They have decided (and as an avid Aperture user, I regret it) to promote a software that pushes simplicity and takes control of managing the pictures. Hence their "automatic moments" organisation and also the use of IA to retrieve people but also whatever object you want to search. The recognition capabilities are still in the infancy, but that is their vision of the future.
On top of that, and as mean to their global "halo" strategy, they're working hard to converge user experience on every device (computer, smartphone, tablet), and Photos is part of that.
So the point is that Apple didn't want you to put your iPhones pictures in Aperture anymore, and this was more important than the Aperture revenue on non-iphone users.
And by the way, Photos is very efficient in retrieving instantly your picture in a big database on different devices.
Coming back to the subject, there are no technical impossibility in building a better DAM for PhaseOne, even if Aperture history (early versions performance issues) suggests that it is not a simple job.
I believe the issue PhaseOne is faced with is : what the point of building a better DAM when the way of the future on the consumer market is going cloud, IA and multi-platform (mean: computer + tablet + smartphone), which is arguably not where PhaseOne can become leader.
The problem with thinking that way is that they are reducing themselves to a niche market of professional photographers that work mainly on a session based workflow, and don't care much about DAM management because they don't need it. And will become more and more challenged by the functionalities available on consumer apps.
I don't know enough about the rentability and metrics of PhaseOne to offer a definitive view, but my feeling is that if they don't improve significantly the DAM aspect of Capture One, it will become very difficult to justify for a 180€ a year recurring revenue, except for a pro niche market that will go diminishing.0 -
'I took a Photo 101 class in college and we were taught to use Lightroom and all the computers in the room were iMac. I must have been the only one in class that ditched Lightroom for Bridge afterward than I found Capture One. '
Interesting lessons you have, it is good that today students are taught to use Lightroom. I thought Lightroom is taught only on some courses.
0
Post is closed for comments.
Comments
22 comments