Speed up keyword search
Hi there
agian CO1 is giving me the spinding ball
is there a way to speed up searching by keyword
i created a smart folder for 2015 with approx 6000 Raw photos in all with keyword but it takes more then 20 sec. before i can search in it and that is sometimes even slower
i run Mac 10.3 on a 1 T Bssd hard drive with 16 GB ram and still is slow
if i do the same thing Lr the photos that i look for are in milli seconds
Henrik
agian CO1 is giving me the spinding ball
is there a way to speed up searching by keyword
i created a smart folder for 2015 with approx 6000 Raw photos in all with keyword but it takes more then 20 sec. before i can search in it and that is sometimes even slower
i run Mac 10.3 on a 1 T Bssd hard drive with 16 GB ram and still is slow
if i do the same thing Lr the photos that i look for are in milli seconds
Henrik
0
-
Make a bug report/improvement request. I've done it last year and nothing has improved in any of the new versions. ☹️
Especially the combination of date + keywords performs poorly (due to the way how they are stored in the database and poor database design). My workaround was to add all images of a year in an album and then search for the keywords within that album (don't know if smart folders can have a condition about being in a album). That is still too slow (several seconds) but at least usable.0 -
Yep i was doing the same thing but still makes me wonder if i should stay or should i go 😊 with CO1
i have found a few bugs and none has been addressed yet0 -
[quote="Lorenzen" wrote:
Yep i was doing the same thing but still makes me wonder if i should stay or should i go 😊 with CO1
i have found a few bugs and none has been addressed yet
I feel the same and am starting to loose confidence. It is about time for 9.0.3 to get released and it better solves some of the main issues like performance and memory use.
My subscription is for a full year and expires in October. I think the problems are all solvable and it should not take years. I would prefer not having to switch again (after coming from Aperture) and then it would probably be Lightroom, which would at least also include Photoshop for the same price but I never liked LR. Unfortunately LR does not have an importer for C1 catalogs, so that switch sounds more painful. I am not even done yet with my Aperture > C1 migration. ☹️0 -
Well i have been thinking the same
i might go back to Aperture it works so instead of using software to get better results i get better gear
i for one have chosen no to renew my sucripstion until they get it fixed
to many bugs and to slow, but lots of adjustments to do that is CO0 -
Right now I am "forcing myself" to use CO9 and continue my slow migration from Lr. CO8 almost got me to switch from Ap, but not quite - I went to Lr instead. Lr is very competent software, but just doesn't suit me - modal, weirdly cross-platform, and all.
CO9 is so close. 9.0.2 is still buggy in some places and strangely awkward in others. But its DAM has reached the point of minimal usefulness (if we can ever get stacks I'll be happy). The interface is nice and it's very customizable.
Perhaps this year, it will be reasonable to use CO and Affinity Photo in place of Lr and Ps.0 -
I'm also still a bit frustrated by C1's DAM capabilities, although keywording made it pretty useful for me. I still want better Finder integration and hope they significantly improve overall C1 launch speed and speed when changing folders/albums/etc. And the Print and Import preset capabilities have lots of room for improvement. But I love C1's editing capabilities.
So: I've been exploring apHUB, which lets me use C1 for edits and Aperture for everything else, and it's looking really good. I'm hopeful that Aperture's DAM and printing capabilities will work for a long time. At this point I couldn't care less about Aperture's Raw conversion and editing capabilities.
There was a lot of discussion about apHUB in various forums many months ago, but the talk seems to have subsided considerably. Here's info:
http://www.aphub.de0 -
[quote="Lorenzen" wrote:
if i do the same thing Lr the photos that i look for are in milli seconds
I just switched from C1 to LR and you are right, within a second it displays search results in a large catalog (largest I've tried has 42K images) for keywords and in combination with dates etc. That's how it should be and made me realize what a pain it was to use C1 and it is not too much to ask for.0 -
The speed and the stability of the keyword / DAM features are the reason why I am not using them. I am looking forward and I am waiting for a lot of improvement from PO here. The current state is usable for a small shooting, but not workable even on a one day equestrian event. I never dared to try it on a full year's collection or more.
That's one of the two major issues I have with CO.
I stick with CO because of the look and feel of the gui, the way the adjustments apply, and the results I get. But if I was in more need of a good DAM and keywording, CO would not be my 1st choice.
Regards,
Hans0 -
I stick with CO because of the look and feel of the gui, the way the adjustments apply, and the results I get. But if I was in more need of a good DAM and keywording, CO would not be my 1st choice.
Amen!0 -
[quote="harald_walker" wrote:
[quote="Lorenzen" wrote:
if i do the same thing Lr the photos that i look for are in milli seconds
I just switched from C1 to LR and you are right, within a second it displays search results in a large catalog (largest I've tried has 42K images) for keywords and in combination with dates etc. That's how it should be and made me realize what a pain it was to use C1 and it is not too much to ask for.
The really frustrating thing is that this is really not rocket science. All you need to do is set up ENOUGH index tables inside your SQL database and then any query should fly. It seems that Phase One either lacks basic SQL coding manpower or the whole database has been set up the wrong way.
Nothing which could not be easily fixed with a little expert input. It's not like the imaging algorithms, which are extremely more complex and difficult to handle and improve compared to SQL code.0 -
[quote="EnderWiggins" wrote:
The really frustrating thing is that this is really not rocket science.
Absolutely agree and I've first reported performance issues related to keywords more than a year ago. No progress since then, in contrary, the new keyword library features only made it worse. I am not willing to wait any longer and hope for a miracle. It doesn't seem to have priority for Phase One.
Today I regularly had to force quit C1 as it is just not responding any longer. Of course that damaged the catalog database... (but I have my backups).0 -
Awaiting PO rep to reply... 🙄 0 -
[quote="Wesley" wrote:
Awaiting PO rep to reply... 🙄
There may be one.
But this is intended to be a User to User forum - so no guarantees.0 -
[quote="EnderWiggins" wrote:
[quote="harald_walker" wrote:
[quote="Lorenzen" wrote:
if i do the same thing Lr the photos that i look for are in milli seconds
I just switched from C1 to LR and you are right, within a second it displays search results in a large catalog (largest I've tried has 42K images) for keywords and in combination with dates etc. That's how it should be and made me realize what a pain it was to use C1 and it is not too much to ask for.
The really frustrating thing is that this is really not rocket science. All you need to do is set up ENOUGH index tables inside your SQL database and then any query should fly. It seems that Phase One either lacks basic SQL coding manpower or the whole database has been set up the wrong way.
Nothing which could not be easily fixed with a little expert input. It's not like the imaging algorithms, which are extremely more complex and difficult to handle and improve compared to SQL code.
My understanding of the requirement for tuning SQLite is based on reading around the subject and discussion with the development lead of a business application that runs only on Windows. I have been a beta tester for that software dsince before it moved onto SQLite and for one release I focused on performance challenges through various iterations of the code pre-release.
It seems that for most applications you can easily set parameters to allow for fast searching but this will compromise data loading for large databases unless run on extremely fast systems. The tests were fine up to about 50k records when adding data to the database (it is an ETL application so some similarity to a photo editor but for an entirely different purpose.)
At that point things started to slow and became progressively slower as the load size increased.
One could speed up the load by having fewer indexes to create on the fly. Or, in a business operation, run the process in a form of batch mode so that the load processed relatively quickly (freeing up the data source which might be a live read from another application) and then processing indexes in slower time where possible.
Finding a good compromise between the two could be a challenge especially if a default index was being populated with information that might be, shall we say, unhelpful as a source of useful identification data. I.E. pointless for that particular process.
In the end the only option was to find the best default compromise that would be likely to work acceptably well for most users and then seek further enhancements over time whilst in parallel looking for any further developments that might be useful for future releases.
Part of the problem was that the nature of the application would have benefited more from faster processors because it had linear processing requirement but at the time the processor developers were more interested in splitting performance across cores for parallel processing where the main benefit was for machines running multiple applications rather than high throughput from a single application.
Based in that background, and making a few guesses about the way that C1 works compared to guesses about how some of the other applications mentioned often here are possibly working, there may be some differences that to not lend themselves to an easy or swiftly executable compromise between, say, editing performance and DAM performance - exactly the same areas of data wrangling with which we grappled a few years ago with the business application I mentioned earlier.
I assume one could adopt a more powerful DB solution but at what cost? And would the benefits be of any use to the majority of users. Based on my personal experience - no, but then I use session and run on Windows and so I clearly have different needs to many in this thread. And of course that hardware difference and the need to support 2 OSs is unlikely to be making anything easier for the developers.
So all in all it's a task that is likely to have some obvious challenges and some other things that are far less obvious unless you are immersed in the code with at least one eye on legacy needs, one on now and your third eye on the future areas of change that you might predict to be approaching.
Quite a tricky place to be.
Grant0 -
[quote="BeO" wrote:
hm, well, with very similar application and volumes I guess, it seems LR uses SQLite too:
cheers
BeO
Yep.
And it was not so long ago that we saw the odd comment around suggesting that LR was slower than C1.
Then LR announced a new release that, it was claimed, was significantly faster ....
I have no idea and no way to compare.
However whether the applications are processing in a similar way may not be easy to assess.
Some year ago I compared the application I used at the time (a non catalogue application and in many ways similar to C1 conceptually if using sessions) with LR which at the time was probably late version 1.
User of the application I liked were commenting that it was not fast to process (forget DAM and Keyword stuff, they were not the issues - just the processing) compared to LR on the same machines.
And sure enough that is exactly how it appeared watching the screen.
However the total elapsed time for a measurable process ( I probably used an Export of an edited file to compare things since that seemed to be at least a measurable start and end point and long enouth to time) was about the same. In fact "my favourite" app was a tough faster.
How so?
Well, a little like C1, the "favourite" processed from the RAW up most of the time - slightly different since the tools were stacked and what changed in an image was, so afar as I could tell, mostly based on the output of the previous tool - so a change somewhere in the middle of the stack might start from there rather than the bottom. But forgetting that for the start to finish process it started at the bottom, shuffled a few screen pixels to tell you it was doing something, and then presented the final image - often drawing blocks of screen area as the split calculations were brought back together.
On a fast machine you might not see that but on a slow one it was quite obvious.
LR, by comparison, wrote some highly visible change on screen almost as soon as it had started the process. In particular anything that had an impact - like sharpening for example and vivid saturation. Job done in fractions of a second . but you didn't get the cursor back for several more seconds. Just windows being slow to do its file management stuff?
Well, looking carefully it seemed not because the the entire waiting time seemed to have LR making small almost imperceptible changes all over the screen - resolution changes, colour adjustments, backed out some of the sharpening (or appeared to ) and a host of other things. In other words it was appearing to be fast and then using the users response time as they thought to themselves "Wow that was fast." before moving on, to finish the job completely. To see these adjustments happening on the machine I had back then required that you watched the process at 100% view so it was having to do some work - even when using jpg files.
I am almost tempted to download an LR trial to see if it gives me the same impression these days - many years later.
Obviously things are likely to have changed in the intervening years and there was nothing troubling about the way LR processed the files but it was interesting to discover that the clear and evident speed advantage for an end to end process (as watched on screen on that machine) was not exactly what it seemed, visually, to be at the time.
Ever since then when observing performance, especially on screen performance, I have sought to double check what it really being presented if comparing two products that, nominally, are performing the same tasks in their own ways.
Grant0 -
I used Firefox Addon SQLite Manager
New catalog
imported 1 image
added 1 keyword
added 1 Local adjustment layer
cloned variant
this results in
1 image, 2 variants, 4 layers, right?
The database holds the following records:
1 image, 2 variants, 8 layers of which 4 records have my keyword
The keywords would be better stored in the ZVARIANT table (where the 2 records are) and not in ZVARIANTMETADATA with 8 records, then the performance should be better, even if perfect indexing is assumed. Same should apply to all the other metadata belonging to a variant!
Database design seems to be "historically grown" and made by application developers, not database designers. But there are always application design reasons to deviate from a database design which might only on the first sight be better. That is reality in many companies.
The good thing about it is that there is probably room for improvement.
Just my humble thoughts...
If P1 staff reads this, take it as a suggestion, not a criticism.
Cheers
BeO0 -
I know a single case is not much of a proof, but nevertheless…
I have C1 9.0.3 and the latest LR (subscription). I have converted the entire Lr catalog of 60K images into C1. I can tell you that Lr is lightning fast in comparison, on the same computer. Catalogue on internal SSD, images on external HD. Heck, even my 3 year old copy of Aperture outruns C1.
If people complain about the speed of Lr, they should try C1. They won't complain anymore!
I'm hoping Phase One can get this sorted out, because I really would like C1 to become the best program there is!
Cheers,
Peter.0
Post is closed for comments.
Comments
18 comments