Zum Hauptinhalt gehen

⚠️ Please note that this topic or post has been archived. The information contained here may no longer be accurate or up-to-date. ⚠️

Slow Advanced search speedup

Jerry C
THE PROBLEM: When I attempt to do an advanced search on my 50,000+ catalog, the process is very slow. First, I have to select "All images" and this takes a couple of minutes. Entering search criteria is quite slow. Capture One acts as if it is scanning the entire catalog with each click or keystroke used to construct the search. Each one of these can take up to 30 seconds for me. After the last of the criteria have been entered, it can take up to a minute to complete the search. Cancelling the search so that all images can be seen can take several minutes as Capture One undoes the search, depending on how you remove the search filter.

WORKAROUND: Create a Smart Album. Creating a smart album automatically searches the entire database without having to select an album to be searched. A smart album would not be very smart if it did not.

So, create a smart album and enter the various criteria. There are no delays in selecting or entering text. Also, the smart album finds all of the images described in the search criteria in about 10 seconds. Using a smart album to save search criteria is a practical way to save a search or edit a search. Capture One does not have an alternative for saving searches.

I can't explain why this works blazingly fast, whilst entering the same information directly into an advanced search does not. After much back and forth with some real nice smart support folks, the reasons for my problems with advanced search as opposed to creating a smart album remains a big mystery.
0

Kommentare

11 Kommentare

Datum Stimmen
  • peter Frings
    [quote="NN635680879799322049UL" wrote:

    I can't explain why this works blazingly fast, whilst entering the same information directly into an advanced search does not. After much back and forth with some real nice smart support folks, the reasons for my problems with advanced search as opposed to creating a smart album remains a big mystery.


    Creating a smart album is a good idea, but a bit tiresome if you frequently want a one-of search for a particular image. But as you said, it's a workaround…

    <guesswork>
    The reason for the slowness of the advanced search *could* be because they *might* search the entire collection every time you make a change.

    I don't know how they've implemented the filter, but it's not always easy to get it right. E.g, the default search (after you type the +), is "any contains", followed by an empty field. If you type a '5', it searches your entire collection of 50K+ images. It has to. Say it finds 1000 images. Now you type a '6'. If it's smart, it will only search those 1000 images because your change narrows the search: '56' can only be in those images that contain a '5'. If it's not smart, it will search the entire collection for '56', which is clearly a waste of time. However, if you delete the first '5', or if you add an extra criteria with "Match any", it has to search the entire collection because your change widens the search. So, in order to be smart, the implementation needs to analyse the filter and the change made to it. Or, an implementation can opt for the 'safe' way and always search the entire collection on every change you make. With a 50K collection, that can indeed be slow.

    With a smart filter, there is no live change needed, so in theory a single query is all you need.
    </guesswork>

    Now, maybe I'm completely wrong about the advanced search. I guess that my theory could be checked by saving the advanced filter as a preset and as a smart album, and then timing the difference between the two. If my theory is correct, both preset and smart album should have similar timings. If the preset remains much slower, my theory is wrong and something else is afoot.

    Only Phase One can tell (if they'd want to).

    Cheers,
    Peter.
    0
  • Jerry C
    Indeed, if you do an advanced search on All Images, CO appears to search though all images with each change in the advanced search filter, as Peter describes. Using a smart album appears to execute the search in one very fast step.

    Regarding the difference between a preset and a smart album: Before going to the workaround, I tried configuring and saving an advanced search as a preset.I did this in a small album to avoid the delays at the time of entry. I then displayed my 52000 images (All Images). This took about 30 seconds to display. Applying the saved preset, CO took 35 seconds to complete the search and another 3.5 minutes to finish displaying all of the images and for the spinning beachball to stop entirely. I have no idea what CO was doing during this time. So, the preset route is a nonstarter for searching a large catalog.

    With a smart album, nothing happens until you click the OK button after entering the search criteria. You get the results of the search of the entire catalog without having to display any particular album in 10 sec without the spinning beachball. To do a new search, just edit the smart album search criteria.

    I have no insight as to why the workaround works so much faster than applying the preset, which I suspects just replays the events you see when doing an advanced search on 52000 images displayed in the ALL Image album. Clearly, CO is doing something different, but much less efficiently to reach the same outcome. An advanced search should operate just like smart album. I have referred this to the engineers and perhaps they can explain it.

    Jerry C
    0
  • peter Frings
    Hi Jerry,

    I did some tests on my largest catalog of 8000 images (I gave up on catalogs larger than 10K).

    I did not notice a real difference between the smart album and the preset (selected in the browser, not in the search dialog). Also, I didn't notice a really difference when tying it in.

    But, it looks like C1 does some caching of some sort, because when you edit the advanced filter and then change it back, the results often appear very fast.

    But, I've also noticed that the CPU use goes up significantly _after_ the results are shown, and that it remains high for several seconds. So it's doing something after showing you the found results.

    BTW, what was your search criteria?


    For me, the main disadvantage of the advanced filter is that you have to pass via the 'all images'. Your smart album trick is the way to avoid that.

    Cheers,
    Peter.
    0
  • Jerry C
    Most of my experiences with advanced searches for All Images are that search time and speed entering criteria slow down as the All Image count goes up. My back of the napkin calculation indicate that this is linear, as one would expect with an iterative process. So, I would expect the delays with my 52000 images to be about one-seventh that observed with 8000. In that case, 20 seconds, which is not so slow and the entry of each variable in the danced search would take about 4 seconds. Noticeable, but not a big handicap.

    I did the test advanced search using all occurrences of a specific key word. I repeated it using any conditions under which one key word was found or any occurrence of a particular text in metadata. Both took equivalent times to complete. I was surprised that increasing the number of search criteria from 1 to 2 did not substantially affect the search time. maybe I need to time things more carefully.

    Regarding the CPU activity after the results are shown, with a very large catalog, this can be associated with a delay in Capture One becoming responsive. You might not notice this with a smaller catalog.

    For me, searching a large catalog, the smart album approach works. I would not mind going to get coffee while the search was conducted so much as being driven nuts with the slowness of entering the search criteria directly into the advanced search filter.
    0
  • Jerry C
    Added detail: I selected a preset I previously configured (using an album with few images to avoid the delays in entering the search criteria). I then selected the preset from the search button to the right just above the browser. The search criteria were set to find all occurrences of a specific text entry. This requires searching all metadata fields and should be a good "stress test" of the system. Indeed, this was quite fast, taking 23 seconds to search the entire 52000 images. Also, there was essentially no delay in returning control after the search. While the search was about half as fast as using a smart album, it was quite snappy. I expect if you searched 8000 images rather than 52000, you might not notice a difference of a second or two.

    I did notice that if I clicked the "X" on the search button to cancel the search, it took 90 seconds to undo the search and return control, whereas, clicking on the text word on the search button and deleting it, to 60 seconds. Using a smart album, the search is cancelled and All Images are immediately displayed and there is no delay in returning control.

    One other problem that using Presets presents is that you need a new one for any different search, which means making a new one. To me, editing the smart search album's search criteria is more efficient.

    Now comes the confusing part and a wild guess. Why should any approach to a search that returns the same results not be done in a single manner that is most efficient? Why should there be different timings resulting from cancelling a search? One can achieve the same output from the same input in a variety of ways. Perhaps different engineers wrote the code for searches done via smart albums than those who wrote the code for advanced searches. Differences in coding could account for what we have observed. The reason could be that the code for the advanced search is a legacy from when it made sense to scan the database for each additional entry of criteria, so that each iteration searched fewer files, and the smart album is a later feature written with a much more efficient code that took advantage of the features of newer hardware. Clearly, the answer would require knowing how the code for Capture One evolved and and be informed by the engineers who oversee the current code. I have a friend who was the program manager for OS2 WARP, and he describes how coders, working in isolated groups can produce code that works, but is not as well-integrated as it could be.
    0
  • peter Frings
    The discrepancy between presets and smart albums *might* be explained when we assume that the contents of a smart folder is updated as soon as you change something in the catalog. If that would be the case, selecting a smart album does not execute the query, but simply displays what it has. Another thread in one of these fora suggests that this might be the case, as the OP claims that removing smart filters and the Filter tool speeds up the program considerably. In contrast, a preset would keep such an updated list, and so it has to run the query when you select it. Hence the slower time.

    Again, this is pure speculation and one would need to set up a few carefully designed tests on large catalogs to get more insight. Then again, C1 has so many threads running doing god knows what that you would actually need to trace the execution to know exactly what's happening. You can only optimise your code if you measure its execution, or think hard about the algorithms you use. I sincerely hope PhaseOne has done such kind of analysis.

    Cheers,
    Peter.
    0
  • Jerry C
    Guess we just have to wait for the engineers to do their thing and figure out what code needs to be deprecated and replaced.

    I also saw the post about removing the filter and smart filter tools.I don't have any other issues with efficiency/speed sufficient to justify removing this feature.

    Jerry
    0
  • SFA
    [quote="NN635680879799322049UL" wrote:
    I have a friend who was the program manager for OS2 WARP, and he describes how coders, working in isolated groups can produce code that works, but is not as well-integrated as it could be.


    WARP was a long time ago and the development environment has changed a lot but with the current "Agile" development concept the same old style problems may indeed still exist even though in theory "Agile" tried to avoid re-inventing wheels and so shares code whenever possible.There are probably far more code control regimes around as well ... but then nothing is full proof against anomalies creeping inno matter how many systems one tries to put in place to avoid problem situations.


    Grant
    0
  • Jerry C
    Note: Both Advanced Search and creating Smart Albums are much improved in version 10. I will post the details, there. There are still a few quirks.
    0
  • Jerry C
    This may be worth trying with version 9.3. I have not tested it, there, however. COP version 10.0.1 behaves quite fast with my 52000 image catalog after moving the Filter tool from the Library tab to a custom tab. Unfortunately, removing the Filter tool from the Library tab took 30 minutes of watching the spinning beach ball, so you need patience and a look at the Activity Monitor to see that activity is actually happening. Thereafter, advanced searches were near instantaneous as each search criterion was entered. I don't know why CO has to scan the catalog with each little addition to the search criteria, but since it was instantaneous, why not.
    0
  • Jerry C
    The good news is that 9.3 allows faster entry of advanced search criteria and faster searches in version 9.3, when the Filters tool is moved out of the Library tab.

    Unfortunately, I spoke to soon about version 10. Today, capture one version 10.0.1 became unresponsive when I clicked on an image in the All Images Collection. I do not close C1 with All Images displayed (take longer to load the next time), so things worked well until I clicked on All Images. After 2 hrs of watching the spinning beachball, I force quit and that trashed the database beyond repair. I have backups, and copied one and opened it with the same results. It was really nice while it lasted.

    I reverted to version 9.3 while waiting for a functioning version of 10.

    Jerry C
    0

Post ist für Kommentare geschlossen.