C1P8, catalog and NAS server problem.
Hello.
I've just added Synology server to my network, hoping that I'll be able to work directly on the images stored there.
After few days of testis in the session mode, I've found that this works well with small sets of images (and with small sized images even in big sets). While there were no problems with the session of 50..100 Nikon D700 NEFS, with D800 files it was different matter - huge network traffic made things extremely slow at the beginning of work (after 10-15 minutes it was fine, till the re-launching program or changing session folder).
So I decided to try catalog mode (which should do the job).
But surprisingly, this is not much different. When I launch C1P8 and open folder (included in catalog), thumbnails are displayed rather quickly. Also several images can be viewed/edited fast. The problems starts after viewing several images, or after switching different folder. When this happens, C1P start to import massive amount of data from NAS with speed 10-15MB/s, and it takes some 10..20 minutes for a 150..200 image set. During this period of time computer is slowed down, C1P does not respond.
Sometimes, when I try to shut down C1P (normally, not forcing program to stop with task manager) and relaunch it, it fails - task manager shows, that it is still active (however, with low CPU usage).
NAS is attached to I7/8G RAM laptop with gigabit network. Of course, catalog files are on the local drive, NEF images are on the NAS.
What I am doing wrong?
I've just added Synology server to my network, hoping that I'll be able to work directly on the images stored there.
After few days of testis in the session mode, I've found that this works well with small sets of images (and with small sized images even in big sets). While there were no problems with the session of 50..100 Nikon D700 NEFS, with D800 files it was different matter - huge network traffic made things extremely slow at the beginning of work (after 10-15 minutes it was fine, till the re-launching program or changing session folder).
So I decided to try catalog mode (which should do the job).
But surprisingly, this is not much different. When I launch C1P8 and open folder (included in catalog), thumbnails are displayed rather quickly. Also several images can be viewed/edited fast. The problems starts after viewing several images, or after switching different folder. When this happens, C1P start to import massive amount of data from NAS with speed 10-15MB/s, and it takes some 10..20 minutes for a 150..200 image set. During this period of time computer is slowed down, C1P does not respond.
Sometimes, when I try to shut down C1P (normally, not forcing program to stop with task manager) and relaunch it, it fails - task manager shows, that it is still active (however, with low CPU usage).
NAS is attached to I7/8G RAM laptop with gigabit network. Of course, catalog files are on the local drive, NEF images are on the NAS.
What I am doing wrong?
0
-
[quote="NN321062UL" wrote:
Hello.
I've just added Synology server to my network, hoping that I'll be able to work directly on the images stored there.
After few days of testis in the session mode, I've found that this works well with small sets of images (and with small sized images even in big sets). While there were no problems with the session of 50..100 Nikon D700 NEFS, with D800 files it was different matter - huge network traffic made things extremely slow at the beginning of work (after 10-15 minutes it was fine, till the re-launching program or changing session folder).
So I decided to try catalog mode (which should do the job).
But surprisingly, this is not much different. When I launch C1P8 and open folder (included in catalog), thumbnails are displayed rather quickly. Also several images can be viewed/edited fast. The problems starts after viewing several images, or after switching different folder. When this happens, C1P start to import massive amount of data from NAS with speed 10-15MB/s, and it takes some 10..20 minutes for a 150..200 image set. During this period of time computer is slowed down, C1P does not respond.
Sometimes, when I try to shut down C1P (normally, not forcing program to stop with task manager) and relaunch it, it fails - task manager shows, that it is still active (however, with low CPU usage).
NAS is attached to I7/8G RAM laptop with gigabit network. Of course, catalog files are on the local drive, NEF images are on the NAS.
What I am doing wrong?
I'm not able to help with your NAS question except to say that I'm not sure personally that a NAS (especially if using RAID) is always going to be a good way to go. There are a number of technical issues about the setting up of NAS systems and the networks they are attached to (even or perhaps especially "simple" local networks) that look to me like more trouble than they are worth unless the files are to be shared by a group of people. For small documents - no problem. But for large files or set of files, hmm.
The starting point would seems to be a nice clean gigabit network known to be capable of working at gigabit network speeds and a good cable. Then start to fine tune the performance. I suspect you may benefit from more RAM as well when dealing with large data sets on a regular basis. (My i7 notebook, Win7 Pro with 24GB RAM and typical use with several programs running shows memory usage somewhere around 12/14GB)
I don't use catalogues but I do have some quite large sessions (several thousand images for example) that I work on, sometimes, from an external USB3 drive. The loading/saving operations are clearly slower than when working on an internal SSD but are still quite usable - what would take half a second might require a whole second - that sort of difference.
The only problem I have seen is when using a long (3mtr) USB cable. Most often the drive, if it has gone into low power mode, disappears and won't respond even to disconnection and re-connection. However, it's not a NAS and so does not allow me to access the same files form multiple machines without swapping the cable over between devices or sharing (will always be slow) through the current host device.
As for your orphaned process after shutting down C1 - I have experienced the same thing and raised a support case.
I think it first arose sometime after I installed 8.0.1.
The solution in my case was to set OpenCL processing to NEVER. On my machine this makes no difference at all since the GPU seems to be bypassed anyway due to low performance. .In fact setting just the DISPLAY use to NEVER seemed to work in my case - no orphaned process on shutdown.
I have now installed 8.0.2 and so far this does not seem to have the same problem whether OpenCL processing is on or off. I suspect that there was a rather obscure problem but I am also mindful that it might just have been some value for something set as part of the normal operation that became "stuck" but was fixed by the update process and is not operating normally once more. Time will tell.
HTH.
Grant0 -
Thank You for Your input.
I've done simple test with co7. New catalog, two folders with 150 and 200 images imported (nefs and several huge tifs) and everything is ok, even with nas.
co8 - import does not work, nedd to add folder/synchronize (as I did it before). Import module generates previews for over 10 minutes, and when I click "load all", it does nothing.0 -
Are you working with the same folders already used for V7 import (to a catalog?)?
The V8 import of the files is to a new catalog?
For the import process the "Import All" button, in my experience, requires one or more images in the preview panel to be selected before the import will start. I think it is more of a "Start Batch" button than a "Select all and start batch" button but there may be circumstances when it performs as a select and run function - just not in my experience. You did not say whether you had selected any images.
The V8 installation set up new preferences. One setting that can have a significant effect on performance when generating preview files (which C1 will tend to do up front to make later usage more responsive) is the setting for the size of the preview file compared to your screen (or screens) native resolution. If you have set V7 to your own chosen default value you may need to do the same for V8. Doing so may make a difference if the settings are not identical.
If that does not seem to make any difference I would suggest creating a Support Case and seeking assistance in interpreting what is going on and why V7 seems to be satisfactory when V8 does not. My non-NAS and non-Catalog experiences have not displayed any significant variation in the times taken by tasks - I guess if I ran test with a stopwatch there might be differences either way but I have not seen anything to suggest I would notice a difference if I did so other than using an external spinning drive on USB3 being a tad slower than the internal fast SSD. I should mention that while I often deal with large number of files the file sizes may not be as large as the ones you are working with, especially not huge TIFFS.
HTH.
Grant0 -
Both files and catalogues for both versions are in the same locations.
Here are further results of my tests:
Import:
C1p7 imports sets of files without problems - no need to do anything else but click "import all".
C1p8 failed to import: first i took very long time to "count" the files, then "imort all" gave no result. Afair, this was the effect I've encountered when importing from sd/cf card (only jpegs were imported). Not sure, why in c1p8 it tooks enormous amount of time to create previews.
In c1p8 I was able to create catalogue by "add folder/refresh folded" sequence.
Working with catalogue:
C1p7 works with NAS without problems.
C1p8 generates excessive network traffic. I used catalogue created with add/refresh sequence, also imported catalogue that was created in c1p7 and was fine there.
Conclusion: support case is needed.
Edit:
Just checked: C1P7 works better in session mode, than C1P8 in catalogue...0 -
I think the same in that you need a support case. You could take a look at your preview size in the preferences, and check it is not set to low. 0 -
Hello.
You mean about resolution too high or too low?
Regards.0 -
[quote="NN321062UL" wrote:
Hello.
You mean about resolution too high or too low?
Regards.
Edit menu > Preferences > Image tab > Cache section > Preview Image Size.
If set too low (1024 px for example), slow performance and heavy network traffic may (will) occur.0 -
Thanks.
I will check this later, but AFAIR, I started with the default previews size, which is rather big. Later I switched to 1024, and nothing really happened (tried to regenerate previews and import i images to a new catalog).
Now I want to check timestamps of files in CO session/catalog folders. I suspect that CO8 regenerates previews every time, when image folder is opened/accessed.
I still don't have idea, why CO7 properly imports images, and C08 doesn't. Support claims that imported modules of 7.2.3 and 8.0.2 are exactly the same, but I think they are not.
Regards.0 -
The usual advice about preview file resolution has always been to set it to the native resolution of your screen (or main screen and then hope the graphics card and driver do a good job of managing things if you use multiple screens).
That should mean that the mass load of previews has relatively little work to do when loading for browser use whereas too large or too small (compares to your screen native resolution) may require a lot of calculation which will start back at the source file (if available when using referenced catalog) and apply all the edits. Some edits that you may use regularly might be quite intensive users of processing capacity - difficult to say without knowing what you do but a possibility.
There are other considerations perhaps.
Are we also potentially talking about different aspect of the software hear? The word "import" could be applied to different parts of the system and so could be confusing the discussion around it.
Grant0 -
Thanks for contribution.
Some progress can be noticed.
1. About import: advice from support is to turn off global filter. it helped a lot. Turning off the filter hiding jpegs made displaying previews faster, and enabled import of raw files (!). It is curious, but it worked. I think there's bug in global filtering rules.
2. Slow operation of NAS can be explained (at least in session mode) with fact that recalculation of some files is performed every time after opening the folder. So far I am almost sure, that all files from the /captureone/settings80 are affected by this issue. For comparison: files from /settings70 are dated back to 2013, to the moment of their creation, no matter these folders were opened many times recently (with version 7).
Regards.0 -
Some more knowledge(?).
CO8 is responsible for a huge traffic, but not only on NAS - it also appears on HDDs.
For some reason CO8 tries to read many file from the folder: previews, original RAW and JPEG images and others - no mater if it opens folder for the first time, or has been there many times for a long time ( long enough to create all previews).
There are probably as many "links" between co8 and files, as many files are located in the folder (including files in sub-folders, created by co8).
It doesn't looks like effective approach...0 -
There are many situations where C1 might be processing files in the background as you work - for example if you have opened a folder (or all folders and images in a session by selecting the All Images album) and you have currently selected a different output recipe compared to whatever you last used.
The most common recalculation is for display size compared to the projected default screen resolution.
So much depends on how you have things set up and what you are doing at the time.
I would suggest opening a support case to request some outline guidance but you may also gain understanding of what is happening in your specific setup using step by step testing analysis making sure you only change one thing at a time and then observe disk activity.
Earlier you described a situation that seemed to occur mainly with Nikon D800 files - is that still the case. D700 ok but D800 a problem?
Grant0 -
Processes are invoked for other cameras too. I've checked it with V1 images. Though, for smaller files this is not a big problem.
Case is opened. I'm waiting answer.0 -
If you have Preferences/Image/Metadata/Auto Sync sidecar XMP set to anything other than "None" it will likely account for some traffic as well.
Cheers,
Mogens0 -
No, no sidecar files.
Finally, I'm almost sure, that the problem is with recreation of the content of /settings80 folder.
CO8 re-writes all images there, even when no edits were done. It is enough to switch for a moment to the different folder with images. And the file re-creation performed for all images. CO7 does it when it is needed: once, when folder is opened, later probably when image is edited.
When I open new folder in CO for the first time (both v7 and v8), both needs some time to create appropriate set of subfolders and files. But CO7 does not replace them with new files them when it is not needed, and works fine. CO8 replaces lots of them with no reason.
This causes traffic not only on NAS server, but also on HDD (on HD it can be easily watched by system tools). Because HDD is faster, it is not a big deal (however, in CO7 I can switch faster between folders, as can be expected).
My NAS easily reaches 800Mb/s via gigabit Ethernet, so even d800 files works fine with CO7. With CO8 NAS is useless, and makes my workflow much more complicated.
Support so far wished me good luck. I hope that they will came with some solutions soon too.0 -
only browsing the files on NAS is a really pain ☹️ i am using Synology with 2x 1 Gbit/s Ethernet and it is slow like hell ☹️ 0 -
Did You tried to use CO7 with Your NAS?
CO7 works fine for me (I use Synology too, but on 1G x1 - It reports about 100GB peak transfer, which means it is about 0.8Gb).
CO8 requires very long time to open folder - (100 NEFs from D800 needs 20 minutes), then it works fine. The problem is - it requires such big amount of time every time folder is accessed, while CO7 needs it only once - when folder is opened for the first time (and it is faster).0 -
[quote="Martin_" wrote:
Did You tried to use CO7 with Your NAS?
CO7 works fine for me (I use Synology too, but on 1G x1 - It reports about 100GB peak transfer, which means it is about 0.8Gb).
CO8 requires very long time to open folder - (100 NEFs from D800 needs 20 minutes), then it works fine. The problem is - it requires such big amount of time every time folder is accessed, while CO7 needs it only once - when folder is opened for the first time (and it is faster).
Are these new images in a new folder or are you opening V7 sessions/catalogs in V8?
If they are originally V7 are you getting C1 to convert them to V8 or leaving them as V7?
Grant0 -
There is no difference if the folder is "fresh", or was opened before with co7 or co8.
Co8 spends a lot of time checking the content of the folder.0
Post is closed for comments.
Comments
19 comments