To NAS or not to NAS?
Hi there
I have seen a lot of posts about poor performance when using a NAS. I hope to purchase a NAS in the coming months so would like to find out if it's going to be fit for purpose, before spending the £2k needed for a properly specified NAS + 10Gb network upgrade.
My current system is: Win 10 Pro. Dell XPS 15 laptop, Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, 16Gb RAM. All CO catalogues and RAW files are stored on external SSD's. I have ageing (and slow) HD's for backup. [NB: Editing off the RAW's on the backup drives is not an option as the HD's take about 20 seconds to spin up to operating speed, and they spin down after seconds if not activley being used. So this causes serious and unbearable delays when editing]. Also if it's useful I use a Sony A7III so my RAW files are typically 50Mb.
I would use the NAS as the storage medium for all my RAW files. I only need to access them from a single computer, currently the Dell XPS laptop, and I intend to keep the catalogue files on external SSD's, so database management is as fast as possible. I intend to use a 10G network, and to instal fast NVMe SSD cache on the NAS to boost processing and transfer performance. I want to edit directly off the NAS rather than to keep local copies of RAW files on SSD's, as the workflow needed to copy RAW files from NAS to local SSD's when editing - and then back again - is to be honest a total pain, especially since relocating / reconnecting referenced files in CaptureOne is not exactly fast at the best of times. (Yes - I continually experience the long delays caused by re-indexing the catalogue once files have been moved that everyone using Win PC seems to be reporting).
I believe my intended system 'should' perform amazingly well (after all it is going to be fast enough to edit multiple 4k video streams directly off the NAS - so it should be able to cope with editing RAW photo files! I've done a lot of reserach on NAS systems and networking (I recommend 'NAS Compares' YouTube channel, for anyone ineterested), and I have a lot of experiece working with CaptureOne catalogues (at least on local systems). But I am VERY concerned by the comments I read that:
a) general performance is commonly 'very slow' with Capture One when using a NAS, so there is no advantage to using a NAS in terms of editing efficiency
b) it is not recommended to keep your catalogue files on the NAS (even if you place them on a NMVe system drive for super fast access?), so there is no advantage to using a NAS in terms of centralising ALL assets (database files and media) thereby enabling access from ANY networked computer running CaptureOne.
c) every time Capture One connects to a NAS it has to re-index the catalogue, so there is no advantage to using a NAS in terms of speed of access
In summary - everything I understand about networking technology and NAS systems tells me that it's a no brainer, and that so long as it's specified correctly will provide a number of great advantages and efficiencies. BUT everything I understand about CaptureOne tells me it just might not work as inteneded.
I'm hoping someone can tell me I'm wrong - and that using a NAS today with CO is not a problem at all! Or that someone has experience with using a NAS and can recommend a workflow/setup that is proven to be fast and efficient for editing. Or that someone from CaptureOne development can provide detailed information regarding what does work now, and what is definitely coming in a planned future upgrade.
Sorry, long post I know. But I think there a lot of users like me with similar interests that would like to know the answers.
Thanks and Best Regards
Mark
-
Thanks a lot for your reply Christopher.
I don't have a NAS yet, so can't test your suggested workflow. It does seem however that even using your approach the benefits of using a NAS would be somewhat limited, and that the workflow seems more 'workaround' than the alternative scenario of working without a NAS to transfer files needed from offline slow HD to fast local SSD when needed for editing. It's such a shame that C1 does not seem to be built with NAS users in mind! Video editors don't seem to be faced with the same problems. I think maybe a DAS is better for me, and then I will just to use a simple NAS for backup (not then necessarily needing 10Gb network for example). But before I go down that path, can I ask have you set up your NAS content as a mapped drive or as an iSCSI target??
Thanks again for your feedback!
All the best
Mark
0 -
> Mark Porter: ...I only need to access them from a single computer, currently the Dell XPS laptop
Then, why do you prefer network attached storage over directly attached storage?
0 -
Hi Mark,
c) every time Capture One connects to a NAS it has to re-index the catalogue, so there is no advantage to using a NAS in terms of speed of access
What I can see daily on Capture One for Windows: Capture One every time you start a catalog "re-indexes" the images in the selection you are focussing this moment. If your catalog is a large one this will last long - depending on the transferrate of the medium you are using too (but not only).
As most NAS for home users are not very speedy this can be an additional issue.
Kind regards
Ernst0 -
I just tested this. Disclaimer, i am using a very, very fast Windows PC with a 10GB synology DS1621xs as a test environment. My normal workflow is sessions on the local storage that get synced to the synology, and i rearrange that sync folder by archiving older sessions when space get's low (archiving meaning i put them just on the NAS).
Further, as this was asked: NAS has a lot of advantages, especially good ones like the Synology series. The OS of the NAS allows you to do so much more than a DAS. Disadvantage: speed. My local SSDs are PCi 4.0, the are factor 5x faster than the 10GB connection. Advantages, for me, primarily: automatic backup to the cloud, availability of my files in my private cloud, foldersync. Allows me to share work between laptop and PC easily. With the "only have the most recent sessions on my PC, rest on NAS, and backup with drivesync those sessions automatically" i get the best of both worlds.
And an additional disclaimer, I normal do use a session workflow, not a catalogue, so i might have done something silly, or i am missing the experience.
I imported a whole subdirectory with a few files from the NAS. The first bad thing i notice is that there is no progress bar, so you don't really know when the "search through the tree" is done. You can, at any time, use "pick all" and then "import all" and you will just import the, let's say 28% of the files that were scanned so far.
This particular directory contained ~45K pictures, organized in sessions, so with all the metadata, 2700 folders, weighting in at 750GB.
Waiting for the import dialog to be done scanning so that I could finally click "import all" took a while. Be patient. I am talking about... > 30min. That is though, not surprising. The import is not designed to do this kind of work regulary, so for the one time setup I can life with this.
Importing the images (not moving them into the catalog, just importing) was a task that took another 1.5h. Again, not too surprising, given what's getting done here. Generating previews for all of that would take another 5h, which is shamefully bad, considering that my machine is pretty much idle in that time (cpu/gpu usage is very low). This part could be considerably faster. I did not wait for that to finish. Closed the catalog, and did not backup anything. Waited for the CaptureOne background process to finish (another 1min).
Open captureOne again, will load that catalog by default. No performance issue. Switching between star ratings or colortags is more or less instant. If you are interested to see, i put a screenrecording at this link:
http://gofile.me/5IlsV/WCWyhXx1C
Frank
0 -
Because i reread the thread and there was a mentioning that CaptureOne will reindex the currently selected portion, I did make sure to have "all images" selected. All i can see that is getting reindexed is the "folder list" of the network share that i am using. Everything else i neary instant. Not as fast as on SSD, but totally useable.
Frank
0 -
Usable performance, indeed. How big are your raw files?
As expected, previews are being generated when browsing/filtering through your catalog, most of time in your video:
0 -
Most of the RAW files are from a nikon z7 II, so around 50MB in size. And the previews are being generated, because I did not wait for the 5 hours to finish generating all previews. I am on a 4K screen, so generating around 45K of previews in that size takes a while.
That finished eventually, and then it was no longer an issue. The main point is that there are claims here about "constant rescanning, waiting for a long time etc". There is no waiting, at least not for me. All i am saying is that a NAS can be made usable - but you need the correct setup. I did a similiar test over a year ago, using a DS 1517+ and a 1GB connection, and that was not as pleasant an experience :)
Frank
0 -
Frank, thanks so much for your detailed reply. You imply, if I am correct, that the first setting up/importing processes needed when setting up CO with a NAS can take some time - naturally depending on catalogue size, in a similar way you’d expect if doing so using any regular (non networked) HD. And once the catalogue previews have been generated the access performance is good (assuming a decent NAS with a 10G network). Is that correct? That’s what I was hoping for and in some way expecting as I still find it difficult to believe that CO would need to spend hours indexing the catalogue EVERY time it’s used (which is what I believe other users had reported). So you give me some hope that if I make the investment I will be able to make a worthwhile NAS solution for my photo storage and editing workflow . Thanks. I hope this thread can stay open with others also contributing their NAS experiences with CO ;-) All the best, Mark
0 -
Mark, you are welcome. Yes, I am implying that this can work well.
There are a couple of factors for the end performance to consider:
- Network bandwidth: 10G i think is a requirement, 1G is, given the filesize of current digitial cameras just to slow.
- Seek performance of the NAS. If you are working with 45K files, seek performance over the network is important. In my case, this NAS is used by several people at the same time, which is one of the reasons I deployed a high performance NAS for this scenario. The different NAS models from Synology (e.g.) have very different performance characteristics. If you want 10GB, one of the better options (i have one as well here in the office) that should suite you is the 1621+. Vs the 1621xs, you give up a little bit of computing power (ok, more than a little, but for your usecase does not matter), get more flexible RAID options and you save a lot of money :)
- of course, the host system itself. Generating previews, indexing into the catalog (that is stored locally - never tried having the catalog on the NAS) etc. All those things need to be snappy, and in the tested scenario the directory holding the metadata for this catalog is around 10GB in size. The size is not the problem, but that's data that gets constantly updated and accessed. This needs to be fast (SSD).
Disclaimer. As i usually use sessions, i have no long term experience here. So if this crashes down on you after 3 months - sorry, no idea :). But performance wise, this can work just fine.
Frank
0 -
Frank has his catalog locally, which means all data C1 needs is stored locally with the exception of the folders' and images' online/offline status, and the raw files (and eventually derivative "original" files like tiffs etc.). (And maybe xmp metadata files).
You can see in his video that C1 builds the folder structure on startup. I don't know when exactly the online/offline status will be determined (maybe only for the images which are in the currently selected collection). If you want to compare e.g. 4 images at a time in the viewer, with 100% zoom (or any other zoom level for which C1 needs the raw file), and select the next set of 4 quickly, then maybe there is a noticable slow-down? But if you do so, e.g. for culling new images, so you'd probably this locally anyway, e.g. in a session, and import them into your catalog after your main culling.
A few years back I used a 1Gb network slow Synology NAS (RAID setup for redundancy, not performance) and I cannot remember a principle problem (e.g. folders offline, or C1 crashes, due to sleepy NAS or anything like that). (I cannot remember does not mean it didn't happen). I abandoned this setup only for performance reasons.
Chances are higher that something goes wrong, compared to local storage, and I think C1 still does not recommend a NAS setup, but I can imagine going Frank's route in the future too. Currently I use my old NAS only as one of the backup media.0 -
I just tried that. During the workday, i let CO catch up with regenerating all the previews. Now, all is done, and when i close CO and open it up again, there is no more regenerating of previews. I can select a few pictures and then some different ones, and i get pretty much instant responses. I need to check this with sessions, maybe i just switch to this workflow in the first place (instead of relying on folder sync).
Frank
0 -
Any update on this OP? Did you make the move, and any issues so far? I'm considering a similar setup with either synology ds720 or ds920
0 -
I'm curious as well as I'm looking to do NAS.
I'm on a Mac Mini M1 running Big Sur 11.6.5.
0
Post is closed for comments.
Comments
13 comments