Catalog corruption!
Hello. I'm really hoping someone can help me with this as my catalog seems to have suddenly gone corrupt. C1 attempts to do a verify and repair, but returns with "Database connection FAILED. The requested document type is not recognized. The provided database is of an unknown type". This happened after I did a computer restart and then tried to reopen C1. Was working fine before that.
I am so incredibly bummed because I'm working on a tight deadline and spent 70+ hours over the past 5 days toning 2000 images from a shoot. This is my first time using C1 after using Lightroom for the past several years, and thought I would give it a try. I'm currently using the trial version.
I'm hoping there is a computer wizard out there who can help me. I'm seriously willing to pay money to anyone who can somehow fix this for me. Tried calling customer support but they are closed right now. ☹️
Aaron
I am so incredibly bummed because I'm working on a tight deadline and spent 70+ hours over the past 5 days toning 2000 images from a shoot. This is my first time using C1 after using Lightroom for the past several years, and thought I would give it a try. I'm currently using the trial version.
I'm hoping there is a computer wizard out there who can help me. I'm seriously willing to pay money to anyone who can somehow fix this for me. Tried calling customer support but they are closed right now. ☹️
Aaron
0
-
Are you running time machine (or in fact any) backups ?
If so you may be able restore an earlier catalog and retain most of your work.0 -
I don't have time machine set up. I have another way of backing up files, but unfortunately the catalog has not been backed up since I began using it 5 days ago...
When I view the contents of the catalog package, I see an extra file that ends in *.cocatalogdb-journal. I'm not sure, but I believe the corruption may have occurred because I was trying to restart the computer and C1 was still in the middle of writing the new changes into the catalog file but wasn't able to complete it before the computer shut down. Really hoping there is a way to repair it.0 -
Did you do a hard reset or force quit Capture One?
The .cocatalogdb-journal file tells us that Capture One was busy writing data into the SQLite database and wasn't able to complete that operation, just as you wrote.
Don't have an idea how to fix this and don't understand how you can work on such a large project with a tight deadline using a new software and no backups.
There are some tools and instructions online on how to fix a SQLite database. Maybe there is a way to repair it. Of course always use a copy of the .cocatalogdb file for any of those experiments. You can copy the file out of the package and then rename it to .sqlite (some tools only work when it uses the correct extension).0 -
Thanks so much for the information, Herald. That's what I suspected.
I always make catalog backups once a week, per the programs default suggestions. I've been using brand new versions of Lightroom since it came out several years ago and never before had a catalog failure.0 -
Hi.
I ran into the same problem with the at present extremely buggy Capture One 9. Nothing I tried helped until I attempted this step:
I created a new empty catalogue and imported the old one that I was unable to repair. It took a while (more than 10000 images) but all my work was recovered.
Hope this solution is not too late for you and works.0 -
Thanks, Bernd. I actually happen to cross paths with a programmer online yesterday who was able to run the corrupt catalog file through some of his software, which fixed whatever was causing the issue. Amazing. All my work is there just like how I left off. Probably took him 2 minutes to fix it, and saved me from having to redo the last 5 days worth of work.
On the other hand, Phase One Capture One technical support was utterly useless. I sent them a message explaining what happened along with the catalog file, and asked if there is any way this can be repaired, and they responded with just "you should backup your catalog", without addressing my question or offering any further support. I do backup my catalog once per week as the program suggest by default, but that doesn't account for the last 5 days of work when a catalog file goes bad. Lesson learned that it should be backed up every day.
But even a days worth of work can be a lot to lose. Instead, perhaps Phase One should build more reliable software, or at least something built in to automatically safeguard their users from losing the relatively simple, yet time consuming adjustments made to raw files when their buggy software fails.
As I said, I've been using Lightroom for 8 years and never had a file go bad. I didn't think it was really necessary to have to build a backup catalog every single day, which seems like overkill. I know a lot of photographers, and none of them do it that often.
Clearly, the database of information was still there in the "corrupt" catalog file. It was probably just a matter of a couple lines of code that were off, but for some reason, Capture One's catalog verify/repair tool isn't sophisticated enough to retrieve any of the data.
Point is - if this same glitch happens to anyone else, you can expect Phase One's tech support to leave you high and dry, but there are other people out there who may be able to help.0 -
Hi Aaron, [quote="Aaron Morris" wrote:
This happened after I did a computer restart and then tried to reopen C1. Was working fine before that.
You didn't answer my question about the restart. Did you quit Capture One and then restart the OS using the restart function?0 -
[quote="harald_walker" wrote:
Hi Aaron,[quote="Aaron Morris" wrote:
This happened after I did a computer restart and then tried to reopen C1. Was working fine before that.
You didn't answer my question about the restart. Did you quit Capture One and then restart the OS using the restart function?
Capture One appeared to be closed out of when I did a restart, but it probably wasn't fully shut down. For some reason, C1 began using up all the memory, which my computer notified me of, but everything stopped responding. After waiting for a little while, I had to force my computer to restart using the power button, which is what I imagine contributed to the catalog error.0 -
[quote="Aaron Morris" wrote:
C1 began using up all my computer's memory, and everything stopped responding. After waiting for a little while, I had to force my computer to restart, which likely caused the catalog error.
So related to that memory bug. But the same would happen in Lightroom if you just hard reset the computer while it is writing to the catalog. There is nothing the software can do if you just force quit the application or do a hard reset. This memory bug is a serious issue but we all know that other software is not bug free either, including Lightroom and I've heard enough horror stories from Lightroom users.
Your computer offers you Time Machine for free which can take regular incremental backups. It takes hardly any effort to let it run each day or even multiple times throughout a day if you are worried about loosing a days work.0 -
[quote="harald_walker" wrote:
[quote="Aaron Morris" wrote:
C1 began using up all my computer's memory, and everything stopped responding. After waiting for a little while, I had to force my computer to restart, which likely caused the catalog error.
So related to that memory bug. But the same would happen in Lightroom if you just hard reset the computer while it is writing to the catalog. There is nothing the software can do if you just force quit the application or do a hard reset. This memory bug is a serious issue but we all know that other software is not bug free either, including Lightroom and I've heard enough horror stories from Lightroom users.
Your computer offers you Time Machine for free which can take regular incremental backups. It takes hardly any effort to let it run each day or even multiple times throughout a day if you are worried about loosing a days work.
Hindsight is 20/20.
There is other automatic backup software I use, but generally have it turned off while I'm working because I've had issues with it interfering with my computers performance while running heavy programs like Capture One or Lightroom.
I'm sure Lightroom has it's fair share of problems as well. But I have to say, I've had to do LOTS of hard resets on Lightroom over the years and still never had an issue.
Again, I don't see why this sort of software is developed to rely so heavily on a single file, to where if a single line of code becomes messed up, everything is lost, and it's the fault of the user for not having their computer make automatic backups several times a day. Why can't Capture One make automatic backups several times a day, just incase?
What was more frustrating for me was the lack of support from Phase One. Obviously there was a solution, probably a fairly easy one that could have really helped out one of their customers and they failed to even try.0 -
It makes automatic backups of catalog file weekly. So in case of corruption you may revert.
Catalog files are always a single point of failure in any DAM, so back it up or sync to other drive to prevent cases such you are describing. I've got similar case with Lightroom and it cost me 2 weeks of work restore to correct state.0 -
[quote="Andriy.Okhrimets" wrote:
It makes automatic backups of catalog file weekly. So in case of corruption you may revert.
Catalog files are always a single point of failure in any DAM, so back it up or sync to other drive to prevent cases such you are describing. I've got similar case with Lightroom and it cost me 2 weeks of work restore to correct state.
I think the point I'm trying to make is why not design a program like this to include multiple redundancies just incase a catalog file was to become corrupt? Instead if placing all the responsibility on the shoulders of the user to make multiple backups daily incase this SINGLE file was to become corrupt? Catalog failures have obviously been a problem that happens from time to time which has caused people to lose work. I just don't get why this is too much to ask for...0 -
That is why C1 automatically back them up on a weekly basics.
You can speed it up by clicking File -> Backup Catalog every time you want to.
Just backing up every time not makes sense for all users.
Multiple redundancies == multiple writes == slow disk operations, and as pointed out before all DAM software is catalog dependant like Ligthroom or Aperture, so all have same issue.0 -
[quote="Andriy.Okhrimets" wrote:
It makes automatic backups of catalog file weekly.
And you can even set it to always or daily but these backup files also take quiet a bit of space.
I don't quiet understand why it makes backups in the backup location AND in the catalog file package. Same database files, so using the disk space twice. I would also like to be able to configure how many backups C1 keeps. I really don't need months worth of backups.0 -
Nice feature request. Send it to PO, currently software do not manage backups it just creates them. But catalog files are relatively small(it is actually small database), so I have no problem keeping them forever.
It keeps only last catalog backup inside catalog folder as I recall.0 -
[quote="Andriy.Okhrimets" wrote:
But catalog files are relatively small(it is actually small database), so I have no problem keeping them forever.
I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.0 -
A small hack here I always specify backup catalog location folder to NAS (do you know that you can change that) and store them forever. ) 0 -
[quote="harald_walker" wrote:
I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.
You don't have to have the back up on your local SSD, just set it to be stored somewhere else. As for backing up the DB files, there a plenty of software that can manages that and do it well.0 -
[quote="Paul Lindqvist" wrote:
[quote="harald_walker" wrote:
I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.
You don't have to have the back up on your local SSD, just set it to be stored somewhere else. As for backing up the DB files, there a plenty of software that can manages that and do it well.
Please read again what I first wrote, it also makes the same backup within the catalog file on the SSD. I know that I can change the backup location in preferences to an external disk.0 -
[quote="harald_walker" wrote:
[quote="Paul Lindqvist" wrote:
[quote="harald_walker" wrote:
I have multiple catalogs and within each catalog the SQLite database file is between 100 and 250 MB. Catalog are all on the local SSD disk and space is limited there. Each GB counts. I was able to regain several gigabytes of disk space by cleaning up old CaptureOne backups. For incremental backups I have my time machine backups.
If C1 would keep 1 or two copies, that would be more than enough and is basically just for disaster recovery like Aarons problem.
You don't have to have the back up on your local SSD, just set it to be stored somewhere else. As for backing up the DB files, there a plenty of software that can manages that and do it well.
Please read again what I first wrote, it also makes the same backup within the catalog file on the SSD. I know that I can change the backup location in preferences to an external disk.
Im not sure i understand, are you saying that it backups the database on two locations, one always in the catalog folder and one in the location you specified in the preference panel ?
I just tested this and i have no backup databases in the original catalog folder, just get them in the location i specified.0 -
[quote="Paul Lindqvist" wrote:
Im not sure i understand, are you saying that it backups the database on two locations, one always in the catalog folder and one in the location you specified in the preference panel ?
I just tested this and i have no backup databases in the original catalog folder, just get them in the location i specified.
Yes, exactly that. Here is a screenshot.
https://dl.dropboxusercontent.com/u/975141/captureone-backup-in-catalog.png
I guess this is another bug then.0 -
[quote="harald_walker" wrote:
[quote="Paul Lindqvist" wrote:
Im not sure i understand, are you saying that it backups the database on two locations, one always in the catalog folder and one in the location you specified in the preference panel ?
I just tested this and i have no backup databases in the original catalog folder, just get them in the location i specified.
Yes, exactly that. Here is a screenshot.
https://dl.dropboxusercontent.com/u/975141/captureone-backup-in-catalog.png
I guess this is another bug then.
Strange, could you please post a screenshot of the location where you specified the backup's to be stored ? Does the timestamps and size correlate ?
I tested this and it does not happen on my installation.0 -
One thing I have observed with Capture One Pro is that faced with a big job, it may become non-responsive for minutes or even more than an hour. I have learned that if you interrupt its work by killing it, you will loose something. There are times when it crashes, this it announces with an error message, and then you have lost some work also.
When COP becomes non-responsive, I launch the activity monitor, and check how how much % CPU and Total Memory (GB) it is consuming. If it is non-responsive and see it consuming less than 1% CPU (which happens to other SW but not in my memory to COP) then I would know it has become lost and I must reset it. But more commonly I see it consuming a high % of CPU, and the amount of RAM may rise to many GB. Once the virtual RAM consumed is is many GB larger than the physical memory, then I know there is a very big job - I have seen COP crash when it has consumed over 60GB of RAM on my 16GB machine (the extra 45GB is virtual RAM) - and then you get an error message.
But if the CPU is still being consumed, and the the Total Memory is not too much larger than yopur installed RAM, then just let it keep running. On a large import I have seen it be unresponsive for 75 minutes.
I must say that being unresponsive for such a long time with no indication to the user is a poor user interface design. The processing time is a limitation that must be acceptedas it is a big challenge to reduce and is affected by the users HW, but the non-responsive part is simply a poor design choice IMO.0 -
I have exactly the same issue as Harald describes - my main catalog on an SSD drive had grown to over 25Gb in size (all referenced images!) but after crashing due to the memory leak in CO 9.0, became corrupted. In trying to restore from a recent backup, I discovered that the catalog package contained over 28 backups taken over the course of the last 12 months IN ADDITION to the backups I had stored on a separate drive.
One by one over the next few days, other libraries and several backup copies became corrupt after crashing. In 9.0.1 things have improved but the crashing still happens far too regularly. Trying to restore or import from a backup, C1 is unresponsive for long periods and consumes vast amounts of memory. I tried leaving it overnight to import an backup library into a new one which it said would take 12 hours. Checking on it an hour or so later, I was informed my system had run out of application memory and my boot drive had run out of space.
I've got just 11 days left of my trial - how on earth can you properly test this out when it doesn't seem to run for more than half an hour before it crashes? Shame because I really like what I've seen of the new features but it is slowly driving me nuts!0 -
Poor Aaron just wants a solution to his problem - not a bunch comments about why he did or did not back up his catalog.
Shame on all you perfect people for trying to malign him instead of helping him with his problem.
I agree - released software should have extra safeguards in it to prevent data loss. I've used both Aperture and Lightroom for years with approx 30-35k images and never had such a problem.
Data loss is the one thing a Digital Asset Manager should never cause.
Now I have the same problem Aaron does.
In my case, I was simply adding a keyword when Capture One hung and the fans on my machine cranked up.
Activity Monitor showed Capture One using between 500 and 600% of the CPU.
Now I've lost my entire catalog...
Not such a big deal for me - this is day 2 of my 30 day trial and I just imported a database from another program.
But still - not confidence inspiring.
I can't make hierarchical keywords work on my DB import - but I'll ask that question in another topic since this one is on corruption.
Cheers!0 -
[quote="CaptureMann" wrote:
Poor Aaron just wants a solution to his problem - not a bunch comments about why he did or did not back up his catalog.
Shame on all you perfect people for trying to malign him instead of helping him with his problem.
This!
There is a flaw in the system, period!!!!
My catalog was backed up, and guess what, the backup was corrupt as well. I ended up having to rebuild the catalog. This took a couple of days.
I have been a C1 user for quite a while, and while the RAW conversion has taken huge leaps, the catalog leaves much to be desired.0 -
I am using C1 Pro for Sony 9.0.1 on an iMac with OSX 10.11.2. I have now had multiple catalogue corruptions (my database is less that 10k images), and have tried all the suggestions used in the forum. I can still work with a slow system (much slower than C1 😎, but having to create a new catalogue and import all images again on a weekly basis is just too much.
I absolutely loved C1 for its workflow and results, and the addition of the DAM function made me think this is now the best of all worlds. However, given the experience since C9 came out, I am now back to a combination of LR6 (for DAM purposes) and DXO Pro10 for raw processing. In my humble view not the best solution, but at least I will be able to do my work without having to wait for previews and having to rebuild catalogues every now and then.0 -
Just briefly reading into this thread. Did you try these suggestions? [quote="BerndInBerlin" wrote:
Have posted this for another person who had the same problems:
I managed to "recover" a destroyed catalogue by simply creating a new one and importing the old damaged one.
Dear Phase One - with all due respect for the great work on this software. Maybe it is time to make a public announcement that Capture One 9 is at present NOT working reliably with Canon files.
A pretty serious issue that would affect one or two photographer out there.[quote="HansB" wrote:
Rebuilding is also possible from command line. I don't have a broken catalog to test it, but maybe somebody wants to try it on a copy of a broken one? It's command line, so cd into the catalog file.
cd <YOUR>.cocatalog
mv <YOUR>.cocatalogdb <YOUR>.cocatalogdb.yourbak
echo .dump | sqlite3 <YOUR>.cocatalogdb.yourbak | sqlite3 <YOUR>.cocatalogdb
Another option might be to query VACUUM; on the catalog database.
Not sure if this works, but maybe it helps creating a solution.
Regards,
Hans
Regards,
Hans0 -
[quote="Andriy.Okhrimets" wrote:
It makes automatic backups of catalog file weekly. So in case of corruption you may revert.
Catalog files are always a single point of failure in any DAM, so back it up or sync to other drive to prevent cases such you are describing. I've got similar case with Lightroom and it cost me 2 weeks of work restore to correct state.
CO actually uses a database to store - most of - its data (basically all meta information but not the images or thumbs). And the one used is actally not bad if employed the right way, eg making use of transactions. It would be interesting to find out when CO actually considers a piece of work to be completed (eg after an individual adjustment or when you move to the next image). Depending on when such commits are done, you basically never lose anything. The most likely use case for losing data would then be a crash of your HD - even pulling the plug should actually NOT harm your database...0 -
The same thing happened to me this weekend. This is definitely related to the memory issue (iMac 5k). Program beach-balled forever after trying to combine two identical keywords. I eventually force quit CO9 and when I went to open it back up it was "Database connection FAILED".
I have my database on Dropbox so I theoretically have infinite backups. I am hopeful I can restore easily.
I did not find CO8 to be nearly so buggy. I am only a hobbiest, so delays are not deadly for me. I can't imagine how a pro could deal with this. Maybe they run better hardware?
Disappointed, to say the least.0
Post is closed for comments.
Comments
67 comments