Corrupt Catalog / Error -36 / Unable to Optimize

[Robert Gallucci]

Need help. My master catalog is damaged.I know, I know that I am a fool for letting it get so large (12.28 GB / 500,000+ images) but that is what I did.

The good news is that I was able to make a copy in the same directory and rename it which, for some miraculous reason, now allows me to open it.

I cannot copy it to another drive (error -36)
I cannot optimize or back it up. My last successful backup is from 9/27/2016.

I have tried dot_clean. It did not work.
I have tried the sqlite3 and it did not work.

I attempted to export my 2016 images to a new catalog on a new drive with mixed success. The catalog was created but 1/2 the images to not move.

Adobe support told me to export to a new catalog without copying the negative. I received a message that smart previews could not be built.

None of my actual images reside on the hard drive that holds my catalog, so they are all intact.

In both new catalogs flags remained but color tags did not.

Ideally, I would be able to create exports for each year in the master catalog.

Any help is greatly appreciated.

-RG

 

[clee01l]

Welcome to the forum. If your last good catalog backup is 9/27, you could begin with that as a new master catalog. Then your options become:

A: Import all new images since 9/27. You can probably achieve this by simply sync'ing the folders to pick up any images in them and not in the 9/27 catalog file.
B: OR Export from the recovered master catalog those images added since 9/27 and Import these into the 9/27 master​

You are probably using a different color label set on these new catalogs. The Label field was transferred just like the flags and stars. So I don't think there is a problem with the label values, just the colors that LR associates with the label values. From the menu {Metadata}{Color LabelSet ▶︎}, pick a ColorLabel Set that matches the color that you expect to the labels that were transferred with the Export.

 

[Robert Gallucci]

That is a great idea. I will restore the 9/27 cat today. If I sync the files back will their edits come with them?

Yes! I will change the labels this morning.

I guess it's time to come up with a better catalog strategy. Think I will also start duplicating my images on import as well. My images were not affected, but but after this incident I'm nervous about the integrity of everything.

This is a great form. Did not know it existed. I'm looking forward to going through the different topics to learn and contribute.

 

[clee01l]

That is a great idea. I will restore the 9/27 cat today. If I sync the files back will their edits come with them?...

Nope. Your image adjustments are stored in your master catalog or your last good backup (Save early save often) If you were automatically saving the metadata to XMP, there would be an XMP section in each master JPEG, DNG or an XMP sidecar file for proprietary RAW originals. This would only preserve your last develop adjustments and not any collection membership or develop history.

I backup my master catalog after every LR exit though I don't always exit every day

Duplicating your images on import is not a backup strategy. The "Make a second copy to" option on import is to make a temporary second copy so the you can reuse the camera card before your regular system backup software can do the job it is supposed to do.

You need a bulletproof backup scheme using dedicated backup software for all of your critical user data including your LR catalog files setting and original image copies and any other user data (Word, Excel) that you need to preserve in the event a disk failure. Apple includes TimeMachine for this eventuality. You are not using sound judgment if you don't take advantage of TimeMachine's functionality.

 

[Robert Gallucci]

I do use Time Machine. However, I have only used it for my internal disk, not any of my external disks. Just did not think of that. I am moving to a dedicated, LaCie RAID 0 Thunderbolt drive for my catalog and another LaCie drive for my "working" images. I quote working because I am going to switch to the smart workflow you suggested in another post. Yes, I know about RAID. I am choosing RAID 0 for performance instead of RAID 1 for redundancy. If I use TM to backup the disk I should be fine.

If I stick with one master catalog then I will be moving older photos to either an 8 or 5 TB EHD.

My reason for external disks is that when I purchased my iMac 5K I intentionally went with a smaller, faster 750GB SSD drive with the intention of putting lightroom catalogs and photos on Thunderbolt drives.

I do automatically save XMP data. I just finished syncing the photos back into LR and the develop data is there. That may be a stroke of luck due to my workflow. I typically make initial adjustments and then move to PS to gain the benefit of layers for other adjustments. I may make some final adjustments on the resulting image. It is disk intensive, creating two images (original DNG and a PSD) but in this case it seems to have saved all my edits.

Do you save your backups to the same disk as your catalog or on a separate disk?

FWIW - your advice is rock solid and I am learning a great deal. Thank you.

 

[clee01l]

Do you save your backups to the same disk as your catalog or on a separate disk?

I too have a 5K iMac with a 1TB Fusion drive I forget about the size of the SSD part of the fusion Drive but it is 24GB. I trust MacOS to keep my active files there. The OS, LR app and catalog and the most recent 3 months of images are kept on this Fusion Drive. Images as they age past 3 months are moved (using LR) to another TB2 EHD. By keeping my most active files on the fusion drive and the most accessed files on the 24GB SSD part, I optimism my performance in LR. I only need older images doe the occasional export or print so that can reside on the slower TB2 EHDs

I have 9 EHDs. Two used for backup (I'm a belt and suspenders kind of guy) Some are JBODs that are composed of several 6-8 year olds SATA disks that are used to store expendable data. My Time Machine backup is a 8TB Thunderbolt2 HDD. My local CrashPlan backup is a 3TB USB3 HDD. (CrashPlan also does an offsite Cloud Backup). Time Machine Backs up the primary drive, the TB2 Drive that contains my master image copies and the recent backup catalog copies on one of the expendable older EHDs (to answer your question as to where are my catalog backup files)

On Import, I send the "Make a Second Copy" images to one of the expendable older EHDs. Although I could, I don't delete these simply because I have the space to keep them. When this EHD fills I'll then delete the folders "Make a Second Copy" images.