• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Saving the life of a corrupted catalog

Status
Not open for further replies.

DLee

Member
Joined
Aug 12, 2020
Messages
74
Lightroom Version Number
LR Classic 9.2.1
Operating System
  1. macOS 10.13 High Sierra
Hi, I have a large LR work catalog that became corrupted after I moved it to a LaCie mirroring program.

The 135,000 file catalog was originally about 150-200GB in size and living on my laptop, suddenly reduced itself to 14GB after attempting to 'rebuild' the catalog. Unfortunately the mirrored catalog on the LaCie backup also started reducing itself and I was able to stop it from 'mirroring' at around 90GB.

Every preview is missing except for what's currently connected via HD in either the original or mirrored catalog. Is there any way to save (any of) this catalog before starting to rebuild file by file?

I have one file called TemporaryRepairCatalog Preview.lrdata that is close to 60GB in size.

Is there any way that I can tie this repair catalog into what's remaining of the original LR catalog?

Or do I need to start from scratch and rebuild the entire thing?

My latest backup is about a year old.

Thanks

Dennis
 
I note that this thread started off talking about "Previews" and then later changed to "Smart Previews". These are different things and may be causing some confusion?

Regarding Victoria's advice, are you referring to her first post immediately following your original post? Where she says "Try renaming the Preview.lrdata, make sure it opens ok"? I read that as a directive to rename the previews directory (which is not the Smart Previews directory) to some other name so that Lightroom won't find it when starting up, thus forcing Lightroom to rebuild its previews. Which is effectively the same advice that everyone else has been offering, isn't it?
 
Regarding Victoria's advice, are you referring to her first post immediately following your original post? Where she says "Try renaming the Preview.lrdata, make sure it opens ok"? I read that as a directive to rename the previews directory (which is not the Smart Previews directory) to some other name so that Lightroom won't find it when starting up, thus forcing Lightroom to rebuild its previews. Which is effectively the same advice that everyone else has been offering, isn't it?

If this is the case then wow, all that discussion was for nothing wasn't it? lol. I have no idea. It probably would have been good if someone had pointed this out earlier. Anyway, the 'Smart Previews' have been rebuilt and that catalog is up and running again. The comprehensive gallery is on it's way.

At some point here I will make a small LR catalog for testing purposes only and build previews and then save them, and remove them, and who knows what all. But it's time to learn this stuff.

Thanks. d
 
Cletus, if the catalog is not tied to a hard drive supporting the files, which mine almost never is

You say that not everyone works like Cletus does. I'd swap that around. The vast majority of Lightroom users are working in a mode far closer to the way that Cletus does. The way that you work, whereby the drives that contain your all of your images are routinely offline is definitely not typical. Some accept that position on a temporary basis so that they can work on a small subset of their images while working remotely. But I don't know of anyone who so completely relies on smart previews for their entire library because they are never connected to the drive containing the image.

I think that is important to understand. That's why you feel that Lightroom isn't helping you as much as it could, because it isn't really expecting you to have 135k smart previews that are you so utterly reliant upon. You need to understand that the Lightroom hierarchy starts with the catalog at the top and everything is underneath that; the physical image file, the preview and the smart preview. That's why any recovery process needs to be library driven. Adobe make a big issue of all file management of your images needing to be done through the Lightroom application. This is why.

My point, again, is that had someone helped me understand how to rename and load a 'saved' Smart Preview file I would have moved it over and just updated the most recent work. I could have been done an hour after the purge.

Think of it from Lightroom's perspective. It has a catalog file, a preview file and a smart preview file. It expects those files to have the same date because they were all last updated at the same time. Because of some issue that Lightroom does not understand, you now want to bring in a smart preview file that is one month older than your catalog file. So Lightroom automatically knows that there is potential conflict between your smart previews and your catalog. And it has no idea what those conflicts are; images in smart preview but not in catalog, changes made to images in catalog but not reflected in the smart preview, file reference changed in catalog breaking its link to the smart preview.... Only one of those three examples is something that you could fix if you "just updated the most recent work."

Now, you could be lucky, the following workflow *could* work for you:

1. Copy the directory with your library, preview and smart preview files to another directory so you can restore them should this go badly.
2. Delete your preview and/or smart preview file dependent upon which one is really causing you problems.
3. Restore your old backup copy of whichever of those two files you deleted. Make sure that it has the same name as the catalog (i.e. the bit before the file extension).
4. Fire up Lightroom on this instance of your library and see what happens.
5. If it looks OK, force a preview refresh on images that you believe have changed since your backup.

*But*, I can't emphasise how much I would caution using this approach rather than rebuilding your previews. You have no idea how successful you've been until you've tried to do something with each and every one of your 135k photos. And if you discover an entire corrupted album two years down the line, it is way too late to do anything about it at that point.

That's why I would say follow the advice of others here. Force a recovery of the previews from within the Lightroom application itself. It is the only fail-safe solution available to you.
 
You say that not everyone works like Cletus does. I'd swap that around. The vast majority of Lightroom users are working in a mode far closer to the way that Cletus does.
Moreover, Cletus is one of the most respected contributors to this forum. He is extremely knowledgeable and puts in lots of time answering lots of questions. I personally have benefitted from his advice many times. You would be wise to pay careful attention to what he suggests, instead of denigrating his responses.

Most newcomers approach the advice given here with respect. I was slso once a newcomer . I'm nowhere near as knowledgeable as Cletus, but my Lightroom knowledge has grown and I'm a pretty technical PC user, so I contribute accordingly. You could also become a contributor over time. That is why this forum is so great.

Now, you could be lucky, the following workflow *could* work for you:
Or you might get unlucky and destroy all your catalogs. Did you have backups of all your catalogs? (That's also a lot of work if you have 14 drives.)
*But*, I can't emphasise how much I would caution using this approach rather than rebuilding your previews. You have no idea how successful you've been until you've tried to do something with each and every one of your 135k photos. And if you discover an entire corrupted album two years down the line, it is way too late to do anything about it at that point.

That's why I would say follow the advice of others here. Force a recovery of the previews from within the Lightroom application itself. It is the only fail-safe solution available to you.

Extremely good advice.

Phil Burton
 
I am curious though, what is the difference between Previews and Smart Previews files within the LR hierarchy?
The Previews cache contains most of the previews that you will see throughout the interface, particularly in the Library module. Each preview file is in fact a pyramid of multiple differently-sized jpegs, ranging from 1:1 at the top, then standard (sized in accordance with the catalog settings), then various different smaller sizes (for grid, filmstrip, navigator, etc.). So, if you were seeing blank grey thumbnails in the grid, that would indicate that a preview file does not exist. Normally in those circumstances, Classic would automatically create a standard preview....however, if the underlying original image is "missing" (e.g. the containing drive is offline), no preview can be generated, so you're left with the blank thumbnails. When images are online, the user can choose to render either standard or 1:1 previews (1:1 is the same as standard apart from having that additional 1:1 jpeg at the top of the pyramid). Zooming into 1:1 in the Library automatically creates a 1:1 preview.

Smart Previews are used only in the Develop module, and these are a special type of preview which can be used instead of the original when editing images. They are not automatically created, but if they exist for an image that is offline they will allow editing to be done. However, they are only 2560px on the long edge, so some caution would need to be exercised if doing detailed sharpening, for example. The main benefit, of course, it that they allow the travelling photographer to take their laptop with them whilst leaving all their external hard drives containing their images at home. A more recent development allows the user to choose to use the Smart Previews for editing even if the originals are online, on the basis that performance would likely be better when using the smaller smart preview file.
 
Wow, it appears that raising the ire of the community releases the flow of information. Thank you.

I had absolutely no idea the preview/smart preview files might be tied into time stamps with the catalog. In hindsight of course this makes perfect sense.
Thank you Graham for that extremely informative response. Had I been given that information (or linked to it) at the beginning there would have never been any debate or continued questioning on my end. I get it. Although it seems that attempting to do what I proposed could have been at least worth a try on a duplicate catalog... if done correctly.

In the end though, I did take everyone's advice and rebuild the previews. More accurately I rebuilt Smart Previews with Cletus' recommendation of only building the upper echelon. This catalog lives on my laptops hard drive and indeed often travels with me without associated hard drives which, as Jim pointed out, is why I live and work off of Smart Previews.

Believe me, images spread across 14 drives was not planned, hahaha. It was the result of one job growing into a 12 year relationship with a wonderful company. But it made me Love working with Lightroom and the Smart Previews. Fortunately my scattered filing system was punctuated by the training I received working at newspapers and I logged all individual shoots by date and subject. This combined with LR's filing system made locating originals very easy when needed. So working with Smart Previews was always really easy for me. The downside of course was a 200GB LR file on my laptop. That file is 20GB now and it will grow as new jobs come in. But, I am perhaps learning how to temper that growth.

So along with getting this 'working' catalog rebuilt on my laptop I have begun building a duplicate catalog on a brand new 4TB HD that will hold as many associated files as the HD volume will allow. For which I will probably NOT need to build Smart previews, lol. This is something that has been planned for a very long time and because of this 'crash' it's happening. Thank goodness my catalog stayed intact through this disaster.

As a reminder to everyone here. I believe this happened to me because of the LaCie mirroring backup program that comes with their hard drives, it's called Tool Kit. Use at your own risk.

Which brings me to a very important question. Is there a very good backup program that can back up the daily changes in LR from one drive to another? Specifically for a Mac in my case.

I was reading somewhere here that Victoria worked off of Smart previews (at some point) with her files stored in Dropbox. I thought this was a pretty intriguing idea and I should really look into.

Thanks everyone and I'm sorry if I ruffled any feathers here, it was not the intent. The intent was to not only fix my problem but to begin to understand the hierarchy of the Lightroom file system... and methods for repair and possibly 'trimming the fat' for future reference. I'm still pretty unclear on this stuff, so any associated links to this information would be greatly appreciated.

Thanks again,

Dennis
 
Dennis,
One of the main objectives here in the user led forum is to answer the question asked. A secondary objective is to help users to develop a standard workflow that meets their objectives.

The question asked was about lost previews and how to go about recovering the images in the Preview folder. Most of the ice in this thread has been addressed to that primary goal.

Only late in the thread did you reveal your workflow objective and only then did we begin to focus on helping you improve your workflow.

As an aside, you mention building a new catalog on a noe machine. There is no need to do this and especially NOT to reimport images.
Your objective as I see it is to develop your master catalog on your master computer including the paths to all of the image files and associated Preview Folder and Smart Preview Folder.

Once you have that streamlined, you can use a blog article on Victoria’s website on how to move/copy your master catalog to an new computer.


Sent from my iPad using Tapatalk
 
Wow, it appears that raising the ire of the community releases the flow of information. Thank you.

Excerpting for clarity and focus:

True enough in your case, but most unusual for this forum. I could even go further and quote Victoria, "We're a friendly bunch here."
As a reminder to everyone here. I believe this happened to me because of the LaCie mirroring backup program that comes with their hard drives, it's called Tool Kit. Use at your own risk.

My career was a field called "software product management, and thanks to the vagaries of companies in IT or High Tech," I've had more jobs than I would have preferred to. In my experience, most companies, Apple being the noticeable exception, are either good at hardware or software, not both. To hardware companies like LaCie, software is just an additional expense, to be minimized. And when you "minimize" the expense of software, you inevietably miminze the quality.

Nikon is a great example of a hardware company with so-so software and a lousy (until maybe recently?) software strategy.

Which brings me to a very important question. Is there a very good backup program that can back up the daily changes in LR from one drive to another? Specifically for a Mac in my case.
I'm a PC user, not a Mac user, but a great cross-platform program is File Sync & Backup Software | GoodSync

I use the paid version, and I use it on a daily basis to sync files between desktop and laptop and to sync files between my backup program's hard drive in my system case, and an EHD that is the "backup of the backup."
I'm still pretty unclear on this stuff, so any associated links to this information would be greatly appreciated.


Thanks again,

Dennis
People on this forum want to help each other.

Phil Burton
 
Thanks Cletus,

Either you misread my description above or I was not clear in my writing. I am not moving the catalog to a new computer (at least I hope not anytime soon) but instead building a comprehensive Hard Drive that will contain both the Catalog and ALL (or as many as possible) of the associated files so that they are all together in one place.

As I write this though I think that maybe I should keep the 'working' catalog on my laptop, and re-associate all the images in the catalog with the new comprehensive file holding HD. I could build Smart Previews for the files that I'm currently working on and the legacy keepers. The rest would be accessed by connecting the comprehensive HD when needed. This is probably better than having a separate comprehensive catalog on the HD. Wouldn't you agree?

d
 
Thanks Phil, I will check out GoodSynch and see if they make a Mac version.

Your software/hardware comparison is well heard here. I had been shopping for a good program without finding anything with much appeal. When the LaCie program worked without hitch on some simple folders and image collections over time I figured I'd give it a try with the LR catalog. That of course was a huge mistake. Lesson learned.

I appreciate the 'friendly' space as well. My lack of knowledge however, could not wrap my mind around the fact that there was a suggestion to replace and rename a file from one source, the Queen source, and then everyone else said I should rebuild. Now, I understand the complexities (a little better - but not really) and everyone's frustration with my wanting to replace a preview or smart preview file. I get it, I'm over it. I hope your guys are as well.

I really do appreciate all the help. You have all inspired me to move forward here with a better 'working' plan for my LR catalog. Hopefully more current backups as well. hahaha.

thanks, d
 
Thanks Cletus,

Either you misread my description above or I was not clear in my writing. I am not moving the catalog to a new computer (at least I hope not anytime soon) but instead building a comprehensive Hard Drive that will contain both the Catalog
But you said
So along with getting this 'working' catalog rebuilt on my laptop I have begun building a duplicate catalog on a brand new 4TB HD that will hold as many associated files as the HD volume will allow.
Rebuild implies “Starting over”. All of you edits, keywords, collections are in the master catalog. Anytime you “start over” you lose the work in the original catalog. You do not want to do that.
The “Catalog” refers to only one file, the “*.lrcat” file All other files and folders support Lightroom Classic and the database that is the catalog File. Get your master catalog (File) right on the current machine first, then you move everything to the new computer. You do I not need to “Duplicate” anything.

I am trying to guide you to use the most efficient methods to achieve your goals. Victoria (the Lightroom Queen) is the expert here and has patiently compiled the most efficient methods working with Lightroom in her books and her blogs. What she has assembled on this website amounts to the collective wisdom of many experienced Lightroom users both within the Adobe community and outside of it. She also has assembled the answers to most of the mistakes that users make in trying to work with Lightroom.
 
Rebuild implies “Starting over”. All of you edits, keywords, collections are in the master catalog. Anytime you “start over” you lose the work in the original catalog. You do not want to do that.
The “Catalog” refers to only one file, the “*.lrcat” file All other files and folders support Lightroom Classic and the database that is the catalog File. Get your master catalog (File) right on the current machine first, then you move everything to the new computer. You do I not need to “Duplicate” anything.

I am trying to guide you to use the most efficient methods to achieve your goals. Victoria (the Lightroom Queen) is the expert here and has patiently compiled the most efficient methods working with Lightroom in her books and her blogs. What she has assembled on this website amounts to the collective wisdom of many experienced Lightroom users both within the Adobe community and outside of it. She also has assembled the answers to most of the mistakes that users make in trying to work with Lightroom.

Cletus,

Allow me to clarify.

By 'rebuild' I meant 'rebuilding the previews' which I have now completed. I have no plans to move to another computer but I have said a couple of times that I am going to make a duplicate of this (current, rebuilt preview) catalog on a new 4TB hard drive and collect all the images, in all the folders, on all my other drives, and put them here on the 4TB in one place. All happy, all together.

Phil has recommended a backup program which will hopefully keep the duplicate back-up catalog on the 4TB up to date.

I hope this clarifies my intentions. If you see any problems with this plan I'm all ears.

Thanks,

d
 
I do have a question about re-associating all the images in the LR catalog to the new single drive. Should I start a new thread or keep going here?

d
 
It won't hurt to carry on here.
 
Great, I'm going to make a couple pictures later this evening to help demonstrate what I would like and what I'm getting as I re-assign files.

I think the pictures will explain better than I can writing.

Cheers,

d
 
Which brings me to a very important question. Is there a very good backup program that can back up the daily changes in LR from one drive to another? Specifically for a Mac in my case.
I would take Chronosync for a try. It does have a file validation option and most synchronization applications don't
 
Phil, I wholly agree with your assessment of Nikon.

Have you used GoodSynch to keep a copy of your evolving LR catalog up to date?

d
Absolutely. I use LR on a desktop at home and a laptop out of the house. I use different GoodSync "jobs" to sync LR settings (on my C: drive), LR catalog (on my D: drive) and bulk photo file store on my E: drive. For both laptop and desktop, drives C: and D: on on an SSD for performance. E: is a normal "spinner" HDD. For the laptop, that's an external WD 4 TB "black" rugged model. Unlike some of the free PC sync programs, Goodsync is bulletproof reliable.

Of course, the mac has a better designed OS, where you don't need to use this 50-year old drive letter system that goes back to the dates of Intel 8080 personal computers running CP/M, a predecessor of DOS 1.0.

I also use Goodsync to also sync all my other data between systems. I do NOT sync program installs. In Windows that would be a big no-no.

For just quick folder by folder comparisons, I use Beyond Compare, from Scooter Software, but I don't think that have a MacOS version.

For true backup, including versioning, I use a commercial backup program called Retrospect. I use Goodsync to sync the backup drive in my system with an EHD that I keep offline, in case I get hit some some ransomware attack that enrypts all my drives. I'm willing to spend reasonable money on utilities that provide real value.

Phil
 
I would take Chronosync for a try. It does have a file validation option and most synchronization applications don't
TimeMachine is the primary system backup for Macs. I would recommend that app as it is included free with your OS. I currently backup 3 volumes to a single (large) backup volume. (I have 7 EHDs mounted to my iMac via TB3 and USB3 ports)
 
Although it seems that attempting to do what I proposed could have been at least worth a try on a duplicate catalog... if done correctly.

When recovering from a disaster, you always have to ask yourself what is of most importance to you. Is it to get back operating as quickly as possible even if it means you can't guarantee the robustness of the recovered system? Or is it to make sure you are the closest that you can possibly be to having a system that looks like it never had a problem even if it takes longer to get there.

I'm not saying that the first option is never the correct one. My background is in telco billing where I've worked on systems that handle 1 billion call records a day. Those companies are not prepared to be down, and therefore non-revenue generating, for an extra six hours to get that recovery quality guarantee. Even in photography, if you are a sports photographer at the Olympics, you are not delaying submitting your images to an agency by six hours because you want a good recovery. That's a surefire way to making sure nobody uses your images.

But, I'd still suggest that for the vast majority of Lightroom users, quality is more important than speed when it comes to recovery.
 
Status
Not open for further replies.
Back
Top