• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Catalogue Corruption Woes

Status
Not open for further replies.

Karen Snedker Black

New Member
Joined
Sep 27, 2016
Messages
10
Lightroom Experience
Intermediate
Lightroom Version
My catalogue is huge - 46+ GB. I shoot RAW, so my library is huge. I do, however, cull my photos and dump those I do not want to keep. I create Smart Previews. The photos I do post-edit and keep, I will transfer to my external drive so that at any given time, generally only my most recent shoots and Smart Previews remain, as well as iPhone photos I may have taken and downloaded.

Due to the size of my catalogue, it takes hours to backup and I am the first to admit I do not do it often enough because of this. I am using Lr 5.7 so it is not zipped.

My OS is a Mac 10.9 Maverick, 4 GB, 2 core.

The last backup I could find on my HD was February, 2016. This does not seem right.

I must mention that during the Spring I was persuaded by a friend to try using Affinity for a month or so, which is what he prefers. I was not happy with it so returned to Lr. Although I set my computer to not have Lr open, it continued to do so when I was importing my photos during that time.

About 5 days ago, I tried to optimize and backup Lightroom in the afternoon. By 11 pm it was still trying. When I woke up the next morning there was a message saying it was not successful and I think it said to try using another catalogue.

Due to catalogue sizes, I only had one backup on my HD so attempted to use it. No luck.

I have gone into file folders and tried cleaning them up within Lr (dumping photos that I had rejected, but not deleted, etc.), and with my external drive attached, tried to 'find' files. It is like Lr is no longer reading my external drive.

I have checked properties and permissions. The 'ignore ownership on this volume' box is ticked at the bottom of my external drive, as instructed in another thread.

I have had no problem opening catalogues, but now there are entire files within Lr missing, and photos not there. It is a mess.

I also had the catalog.lock issue even when Lr was not open. (I have since deleted it).

Here is my question. If push comes to shove, would it be advisable to transfer the remaining photographs not yet on my external drive so that I have all my photos safe, and then start over with a new default catalogue.

I am travelling to Europe in 12 days. I have been struggling with this for close to a week and of course, I also have last minute travel plans I need to deal with. Photography is my sweet spot. I really need to get this fixed so that I can import my photos while I am there.

Help. (Thank you).

Karen
 
Last edited:
Karen,

Start by providing some screen shots of Finder where you have the master catalog. Make sure you have the details view selected which provides the file details.
44 GB sounds huge for a master catalog, but reasonable if you include previews. So we need to separate what the 44GB represents.

And yes, you need to backup more often.

While I am technical, I am not the best at helping explain the steps to resolve/trace the issues you face. So wait for some of our gurus to start helping before you do anything drastic.

Worst case, while waiting to resolve the existing master catalog, you can start a new catalog for the trip and look to merge them when you get back.
 
As suggested by Tspear, I am attaching two screen shots of Finder. This first one shows my catalogue.
Screen Shot 2016-09-27 at 4.44.45 PM.png
 
Yeah, I'm at about 70,000 images and 2.7GB. To be at 46G is a scary large SQLite database.

When you tried to optimize did it give you any more meaningful error than just "not successful". I don't know how much free space is needed for an optimize, but twice the database size does not sound generous, it might be worth freeing up a lot more space.

Do Smart Previews take catalog space? Is that why it is so huge?
 
I just checked and I have just 42,903 photos.

I don't understand why my catalogue is so huge. I have a Smart Preview folder that reads that large as well. This makes no sense to me.

Screen Shot 2016-09-28 at 8.47.57 AM.png
 
Something has occurred to me -- one of the reasons the backup is not working is because of the size of the entire Lightroom folder. I don't have enough memory left.

I've mentioned I shoot RAW which makes my photos large. Johan and Ferguson, are your photos in jpeg format or do you also shoot RAW? I'm struggling to understand why my catalogue is so massive. Surely I'm not the only person who shoots RAW. If not, that means I have a problem.

I have another which I just came to understand last night thanks to this forum.

It has been my habit to transfer picture folders from my HD directly to my 4TB external drive, instead of within Lr. It has never caused a problem before however now, as I try to get my catalogue to work, pictures are missing.

Screen Shot 2016-09-28 at 8.59.48 AM.png
 
The size of the Smart Previews doesn't seem inconsistent with a 43k image catalog, though I am curious as to why there's a "2" at the end of the name. Could you post a regular Finder view (i.e. not the "date last opened" view that you're currently showing) of the contents of the folder that contains the catalog. I'd like to see exactly what you have in there.

The only reason I can think might cause such a huge catalog (43gb for 43k images is rather ridiculous), is if you routinely do hundreds of spot removals or adjustment brush movements. Each adjustment gets added to the history, which is stored in the catalog, and can have a dramatic effect on the overall catalog database size. But you'd need to be doing that for just about every image to cause the catalog to grow to that size. So, do you use the spot removal tool or the adjustment brush to a great extent?
 
Whether you shoot Raw or Jpeg should have little effect on the size of the catalog. Neither the images themselves, nor any of the previews, are stored in the catalog database.
 
The size of the Smart Previews doesn't seem inconsistent with a 43k image catalog, though I am curious as to why there's a "2" at the end of the name. Could you post a regular Finder view (i.e. not the "date last opened" view that you're currently showing) of the contents of the folder that contains the catalog. I'd like to see exactly what you have in there.

The only reason I can think might cause such a huge catalog (43gb for 43k images is rather ridiculous), is if you routinely do hundreds of spot removals or adjustment brush movements. Each adjustment gets added to the history, which is stored in the catalog, and can have a dramatic effect on the overall catalog database size. But you'd need to be doing that for just about every image to cause the catalog to grow to that size. So, do you use the spot removal tool or the adjustment brush to a great extent?


I do make a lot of brush adjustments to add depth and definition to my photos. As for spot removal, guilty as charged for that as well. Unfortunately, somehow, on a shoot at night, a group of us were climbing a trail and dust got in my camera and on the lens and I have not been able to remove it, nor have technicians at a few camera shops I have visited with it. Therefore I do spot removals for each photo I process where needed. (Sky mostly). My camera is a Sony Alpha a58.

Re .lrdata2 -- Yesterday I made a copy of the Smart Previews. If you go back to message #4 above it shows the original smart previews attached to a catalogue (please see message #5) which is the backup catalogue. I've been trying everything I can think of to fix the problem so when my actual catalogue started missing photos, I noticed my catalogue folder did not have a large Smart Previews folder attached yet for some reason the backup did, so I made a copy. (Three days ago I had no problem with photos disappearing, I just couldn't back up my catalogue. It has just progressed to be worse and worse).

Below, the Smart Previews Folder you requested. Thanks in advance (everyone) for your assistance.


Screen Shot 2016-09-28 at 9.32.49 AM.png
 
Status
Not open for further replies.
Back
Top