Missing Lightroom Edits

[PaulTimlett]

Lightroom Version Number

Lightroom Classic version: 9.1

Operating System

I've been going through all my Lightroom images to select work for a new website. I came across some older images (from 2016 and 2017) where the edits have disappeared. The RAW files are there but when I go into the Develop module all the sliders are set to zero and there are no edits whatsoever. I know I've edited them as the edited versions appear in my Flickr account.

There seems to be no pattern to this. For example there is a Fuji RAF file which has jpg and xmp sidecar files, and there is a Leica DNG file with a jpg sidecar file. Neither of these files have any edits although the fully edited versions are in Flickr.

I have Lightroom Classic (subscription) installed on my MacBook Pro. The Lightroom Catalogs are stored on the MacBook, whilst the RAW files are on an external SSD drive as I also occasionally use an iMac for editing. The Catalogs are copied to the iMac so the edits are missing from both the MacBook Pro and the iMac.

I appear to have two Lightroom Catalog files named Lightroom Catalog.lrcat and Lightroom Catalog-2.lrcat. I believe the second one was created during the last Lightroom update.

As far as I can tell so far, only a few files are affected.

Any thoughts as to what has happened and how I can recover the edits would be appreciated. Is this something to do with Catalogs, something to do with where the edit data is stored, or what?

 

[Johan Elzenga]

The only explanation I can think of is that the XMP files are from before you edited the images, or were created by another application, and then one day you used ‘Read Metadata from Files’ and cleared the edits that way. Edits are stored in the catalog, so they will always show if present.

 

[Califdan]

When you say the edits are missing, are you just looking at the image itself or are you looking at the history panel in the develop module? If the history panel shows a significant list of changes (newest are on top) look for a suspicious entry such as "Reset Image" and if you find one, click the line below it and your edits should come back. If, however, the history panel is basically empty, then verify the date of the bottom entry which is the "import" line to see if that corresponds to when you first imported the image or is a much later date.

 

[PaulTimlett]

The history panel shows only the import date. Which is interesting. I'm looking at two images. One shows a capture date of 27 December 2016, the other 1 November 2017. Both show an import date of 5 April 2018! Which suggests I did something on that day to delete all the edits. I've no idea what. Only though I have a huge number of images from around the same capture dates that I've edited and the edits are still there, even though the import date is 5 April. However, the edit history has gone albeit I can see the slider movements.

My oldest Catalog backup is 23 October 2018! Looks like the edits are lost forever.

 

[Philippe Coudé du Foresto]

That suggest to me that you may have had some files marked "missing" by LR and you did an import or a synchronise instead of a "Update location" on April 5th 2018.

 

[PaulTimlett]

That suggest to me that you may have had some files marked "missing" by LR and you did an import or a synchronise instead of a "Update location" on April 5th 2018.

I don't think so. I've now looked at a number of images captured before 5 April 2018. In the Develop module they all show 5 April 2018 as the import date. That may be the date that I bought the MacBook Pro and imported everything from my iMac. It's about the right time. The iMac is a pile of junk so I switched to the MacBook Pro as my main editing tool. I've rebuilt the iMac twice since then (it's still a pile of junk) so my most recent Catalog backup on the iMac is 1 August 2019.

 

[clee01l]

I don't think so. I've now looked at a number of images captured before 5 April 2018. In the Develop module they all show 5 April 2018 as the import date. That may be the date that I bought the MacBook Pro and imported everything from my iMac. It's about the right time. The iMac is a pile of junk so I switched to the MacBook Pro as my main editing tool. I've rebuilt the iMac twice since then (it's still a pile of junk) so my most recent Catalog backup on the iMac is 1 August 2019.

Moving to a new computer means to MOVE the old catalog file. If you did not do that but imported all of your images instead, you have left all of the earlier edits in the old catalog file. Any chance to recover that catalog file form the old computer?


Sent from my iPad using Tapatalk

 

[Philippe Coudé du Foresto]

I've now looked at a number of images captured before 5 April 2018. In the Develop module they all show 5 April 2018 as the import date. That may be the date that I bought the MacBook Pro and imported everything from my iMac

This is exactly what I suggest. After moving to the MPB, you shouldn't have done an import. If you had opened the catalog copied from the iMac, all photo should have had been in the catalog of the MBP.

 

[Hal P Anderson]

I bought the MacBook Pro and imported everything from my iMac.

Did you actually do a Lightroom Import on that date? The normal thing to do is to copy your catalogue and images to the new machine, not to import everything into a new catalogue.

If you had written XMP data to most, but not all image files, you would see what you're seeing if you did indeed re-import everything.

 

[PaulTimlett]

Did you actually do a Lightroom Import on that date? The normal thing to do is to copy your catalogue and images to the new machine, not to import everything into a new catalogue.

If you had written XMP data to most, but not all image files, you would see what you're seeing if you did indeed re-import everything.

I'm pretty certain that I just copied all the images and the Catalogs across. I know that's what you have to do so I can't imagine I'd have done anything different. I'm pretty certain I kept both the MBP and the iMac running side-by-side for at least a few days to ensure everything was working. I also had a Time Machine backup of the the iMac.

When downloading Lightroom onto the MBP I guess it created a new Catalog on the machine but I'm pretty certain I then copied the old iMac Catalog across as I knew that was where all my edits were. And as I said earlier, the overwhelming majority of images have retained all their edits. It's only a small number of images where the edits have gone.

 

[PaulTimlett]

Moving to a new computer means to MOVE the old catalog file. If you did not do that but imported all of your images instead, you have left all of the earlier edits in the old catalog file. Any chance to recover that catalog file form the old computer?


Sent from my iPad using Tapatalk

I have most of the edits. Only a small number of images have lost the edits.

 

[clee01l]

I have most of the edits. Only a small number of images have lost the edits.

The fact that these images shows the import date in Develops as 5 April , 2018 means that you imported (reimported) these image files on that date. If you have on other files in the catalog with this same file name, then you have lost any work done on an image file with that name that was done before that date. Unless you can find the backup catalog file before that date that shows those files with edits.

 

[PaulTimlett]

The fact that these images shows the import date in Develops as 5 April , 2018 means that you imported (reimported) these image files on that date. If you have on other files in the catalog with this same file name, then you have lost any work done on an image file with that name that was done before that date. Unless you can find the backup catalog file before that date that shows those files with edits.

What I don't understand is that the vast majority of files that were created before 5 April 2018 show the final edited version, albeit the history has gone. So far I've found four that I know have been edited but all I can see is the original RAW file.

So for example, let's say I cropped the file and converted it to black and white. With most of these files I can see the crop and the black and white conversion, although in the History all the steps have gone. Nevertheless I still have the final edit. But with the four I've traced so far not only has the History disappeared but so has the final edited version. It's almost as though they have been reset (they haven't).

I could understand it if all of the files created before 5 April 2018 appeared as though they had been reset, but most show the final edit. I don't understand why only four appear as though they've been reset. Doesn't that suggest something else is going on?

I can live without the History but three of the four that appear to have been reset took a lot of time to edit. I guess at least have the final edited version in Flickr.

 

[clee01l]

What I don't understand is that the vast majority of files that were created before 5 April 2018 show the final edited version, albeit the history has gone.

I think Hal has said why this could happen: "If you had written XMP data to most, but not all image files, you would see what you're seeing if you did indeed re-import everything." If you reimport an image file and there is am XMP sidecar file or if it were DNG or JPEG, the XMP data would have been recorder in the file header. On import (reimport) LR reads the associated XMP data and applies that on import. History steps are only stored in the catalog file and never follow the original image file.

Instead of trying to figure out what might have happened. you might concentrate on what might be done to recover the missing edit history. DO you have a backup catalog that precedes the data loss?

 

[PaulTimlett]

I think Hal has said why this could happen: "If you had written XMP data to most, but not all image files, you would see what you're seeing if you did indeed re-import everything." If you reimport an image file and there is am XMP sidecar file or if it were DNG or JPEG, the XMP data would have been recorder in the file header. On import (reimport) LR reads the associated XMP data and applies that on import. History steps are only stored in the catalog file and never follow the original image file.

Instead of trying to figure out what might have happened. you might concentrate on what might be done to recover the missing edit history. DO you have a backup catalog that precedes the data loss?

OK. I don’t fully understand the explanation as I don’t have sufficient knowledge of how Lightroom works. I’m as certain as I can be that I didn’t reimport the files but it’s 20 months ago so maybe I did. But it seems there’s nothing I can do about it now because as advised earlier I don’t have a backup that predates the loss. I checked that before I posted this thread.

Lesson learned I guess although I did research extensively before I made the switch.

 

[clee01l]

Lesson learned I guess although I did research extensively before I made the switch.

Often people ask "How Many Backups should I keep ?" My somewhat flippant answer is far enough back to cover your last "Stupid User Mistake" We all make'em . I have gone back as far as 6 month to recover lost keywords when I removed keywords from over 2000 image when I thought I was only removing them from the one image I saw on the screen. not recognizing there were over 2000 selected. If you back up daily or weekly you can at some point retain one one per month or one per quarter. and delete yer other intervening backups.

If you ever have a Lightroom process that you have never undertaken before, feel free to come to this forum and ask questions. Some one here would have been more than happy to hold your hand through the process. Victoria has lots of Blog articles covering all sorts of Lightroom issues. I would be willing to guess that it has been a very long time since someone came here with a problem that was unique and no one else had brought it up before.

 

[PaulTimlett]

Often people ask "How Many Backups should I keep ?" My somewhat flippant answer is far enough back to cover your last "Stupid User Mistake" We all make'em . I have gone back as far as 6 month to recover lost keywords when I removed keywords from over 2000 image when I thought I was only removing them from the one image I saw on the screen. not recognizing there were over 2000 selected. If you back up daily or weekly you can at some point retain one one per month or one per quarter. and delete yer other intervening backups.

If you ever have a Lightroom process that you have never undertaken before, feel free to come to this forum and ask questions. Some one here would have been more than happy to hold your hand through the process. Victoria has lots of Blog articles covering all sorts of Lightroom issues. I would be willing to guess that it has been a very long time since someone came here with a problem that was unique and no one else had brought it up before.

Many thanks, really appreciate the helpful advice I’ve had here today.

My backup methodology is pretty much as you outlined. Unfortunately the MBP and iMac backups only go back a couple of years.

HOWEVER, I’ve just logged into our ancient iMac that my wife uses to watch TV. I found a Lightroom Catalog.lrcat file on there dated 16 April 2017 and backups dating back to 6 February 2017. I also found a Lightroom 5 Catalog.lrcat file dated 10 October 2016 so my missing edit information could be in those files I guess? Question is though how the hell do I recover data relating to maybe only four images? I assume if I copy those Catalogs to my MBP it will duplicate what’s already on the MBP?

 

[PaulTimlett]

OK I’ve just tried to recreate what I may have done in April 2018 and I think everyone is right. I must have imported the images. I think what I did was to copy them from my iMac to a portable hard drive then imported them from there into Lightroom on my MBP. I must have assumed that since I was copying the Catalog files from the portable drive to my MBP I would be retaining all the edits. I needed to understand what I did for the sake of my sanity!

Either way, as I said above I have found some old Catalogs contemporary with the missing edits on an old iMac. Can anyone tell me how I can recover the edit information for the small handful of files where the edits are missing without messing up all the remaining thousands of images I have?

 

[Johan Elzenga]

Either way, as I said above I have found some old Catalogs contemporary with the missing edits on an old iMac. Can anyone tell me how I can recover the edit information for the small handful of files where the edits are missing without messing up all the remaining thousands of images I have?

Open the old catalog, locate the files (place them in the quick collection for example) and export these a a new catalog. As you do not have to include the 'negatives' when you export the catalog, it should be OK if the images are shown as 'missing'. Then import this catalog into your main catalog using 'Import from Another Catalog'. Lightroom should tell you it has found images that are already in the catalog, and ask you what to do with the edits it has found with these images.

 

[clee01l]

Either way, as I said above I have found some old Catalogs contemporary with the missing edits on an old iMac. Can anyone tell me how I can recover the edit information for the small handful of files where the edits are missing without messing up all the remaining thousands of images I have?

1. Make a copy of your master catalog so that we can fall back if we somehow screw up.
2. Open the most recent catalog file (you might need to unzip it first). If it is not the current version of the LR app, LR will want to convert that catalog file and make a copy that conforms to the current catalog structure. In this catalog search for a file that you know should have the missing edits. Don't worry about missing image files what you want to looks for is that edit History. If it does and you can verify that most of the edits are there, we have a candidate for merging. If not select the next most likely candidate and repeat the process.
3. Once you have a catalog file to recover from, you will want to go back to the current master catalog and identify all of the images that have missing history. It is best to accumulate a comma separated list of file names because we are going to use that list to extract these image files with history from the catalog that you want to recover from.
4. In the recovery catalog with your comma separated list you are going to past the comma separated list of file names into the filter bar as: "Text: {Filename}{Contains} {[filename1,filename 2, filename3, etc.}
5. Select all of the files that are found by the filter and choose "Export as a Catalog ". Leave all of the checkboxes in the export dialog unchecked and you will create a catalog file only.
6. Once created. Close the recovery catalog and open the Master catalog.
7. In the Master catalog choose the "import from another catalog" function and choose the export catalog that you just created.
8. In the dialog that opens, check the box labeled "Preserve old settings and a virtual copy".
9. Import the data from the "Export" catalog.
10. This will create virtual copies for the files found to be existing (hopefully, just those that have no edit history)
11. You can review these virtual copies and delete them if the history stack only shows the 2018 imported date/

 

[PaulTimlett]

Many thanks. In think I’m going to have to go through this very slowly as I don’t fully understand some of the steps. It’s late here in the UK so I’ll do it tomorrow or Monday. First I’ll have to take the old catalog files off the old iMac and transfer them onto my MBP. I assume I can keep all the different catalogs in the same folder on my MBP?

One final question for now, when you say open the catalog how exactly do I do that? Do I open Lightroom then point it to the recovery catalog candidate before opening an image in Lightroom? Or do just double click on it in Finder?

 

[Philippe Coudé du Foresto]

Instead of taking note of the concerned filenames (steps 3 and 4), I would use a target collection

3a - Create a collection. Name it "Recovery"
3b - Right click on the collection and select "Set as target collection"
4 - when you find an image to be recovered, select and press the key "B" to add it to the target collection (Recovery)
5 - Once all files to be recovered have been added to the target collection, right click on this Recovery collection and select "export as catalog".

 

[PaulTimlett]

Instead of taking note of the concerned filenames (steps 3 and 4), I would use a target collection

3a - Create a collection. Name it "Recovery"
3b - Right click on the collection and select "Set as target collection"
4 - when you find an image to be recovered, select and press the key "B" to add it to the target collection (Recovery)
5 - Once all files to be recovered have been added to the target collection, right click on this Recovery collection and select "export as catalog".

Thanks., that sounds like good advice. I wasn’t sure how to make the comma separated list.

 

[clee01l]

Instead of taking note of the concerned filenames (steps 3 and 4), I would use a target collection

3a - Create a collection. Name it "Recovery"
3b - Right click on the collection and select "Set as target collection"
4 - when you find an image to be recovered, select and press the key "B" to add it to the target collection (Recovery)
5 - Once all files to be recovered have been added to the target collection, right click on this Recovery collection and select "export as catalog".

The problem with this approach is that the files that wind up in the target collection, do so in the master catalog. It is the names of those image files that are needed in the backup catalog. It is from the backup[ catalog that you want to generate the “Export as a Catalog” subset of images to be merged into the master catalog.

I don’t really care how you identify the files in the master catalog that have lost edit history. You can do this with a target collection or as I suggested simply generate a comma separated list of file names that can be used to filter the backup catalog to extract these same files. Ultimately you have to find the affected files (by name seems easiest) in the backup catalog.


Sent from my iPad using Tapatalk

 

[PaulTimlett]

The problem with this approach is that the files that wind up in the target collection, do so in the master catalog. It is the names of those image files that are needed in the backup catalog. It is from the backup[ catalog that you want to generate the “Export as a Catalog” subset of images to be merged into the master catalog.

I don’t really care how you identify the files in the master catalog that have lost edit history. You can do this with a target collection or as I suggested simply generate a comma separated list of file names that can be used to filter the backup catalog to extract these same files. Ultimately you have to find the affected files (by name seems easiest) in the backup catalog.


Sent from my iPad using Tapatalk

OK thanks for that. I’ll do as you suggest. However, where do I generate the comma separated list? Just in a Word document?