Missing Photos - RENAMED originals

[FJLR]

Dear All,
I have been reading about my problem quite some time now, until now I didn`t find any solution.

Here is the problem:
After importing images from a vacation I did a backup. Then I started to work on those photos, rate, keyword, did some selectiv processing and RENAME the photos using LR.
In the mean time I did not backup again (not good). Then my photo hard drive gave in. I was quite happy because I had a backup, until I discovered that Lightroom has problems finding photos that have different names on the hard drive compared to the catalog (remember, I renamed the files I lost with the hard drive).

This is way I tryed to solve the problem:
- verify images from faulty hard drive could not be retreived
- copy photos from backup onto new drive
- direct Lightroom to new photo location (LR and Photos are on different drives)
- check how the catalog is doing
> everything exept the folders from the last vacation are ok.

what now?
- select the missing folder and redirect it to folder on new hard drive.
- everything seems fine, check the photos = oh no! still missing
- relocating single files works, but it does not find nearby missing photos

Everything I have learned until now is that my big problem is the different names.
Everybody told me I will have to relocate all the files by hand or reimport and do everything over.
With over 2000 photos I hope there is a easier way.

I was thinking of a prog / plugin that could compare the metadata and make the connection. Good idea/ bad idea?

Any other suggestions, workarounds?
Thanks FJ

Screenshots

 

[Selwin]

Hi FJ, welcome to the forum! We're going to help you out to the best of our abilities.

Judging from your screen shots, you renamed all images the same way. The easiest solution to our problem is to rename your originals (using LR) again. This way you will end up with a new set of originals that will be recognised by your catalog. Another possibility of course is to use 3rd party renaming software.

For the LR option, please follow the following steps:

1. First, backup your current catalog!
2. then, you'll need to create a new catalog to work with. Select New Catalog from the File menu
3. copy the problematic folders to a new location
4. then import the that folder or folders into the new catalog
5. Rename all files exactly how you did the first time. You won't need the new catalog anymore.
6. Open your original catalog and use "update folder location" to direct LR to the new folder with renamed photos

Sometimes, when one or more of the individual missing individual files already have been relinked, the folder update command may not work correctly. If this should be the case then close the catalog, copy back your backup (step 1), and repeat the process without the already relinked photos. You can move them to another (temp) folder and move them back later.

All should be fine now.

 

[FJLR]

Hi Selwin,

Thanks a lot,
Yes, all my renaming is done in a similar fassion. I will try this and come back.

 

[FJLR]

Ah, this would have been to nice.
I tryed one folder and realised that I have deleted photos in the original catalog.
If you compare this screenshot with the top one, you will see that the photo with the donky car has a different number.



Comparing the photocount 167 (original catalog) with 358 in new catalog.

Will this cause problems when doing step 6?

(I allways rename photos after rating them and deleting photos I dont need)

 

[Selwin]

Well that complicates things indeed. However, in your screen shot I don't see missing photos. Did you alreay relink all of what I see there?

another thing that I noticed is that the third image in the top row has no NEF, it's jpeg only. Did you develop your jpegs or your NEFs? Why is that NEF not present?

The problem now is that your file numbering differs from the originals.

I'll need to chew on this one. Let me get back to you later. Meanwhile, maybe one of the gurus chimes in with a good and quick solution.

 

[FJLR]

However, in your screen shot I don't see missing photos. Did you alreay relink all of what I see there?

This is the screenshot of the catalog I created in step 2. I realised somthing might be fishy and did not go on with step 6.

another thing that I noticed is that the third image in the top row has no NEF, it's jpeg only. Did you develop your jpegs or your NEFs? Why is that NEF not present?

I found the NEF, I moved it dow the pile befor renaming. "Added Order" was enabeled.
For this vacation I shot JPEG+RAW. Imported all the images, rated them and got rid of all images I didnt like, including of most JPEGs. Then renamed the remaining photos wit F2.

I'll need to chew on this one. Let me get back to you later. Meanwhile, maybe one of the gurus chimes in with a good and quick solution.

Thank you for your help so far! I hope you or anyone else will have more ideas.

 

[camner]

There is probably an easier way, but one thought that comes to me is to create dummy files with the missing file names (e.g. if IMG_0120 and IMG_0121 are missing because they have been deleted, create dummy files with those file names and then either inside or outside of LR use the mass renaming technique), and then one can delete the dummy files. Since LR5 will recognize different files one could make the dummy files, say, .png files (assuming none of the real files have that extension) which would make them easy to find and delete at the end of the process.

The more files you have deleted the more tedious this technique will be.

Another thought...if the files were originally named in LR, wouldn't the original file name be in the "original file name" field in the metadata? (This assumes the renaming was done AFTER import...if the renaming was done during import, the "original file name" is the renamed version.)

 

[Selwin]

FJ, some ideas:

Renaming your images back to their originals is only possible if the originals aren't missing.

Another idea may be that Lightroom created an automated backup, coincidentally right after you finished your selections and keywording and editing, but just before renaming them. You could check those automated backups (if you have them).

 

[johnbeardy]

I suspect the best approach is going to be with this plug-in http://www.robcole.com/Rob/ProductsAndServices/RelocationServicesLrPlugin/ .

I don't have much useful communication with its author... but I think you use your correct catalogue and tell the plug-in where missing files are located. It can find them even if the names don't match. If it finds a file, it copies it to where Lightroom thought the file was, and saves it with the renamed name too (if that makes sense). So essentially it uses the backup folders to reconstruct the folders you lost.

I think you should investigate it. Probably best to contact the plug-in author directly.

John

 

[camner]

I suspect the best approach is going to be with this plug-in http://www.robcole.com/Rob/ProductsAndServices/RelocationServicesLrPlugin/ .

I don't have much useful communication with its author... but I think you use your correct catalogue and tell the plug-in where missing files are located. It can find them even if the names don't match. If it finds a file, it copies it to where Lightroom thought the file was, and saves it with the renamed name too (if that makes sense). So essentially it uses the backup folders to reconstruct the folders you lost.

I think you should investigate it. Probably best to contact the plug-in author directly.

John

Great idea! But does Cole's plugin work on filenames (to identify the image) or some other data?

 

[camner]

Renaming your images back to their originals is only possible if the originals aren't missing.

Right you are.

Another possible idea…If the catalog exists and the sidecar XMP files are in the right place (assuming these images started as RAW files), perhaps EXIFTOOLs can read the original file name from the XMP files (if LR writes out the original file name data into the XMP file…I know LR does NOT write ALL data into the XMP files) and can copy them back to the file name field?

 

[Selwin]

Ha, I didn't know about that plugin. Some things can't be solved using basic Lightroom functionality. Good luck with the plugin.

 

[Selwin]

Great idea! But does Cole's plugin work on filenames (to identify the image) or some other data?

Apparently it can if I read Rob's instructions correctly.

 

[Selwin]

FJ, after you cleaned up your file name problem, may I make some suggestions?
1. Consider renaming your files on Export, not your originals. If your workflow takes place all inside Lightroom, it doesn't really matter how the file names are called. If you export to jpeg and rename at the same time, you will have a folder with file names the way you want them.
2. Consider not treating your jpegs as separate files. To me it doesn't make sense to have them displayed and certainly not to assign different file numbers to them than the RAW files they belong to. You can change this behaviour in Preferences.
3. Have a look at a backup article I wrote, in conjunction with Victoria's blog. You may think your situation is just a case of bad luck, your drive dying on you, but actually a good backup regime is just there for these occasions.

I hope you'll take my suggestions as friendly advice and not blunt critique .

Good luck!

 

[johnbeardy]

Great idea! But does Cole's plugin work on filenames (to identify the image) or some other data?

I believe it uses other data, maybe even Exiftools to do its comparison.

The OP will have to see if it does what I think - I try to ignore its author when he's rabbiting on about his countless plug-ins! I just remembered it because this one solved what I think is a real problem.

And yes, I've asked Adobe many times to make the missing files routine a bit less pedestrian....

John

 

[FJLR]

Hi all,
checked back this minute.
Thanks for all the imputs, you are very helpfull.
I am off to other commitments for today and will have a look on all the individual posts tomorrow.
Thanks again

 

[FJLR]

Another possible idea…If the catalog exists and the sidecar XMP files are in the right place (assuming these images started as RAW files), perhaps EXIFTOOLs can read the original file name from the XMP files (if LR writes out the original file name data into the XMP file…I know LR does NOT write ALL data into the XMP files) and can copy them back to the file name field?

I lost the XMP files together with the photos when my drive died. So no luck there

 

[FJLR]

I believe it uses other data, maybe even Exiftools to do its comparison.

The OP will have to see if it does what I think - I try to ignore its author when he's rabbiting on about his countless plug-ins! I just remembered it because this one solved what I think is a real problem.

And yes, I've asked Adobe many times to make the missing files routine a bit less pedestrian....

John

I checked the Link provided, when looking at the FAQ I found this:


Still, I will make contact with Rob and see if he has something that will help.

FJ

 

[FJLR]

FJ, after you cleaned up your file name problem, may I make some suggestions?
1. Consider renaming your files on Export, not your originals. If your workflow takes place all inside Lightroom, it doesn't really matter how the file names are called. If you export to jpeg and rename at the same time, you will have a folder with file names the way you want them.
2. Consider not treating your jpegs as separate files. To me it doesn't make sense to have them displayed and certainly not to assign different file numbers to them than the RAW files they belong to. You can change this behaviour in Preferences.
3. Have a look at a backup article I wrote, in conjunction with Victoria's blog. You may think your situation is just a case of bad luck, your drive dying on you, but actually a good backup regime is just there for these occasions.

I hope you'll take my suggestions as friendly advice and not blunt critique .

Good luck!

Thanks Selwin,
rethinking the routine once in a wile is not to bad


For a quick explanation why I do things the way I do it.
1. Before using Lightroom I had a DAM prog I trusted all my files to. Then the company stoped supporting the prog and I wanted to export my files to use with LR. Using the program for this was very tedious. So I wanted to find the files with explorer. I found the images folder, but everything was arranged differently in the folders than in the catalog shown by the DAM Prog.
From that day on I want to know into which folder the file belongs by looking at the name, using explorer, and my LR folder tree has to look the same as the explorer tree.


But maybe adding the original file name into the new file name will help in such cases in future …


2. I like to think of my RAW files as my negatives. Now and again I shoot with a picture style and want to see the JPEG image in my LR catalog. If I don’t like it I can redevelop the raw.
This approach proved especially helpful since I have the Fuji x20. I like the Velvia look, and LR does not give the possibility to switch between profiles for the x20, as I am used to with the Nikon DSLRs I am using. I know I could create profiles myself …


I would love it if LR would stack JPEG+RAW photos automatically and give the possibility to add Metadata to both photos in one go. At the moment I am using syncomatic for this. And then stacking by capture time.


3. I will certainly have a look.

 

[johnbeardy]

I checked the Link provided, when looking at the FAQ I found this:
View attachment 4334

Still, I will make contact with Rob and see if he has something that will help.

FJ

That would certainly be a problem! But I think I read him saying in Adobe's forum that he had extended the plug-in to allow for exactly the kind of problem you're facing. That's the sole reason I remember it!

John

 

[camner]

Thanks Selwin,
rethinking the routine once in a wile is not to bad


For a quick explanation why I do things the way I do it.
1. Before using Lightroom I had a DAM prog I trusted all my files to. Then the company stoped supporting the prog and I wanted to export my files to use with LR. Using the program for this was very tedious. So I wanted to find the files with explorer. I found the images folder, but everything was arranged differently in the folders than in the catalog shown by the DAM Prog.
From that day on I want to know into which folder the file belongs by looking at the name, using explorer, and my LR folder tree has to look the same as the explorer tree.


But maybe adding the original file name into the new file name will help in such cases in future …


2. I like to think of my RAW files as my negatives. Now and again I shoot with a picture style and want to see the JPEG image in my LR catalog. If I don’t like it I can redevelop the raw.
This approach proved especially helpful since I have the Fuji x20. I like the Velvia look, and LR does not give the possibility to switch between profiles for the x20, as I am used to with the Nikon DSLRs I am using. I know I could create profiles myself …


I would love it if LR would stack JPEG+RAW photos automatically and give the possibility to add Metadata to both photos in one go. At the moment I am using syncomatic for this. And then stacking by capture time.


3. I will certainly have a look.

If you use LR to rename files AFTER import (and as part of the import process), LR populates the "original filename" field with (of course!) the original filename, so that if you have a catalog, you always have access to the original filename.

Also, you can use the Autostack feature with the time interval set to 0 to get LR to stack jpgs with their corresponding raw files.

 

[johnbeardy]

If you use LR to rename files AFTER import (and as part of the import process), LR populates the "original filename" field with (of course!) the original filename, so that if you have a catalog, you always have access to the original filename.

The sad thing is that you can't do anything with it. So if you lose your renamed files, LR won't try using this original filename as part of its routine to find missing files (eg in the Imported on YYMMDD folders). How silly is that?

Conceivably it's used by that plug-in, though I think it uses other data.

John

 

[areohbee]

Sorry guys & gals - at the moment, Relocation Services does still rely exclusively on filename for making the association. But I agree that it may be possible to look at some other info. I think OP said XMP files are gone, so would catalog be the only place those exist? (note: SDK does not allow plugins to read original filename - if it did, we'd not be having this conversation ). So it may be possible to use SQL (e.g. via SQLiteroom - one of my "countless" plugins - got any carrots John?) to drum up those original filenames. I assume coming at it from the other angle has already been considered and ruled out - i.e. using a 3rd party renaming tool to rename files before using Relocation Services - is that correct?

 

[Selwin]

So we are at a dead end. Lightroom could rename the images back to the original filename, but only if the original with the current filename (i.e. the renamed original) exists, which doesn't. We haven't heard from FJ if the original filename field in the EXIF/IPCT holds any data, but let's assume that it's there.

Here is the workaround: find some software (Excel with VBA would do) that takes one original (doesn't matter which one) and copies it using FJ's renaming convention. Let's say you have 200 "keepers" and 20 dates. For each date you create the full range of file numbers:

20130126_ReiseNamibia_0001.NEF to 20130126_ReiseNamibia_0200.NEF
20130126_ReiseNamibia_0001.JPG to 20130126_ReiseNamibia_0200.JPG
20130127_ReiseNamibia_0001.NEF to 20130127_ReiseNamibia_0200.NEF
20130127_ReiseNamibia_0001.JPG to 20130127_ReiseNamibia_0200.JPG
.
.
.
20130216_ReiseNamibia_0001.NEF to 20130216_ReiseNamibia_0200.NEF
20130216_ReiseNamibia_0001.JPG to 20130216_ReiseNamibia_0200.JPG

Every possible renamed original is now in this folder, along with lots of others that we don't need.

Set LR to point to this folder and it will "think" the originals are back. Now you can "rename to original filename" and your catalog will be able to read the originals in your backup folder.

Not as simple as I would have liked it to be, but doable and certainly quicker than redevelop.

 

[johnbeardy]

When I've encountered similar problems, I have always fixed them by SQL to put the original filename value in the place of the current filename. But I think the ideal way would be the method I described. I just wish I knew why I thought Rob's plugin did it!

John