Library module Consolidating library from multiple external hard drives

[T-time]

Lightroom Version Number

14.0.1

Operating System

1. Windows 11

Hello,
My current library is a MESS (how many times have you heard that?). I have about 25k photos on three drives. Depending on which drives are connected, several folders show "missing photos". During a recent update LR failed to save my catalog. Fortunately I had a (not too recent) backup but lost a bunch of work and I had to "find missing photos" for all of my folders (several hours of work). For obvious reasons I want/need to consolidate to a single drive which happens to be one of the three drives I'm currently using (SanDisk 4TB). I'm not sure where/how to start (besides making sure that my catalog is backed up!). Can I move my entire library at the same time over to the SanDisk drive then update the folder locations? Or, do I need to move one drive at a time to make sure the folders are visible to LR and not showing as "missing" when I move them? What is the safest and most assured way to accomplish this? It's all my own fault. I've made mistakes from the start and I just cannot relive the inconveniences again and really need to get this all squared away. Thank you to anyone who can help.

 

[clee01l]

Can I move my entire library at the same time over to the SanDisk drive then update the folder locations? Or, do I need to move one drive at a time to make sure the folders are visible to LR and not showing as "missing" when I move them? What is the safest and most assured way to accomplish this? It's all my own fault. I've made mistakes from the start and I just cannot relive the inconveniences again and really need to get this all squared away. Thank you to anyone who can help.

Lightroom Classic doesn't really care where you store your images. I think the first thing you need to do is make sure LrC has a valid up to date path for all of the images in your master catalog. That means "No missing images". Once you have done that, make a backup of your catalog file So that you can fall back if necessary.
If you are comfortable with using Windows Explorer to move your images and the LrC folder panel to find the relocated (missing) folders, then this is probably the quickest way to get your image files and folders where you want them.

The Most assured way is to use the folder panel to move folders from one volume to another using "drag 'n drop". This second method lets Lightroom Classic do both the move and an automatic update of the folder path. Of the two methods, it is probably the slowest but takes less user input.

 

[PhilBurton]

Hello,
My current library is a MESS (how many times have you heard that?). I have about 25k photos on three drives. Depending on which drives are connected, several folders show "missing photos". During a recent update LR failed to save my catalog. Fortunately I had a (not too recent) backup but lost a bunch of work and I had to "find missing photos" for all of my folders (several hours of work). For obvious reasons I want/need to consolidate to a single drive which happens to be one of the three drives I'm currently using (SanDisk 4TB). I'm not sure where/how to start (besides making sure that my catalog is backed up!). Can I move my entire library at the same time over to the SanDisk drive then update the folder locations? Or, do I need to move one drive at a time to make sure the folders are visible to LR and not showing as "missing" when I move them? What is the safest and most assured way to accomplish this? It's all my own fault. I've made mistakes from the start and I just cannot relive the inconveniences again and really need to get this all squared away. Thank you to anyone who can help.

Sometimes you contain a can or worms by buying a bigger can. Several years ago, I did a post on this topic, and i'm going to try to recreate that post from memory.

Please wait until some of the gurus review this post before following these steps.
Step 1. Estimate your total storage need based on the size of your current external drive storage, plus a reasonable allowance for say 3-5 years of new photos.
Step 2. Purchase a new external drive big enough to contain your current library, if necessary.
Step 3. Create top-level folders for all the drives from which you are going to migrate folders. Name them something like drive1, drive2, and drive 3 if necessary.
Step 4. Use Windows to copy, not move, all photos from the first drive to drive1. Repeat for drive 2 and drive 3.
Step 5. Start Lightroom and note all the missing photos.
Step 6. Use this blog post to "find" all the missing photos. https://www.lightroomqueen.com/premium-classic/troubleshooting/missing-files/
Step 7. Optional Depending on your folder naming scheme, you may want to merge all the folders in the folders for drive 1, drive 2 (and drive 3) folders into a new top level set of folders. For a date-based folder scheme, this step should be straightforward. Again, follow the blog post in step 6.
Step 8. Once you are completely confident that you have completed steps 1-7, you can set aside and repurpose the old drives.

 

[Califdan]

Step 3. Create top-level folders for all the drives from which you are going to migrate folders. Name them something like drive1, drive2, and drive 3 if necessary.

this would be on the destination drive. If the images on the old drives already have a common parent on each drive this may not be required unless they all used the same name.

Step 4. Use Windows to copy, not move, all photos from the first drive to drive1. Repeat for drive 2 and drive 3.
Step 5. Start Lightroom and note all the missing photos.

As you "copied" the image folders, they are also still on the old drives so LrC will not show them as missing. Before you launch LrC, disconnect the Old drives leaving only the desitnation drive connected. Now, LrC will show those folders as missing

There are basically two methods you could employ. One is to move the image folders with Windows File Explorer and relink them inside of LrC (as Phiol described) better if they already descend from a single parent folder on each drive) or if you have large numbers of images. The other method is to use the LrC folders panel to move them (better if they don't have a common parent folder. or there aren't all the many folder, or you want to do it bit by bit.

Another option if your image folders don't have a common parent is to put them under a common parent on each drive first using LrC folders panel then move them with Windows and relink in LrC.

One thing I suggest is to clean up all missing folders and images BEFORE you start this merge. It is also a good idea to see what images are in your folders but are not in the LrC catalog. There are a couple of ways to do this depending on which method you use for getting them all on one drive which we can go over if you wish.

 

[Paul McFarlane]

Here's the blog post that details the steps:

How do I consolidate photos from multiple drives?​

 

[T-time]

Lightroom Classic doesn't really care where you store your images. I think the first thing you need to do is make sure LrC has a valid up to date path for all of the images in your master catalog. That means "No missing images". Once you have done that, make a backup of your catalog file So that you can fall back if necessary.
If you are comfortable with using Windows Explorer to move your images and the LrC folder panel to find the relocated (missing) folders, then this is probably the quickest way to get your image files and folders where you want them.

The Most assured way is to use the folder panel to move folders from one volume to another using "drag 'n drop". This second method lets Lightroom Classic do both the move and an automatic update of the folder path. Of the two methods, it is probably the slowest but takes less user input.

I appreciate the feedback. My head is a bit foggy from all of the frustration and uncertainty at the moment but I think I'll be able to come up with a solution using everyone's input. Thanks again.

 

[T-time]

I really appreciate everyone taking the time to offer the help. Let me be more specific about my problem and concern. I've attached a screenshot to share what I'm looking at. When I created this LR catalog I didn't move my entire photo library in. I "cherrypicked." Then, as I moved along, I imported photos from an SD card into LR and added to my library which is stored on external drives. As each of those drives began to fill up (i.e. the Toshiba 1TB drive shown in the screenshot), I moved to another one. So now LR is having to search three different drives for my photos. I can only have two drives connected at a time. So, at any given time, LR will likely be showing some photos as missing. My Samsung 2TB drive is the most complete and up to date representation of my entire photo collection (those linked to LR and those that are not). I want to move everything to my newest SanDisk 4TB drive which LR is already recognizing because I moved a backup folder there from the Samsung drive (to make additional room for storage) which has photos that LR recognizes. I guess this is where my confusion and frustration lies: as shown in the attachment there are missing photos but I know that those photos are on another drive (SanDisk). Most of the rest of the folders shown in the attachment have been found and have no issues, MOST. With this updated information I've provided, can I still follow everyone's recommendations in this thread or does this create new issues or concerns?

 

[Califdan]

As each of those drives began to fill up (i.e. the Toshiba 1TB drive shown in the screenshot), I moved to another one. So now LR is having to search three different drives for my photos

LrC does not have to search drives when deal with images. It has a database called the Catalog which contain a pointer to the physical drive/folder location on disk for each image. In most cases all the info it needs is in the catalog so doesn't even need to look at the image on disk and on the occasions where it does, it goes straight to the image using the pointer (no searching involved)

But, back to your problem, Start by linking all the missing folders. Then exectute the "find all missing photos" command to assure there aren't stray missing images. When relinking (finding) a folder or image you may find that LrC tells you "Already in catalog" or "Image associated with another image in catalog". In this case make sure the other versin in LrC is not missing. Then decide which one you want (check to see which one has the better edtis and metadata) and go with it getting rid of the duplicate.

Now, using the folders panel in Lrc, you can drag and drop folders to move them to the larger drive.

Along the way at some point, you may want to run the "Synchronize" command on the folders to see if there are other images in those folders that are not known to LrC and if so decide what (if anything) to do about that.

 

[T-time]

LrC does not have to search drives when deal with images. It has a database called the Catalog which contain a pointer to the physical drive/folder location on disk for each image. In most cases all the info it needs is in the catalog so doesn't even need to look at the image on disk and on the occasions where it does, it goes straight to the image using the pointer (no searching involved)

But, back to your problem, Start by linking all the missing folders. Then exectute the "find all missing photos" command to assure there aren't stray missing images. When relinking (finding) a folder or image you may find that LrC tells you "Already in catalog" or "Image associated with another image in catalog". In this case make sure the other versin in LrC is not missing. Then decide which one you want (check to see which one has the better edtis and metadata) and go with it getting rid of the duplicate.

Now, using the folders panel in Lrc, you can drag and drop folders to move them to the larger drive.

Along the way at some point, you may want to run the "Synchronize" command on the folders to see if there are other images in those folders that are not known to LrC and if so decide what (if anything) to do about that.

So, I'll probably have to go one drive at a time, link the missing folders and move them over to the new drive? I've heard mixed things about drag and drop vs. copy. What do you recommend if I can't update a location or find the missing photos? I've encountered a couple of those issues so far.

 

[Califdan]

Windows File Explorer will automatcally use "Copy" when you drag and drop to a different drive but will use "Move" if the source and destinations are on the same drive.

I'm pretty sure that LrC drag and drop in the Folders panel always does a Move. (you can check this with one photo or folder)

Copy is safer in that if something goes wrong you still have the source left untouched. However, Copying using Windows File Explorerdoes not update the LrC catalog requiring you to relink the folders at their new location manually. On the other hand, moving with LrC updates the LrC catalog as the filders are moved but is doing a move, not a copy.

If you can't find missing files or folders there are 3 things to do

1. Look harder. Look at other drives, even ones collectiong dust on the shelf. Fire up old computers and see what they have. If you don't clear memory cards dig those out. Check cloud backup locations (OneDrive, iCloud, BackBlaze, CrashPlan, DropBox, etc.). If you don't empty the recycle bin a lot, check there. See if image sharing sites like SmugMug, Flickr, Apple Photos, Google Photos, Etc. may have copies of the images. If you had Windows File History (MS version of Time Machine) turned on check there.
2. Decide if it's not that big a deal to just remove the images from LrC. Maybe the missing ones are part of a set where you still have other similar ones, your you just don't care that much about those images
3. If LrC shows you previews of the missing images, it is possible to build JPG's from those previews. They will be whatever resolution the largest preview is for any particular image but in almost all cases will be much fewer pixels than the original.

 

[clee01l]

I'm pretty sure that LrC drag and drop in the Folders panel always does a Move. (you can check this with one photo or folder)

Lightroom Classic uses a subset the same file system commands as Explorer/Finder. COPY is not one of the subset but MOVE is. So LrC only Moves files in the folder panel.

 

[T-time]

Lightroom Classic uses a subset the same file system commands as Explorer/Finder. COPY is not one of the subset but MOVE is. So LrC only Moves files in the folder panel.

Received. Thank you. So, am I correct in my understanding and plan to (A) find all missing photos (B) backup my LR catalog (C) run "find all missing photos" (D) drag the folders (one by one?) onto the new drive? And regarding step D it sounds like you recommend that I plug in the new destination drive so LR recognizes it then make the move entirely from within the LR folders panel?

 

[T-time]

Received. Thank you. So, am I correct in my understanding and plan to (A) find all missing photos (B) backup my LR catalog (C) run "find all missing photos" (D) drag the folders (one by one?) onto the new drive? And regarding step D it sounds like you recommend that I plug in the new destination drive so LR recognizes it then make the move entirely from within the LR folders panel?

And, pardon my ignorance, but do I need to move my entire photo collection over to the new drive too so I can continue to build on that folder structure and so LR can communicate with it?

 

[clee01l]

And, pardon my ignorance, but do I need to move my entire photo collection over to the new drive too so I can continue to build on that folder structure and so LR can communicate with it?

Lightroom Classic stores the path to each image in absolutePath field of the catalog file. Moving the image folder using drag and drop in the Lightroom Classic app lets the catalog keep up with the new location. The path can be any where, on any drive. You can have several volumes showing in the Folder panel. LrC does not care as long as the absolutePath field of the catalog file resolve to an actual location for that image. Otherwise it will report the images as missing because it is not at the end of the path stored in the Catalog file.

 

[Califdan]

(A) find all missing photos
(B) backup my LR catalog
(C) run "find all missing photos"
(D) drag the folders (one by one?) onto the new drive? And regarding step D it sounds like you recommend that I plug in the new destination drive so LR recognizes it then make the move entirely from within the LR folders panel?

That is probably the easiest (although perhaps not the safest or speediest) method.

An alternative is....

A) Find all missing photos
B) Synchronize each folder to see if there are photos on disk that LrC doesn't know about
C) If the image folders do not already descend from a common parent on each drive, create such a parent and using the Folders Panel in LrC move all the image folders to be sub-folders under this parent. (this is a bit tricky to get a NEW parent folder to show in the folders panel, so if you have trouble, put some random image in it using windows and import that one image. Then it will show up)
D) Close LrC
E) Using Windows, copy that parent folder to the new drive
F) Unplug the old drive
G) Open LrC and in the Folder panel the old parent folder will still show as being on the old drive and will be marked as "missing"
H) Right click on that parent folder in LrC Folders panel and select "Find Missing Folder". In the pop up window navigate to the same folder on the new drive and select it.

Repeat for other drives but use a different name for the parent folder.

Now you have all the images on one drive, but the ones from each drive will be under a different parent folder. Use drag and drop in LrC folders panel to merge them all under one parent folder if desired. Once the disk specific parent folders are empty you can remove/delete them

 

[T-time]

That is probably the easiest (although perhaps not the safest or speediest) method.

An alternative is....

A) Find all missing photos
B) Synchronize each folder to see if there are photos on disk that LrC doesn't know about
C) If the image folders do not already descend from a common parent on each drive, create such a parent and using the Folders Panel in LrC move all the image folders to be sub-folders under this parent. (this is a bit tricky to get a NEW parent folder to show in the folders panel, so if you have trouble, put some random image in it using windows and import that one image. Then it will show up)
D) Close LrC
E) Using Windows, copy that parent folder to the new drive
F) Unplug the old drive
G) Open LrC and in the Folder panel the old parent folder will still show as being on the old drive and will be marked as "missing"
H) Right click on that parent folder in LrC Folders panel and select "Find Missing Folder". In the pop up window navigate to the same folder on the new drive and select it.

Repeat for other drives but use a different name for the parent folder.

Now you have all the images on one drive, but the ones from each drive will be under a different parent folder. Use drag and drop in LrC folders panel to merge them all under one parent folder if desired. Once the disk specific parent folders are empty you can remove/delete them

Ok. Thank you VERY much. I may ask my IT brother-in-law for his input and supervision on this. I just cannot go another round with all that I went through in the last few days. I long for the day that this is all in order, accessible and so forth. Thank you again Califdan and everyone.

 

[Califdan]

If your IT brothe-in-lawr does no come through and you'd like more hands on help than the free posts and replies in LRQ, I provide live 1 on 1 LrC help through Zoom (for a fee).

https://www.danhartfordphoto.com/lightroom-help

 

[T-time]

If your IT brothe-in-lawr does no come through and you'd like more hands on help than the free posts and replies in LRQ, I provide live 1 on 1 LrC help through Zoom (for a fee).

https://www.danhartfordphoto.com/lightroom-help

Good to know. Thank you again.

 

[T-time]

So, I was able to find all of my missing folders and ran through the synchronization process. Everything was fine until a couple of days ago when LR could not recognize ANY of the folders? Depending which external drive was connected it was recognizing the folders in that drive but now it doesn't recognize any, not matter which drive is connected?? The only thing different that I did was backup my catalog from the previous location to a new(er) external drive. I don't think that should matter as it was just a backup. Any insight on this perpetual nightmare I'm living?

 

[Califdan]

Windows may be assigning different drive letters when you plug one in than lightrroom thinks it has. In Lrc, see what drive letter is listed for a drive with all its folders marked as missing. Then, in Windows File Explorer see what drive letter Windows assigned to that drive. If they are not the same, that is your problem.

 

[T-time]

Windows may be assigning different drive letters when you plug one in than lightrroom thinks it has. In Lrc, see what drive letter is listed for a drive with all its folders marked as missing. Then, in Windows File Explorer see what drive letter Windows assigned to that drive. If they are not the same, that is your problem.

The drives seem to be in alignment. However, I'm now noticing that folders which had been created on one drive (say drive A) are now being recognized on drive "B". I had begun the migration process you recommended. I wonder if something somehow got screwed up then. Do I need to find all of the missing folders again before completing the move or can I just continue to copy the folders onto the new drive then direct LR to point to them there? Uggh

 

[Califdan]

However, I'm now noticing that folders which had been created on one drive (say drive A) are now being recognized on drive "B".

If the LrC folders panel shows a bunch of folders under drive A, but Windows File Explorer shows those same folders as being on drive B, that is the exact problem I was referring to. The LrC folders panel is showing the path it has stored in the catalog (drive A). But that path is no longer valid as Windows now considers that drive to have different drive letter (Drive B) than LrC does.

Change the drive letter in Windows to match what is in LrC or re-link the folders in LrC to the new drive letter. Also in Winddows you can tell it to lock in a drive letter for any particular drive such that Windows will always use the same drive letter for that drive if that letter is not already in use. To prevent it from already being in use, choose drive letters higher in the alphabet.

 

[T-time]

I appreciate the additional assistance. This is like herding cats.

 

[T-time]

Lightroom Classic uses a subset the same file system commands as Explorer/Finder. COPY is not one of the subset but MOVE is. So LrC only Moves files in the folder panel.

Thank you for clarifying that.

 

[T-time]

If the LrC folders panel shows a bunch of folders under drive A, but Windows File Explorer shows those same folders as being on drive B, that is the exact problem I was referring to. The LrC folders panel is showing the path it has stored in the catalog (drive A). But that path is no longer valid as Windows now considers that drive to have different drive letter (Drive B) than LrC does.

Change the drive letter in Windows to match what is in LrC or re-link the folders in LrC to the new drive letter. Also in Winddows you can tell it to lock in a drive letter for any particular drive such that Windows will always use the same drive letter for that drive if that letter is not already in use. To prevent it from already being in use, choose drive letters higher in the alphabet.

I've restored most of the missing files on two drives (Toshiba and Samsung). It seems the only files (or folders) still missing are pointing to a the third drive (SanDisk). The location is in a backup folder that I moved from the Samsung to the SanDisk. I cannot bear to go through this "find missing folder" procedure again. So, let's say there are four missing folders, two showing missing on the Toshiba and two showing missing on the Samsung. Am I safe to assume that if I have the Toshiba and the SanDisk plugged in I should be able to locate those two missing folders? And the same for the Samsung? If that is safe and correct, is it also safe to assume that everything should then be found and restored and each time I plug in any one of those drives LR should be able to read the folders available (or found) on each of those drives? After this, I think I would be interested in taking advantage of your paid service.