• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.

Gathering across multiple hard drives

Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Lightroom Version
Classic 7.4
Operating System
Windows 10
I apologize if this has been answered elsewhere. I looked for it but couldn't find anything similar although I'm sure others have struggled with it.. I have photos scattered over 3 internal hard drives, 1 external and now a laptop. I've been trying to find and import all of them. I have lots of duplicates and lots of frustration. In an attempt to see how widespread the files/photos are I ran a "search" through my whole desktop (not the laptop) for nef files. It found 194K + items! I'm not a full-time professional photographer so this is obviously crazy but shows how bad my structure is. Can I do anything with these files from where they were found during the search?

I would like to import all of my photos into the 4TB external hard drive. The goal being to be able to work on the laptop and the desktop. Any suggestions, plans, outlines would be so appreciated. Or if someone can point me to a previous answer that would help that would be great.

I did try to search in the ebook for some terms but I am not finding a solution. Anyone help? TIA.
 
Joined
Jul 3, 2018
Messages
27
Lightroom Experience
Intermediate
Lightroom Version
Ewokchef, congrats, there are probably lots of people in this situation!

You might make a list of all nef's (including path) on your 5 disks.
Append the lists, sort them in excel on the file name (not loosing the path), in this way you'll find the duplicates.
Seperate the duplicates and bring the unique files together on your 4TB disks, then import in Lightroom.

Would that be an outline of a plan?

Cheers Jetze
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
There are certainly a couple of ways to proceed. You have already made a great first step by choosing Lightroom. Using the Library module, you can easily organize all of the photos.
Let's start with a couple of questions.
First, a nef is a raw file - so you found 194k of those? If you take a wild guess, how many of those are duplicates?
Does the external HD have any unique nefs on it?
I assume you did some viewing and photo processing before this. What software did you use, and did you produce jepgs of some or all of the raws?
Did you rename any or all of the nefs as they came out of the camera?
Do you have any organizational structure on your internal drives, such as one upper level folder like "Pictures" that has all the photos in it?

You said you have started to find and import them. How far along are you?

Basically, you can start importing all of them where they are. Tell LR not to import duplicates. Since the raws don't change, you will only get one copy of each nef - unless there are multiple names associated with a duplicated photo.
Then you can use the Lightroom library to drag the photo folders onto the external drive. This will be a good start at the organization, and LR will remember where they all are.

Once we fully understand the current situation, we can help you along a few steps at a time.
 
Last edited:
Joined
Jul 3, 2018
Messages
27
Lightroom Experience
Intermediate
Lightroom Version
I would follow Jim's advice,
Appears to be far superior than mine.:)
Only make sure not to overlook any folder with pictures stored in it.

Kind regard, Jetze
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Only make sure not to overlook any folder with pictures stored in it.
That's why I asked about the current organization.
We could ask LR to look at the whole hard drive to find all the photos, but that probably isn't necessary.
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
First, a nef is a raw file - so you found 194k of those? If you take a wild guess, how many of those are duplicates?
I don't see how I could have more than 30-40K or so of original raw files. I do have some jpg originals, too, from before I started shooting exclusively in raw but I didn't hunt for those.

Does the external HD have any unique nefs on it? I'm sure there are some on the external but I don't know. I can look?

I assume you did some viewing and photo processing before this. What software did you use, and did you produce jepgs of some or all of the raws?'
I've been using LR for probably more than a year and I just wish I had started here . I started with Picasa way back when and then used PS Elements for several years. I've done editing on all of those programs and exported from LR into JPGs for sharing them. I was using Laura Shoe's video series most recently to try to make some sense of the mess I had made of LR; it is a really good series but I don't follow her well so I decided to look for help from LRQ.

Did you rename any or all of the nefs as they came out of the camera? I'm not sure if I renamed them but probably not. Not consciously for sure.

Do you have any organizational structure on your internal drives, such as one upper level folder like "Pictures" that has all the photos in it? I have tried to import the photos into an upper level file called "Photos Go Here". That's a Laura Shoe suggestion and I find that very workable.

You said you have started to find and import them. How far along are you?
I've been trying to do all the 2017 and 2018 since I got the new laptop in May of 2018.

I don't know if I gave complete enough answers for you to make sense out of my chaos and help me to dig my way out but I very much appreciate your willingness to listen and try.
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Ok.
I assume that you only have on LR Catalog. That will make life much easier.
Now let's think about where you want to end up.
You said you want to end up with all the photos on the external HD (EHD). That will work - but you need to consider a back up plan from the start. That will need to be an external HD of the same size. And if all the photos are in one folder, than you only need to back up one top level folder.

So, to start - make a upper level Folder on the EHD called "All Photos". You can make this with Lightroom, so that LR knows about it. You will eventually have LR put all you photo folders into the All Photos folder.

Inside All Photos, You should have upper level folders for all years : 2018, 2017, 2016 , etc.
Some people like to have all subfolders of the year folders as just dates; eg 2017-03-25
They then use collections as a way to subdivide things by key words, such as location
But I don't remember where I was was on Mar 25 2017.
I tend to like subfolders with locations if I am there a lot on different dates.
So my substructure looks like:
All Photos
>2018
>2017
>>Arizona
>>>Desert Museum
>>>>2017-03-20
>>>>2017-04-15
>2016
All photos are keyworded.
Also - when I import my photos, I have LR change their name to a format of Date-text-sequence number.
This is going to be difficult going backwards, as some your processed jpegs will lose their relationship to the raw.
So as you import them, keep the names you have for now.
And just add them in place.

If that makes sense to you, then have LR create the year folders inside All Photos on the EHD.

Now it is time to start having LR find all your photos.
Where are the 2018 photos located that you have already imported?

If I assume they are on an internal drive, you are going to eventually drag them to the 2018 folder on the EXD.

But before you do that, can you show me an expanded screen capture of your folder library structue as it currently exists.

Jim
 
Last edited:

PhilBurton

Lightroom enthusiast (but still learning)
Premium Classic Member
Premium Cloud Member
Joined
Nov 16, 2015
Messages
2,082
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic
Ewokchef, congrats, there are probably lots of people in this situation!

You might make a list of all nef's (including path) on your 5 disks.
Append the lists, sort them in excel on the file name (not loosing the path), in this way you'll find the duplicates.
Seperate the duplicates and bring the unique files together on your 4TB disks, then import in Lightroom.

Would that be an outline of a plan?

Cheers Jetze
You didn't say if your system is Windows or MacOS. if Windows, a great duplicate detection utility is this Duplicate Cleaner - Find Duplicate Files. Just drag all your photo folders to the right side of the Where To Search window and then let it run. The free version works great, but I upgraded to the Pro version so i could see duplicates by folder. Another utility that really helps can compare two folders and show your matches and differences. Scooter Software: Home of Beyond Compare. Highly recommended.

Just be sure to not delete files that have been edited, so you'll need to locate each catalog that contains a file that is a duplicate.

Good luck.

Phil Burton
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Jim,
I'm using Windows 10 and definitely only one catalog. I'm not sure how to create the "all photos" folder through LR. I think that folder would be the same as my "photos go here" folder? That's where I've been importing them into up to now. I'm attaching a snip of my current structure. The folders under the way it looks for 2018 is the way I'd like to end up.
1532123494596.png

Having a date under the year so they'll be chronological and then event or description because I also need a reminder more than the date. . And I want to use keywording.
Re: backup, I can't really afford another EHD right now. I do have a large non solid state on my desktop that possibly could be used for backup for now.
Also I'm using Carbonite as a backup program too but only have it on my laptop right now.

I do want to be able to move the EHD from the desktop to the laptop and be able to use both if I can. I'm not sure how to find all the photo folders.

Thanks so much!
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
This is looking good.
I would suggest that you simply import from a single internal disk into the folder "Photos go here". You could put them into a folder called "To be Sorted". Be sure you check the box to not import duplicates. LR will find all the photos. Then, using the Library module, select All Photos; or select all photos in the folder called "To be sorted". Under Metadata, look at them by date. You can then make a folder in the appropriate year by month, and drag all the photos from that month into that folder. Rename the folder as you have done in 2018.
Repeat this for all your drives.
This will take some time, so have at it, and have fun.

I don't know how often you will change editing locations - from the desk top to the lap top. The brute force way to do that is to also put the Lightroom Catalog and previews onto the EXHD (as you have done). Then the pc will be transparent and you will keep everything in sync.

But I strongly suggest that you find a way to backup the EXHD. Hard drives will fail - and they won't tell you when it will happen.
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
This is looking good.
I would suggest that you simply import from a single internal disk into the folder "Photos go here". You could put them into a folder called "To be Sorted". Be sure you check the box to not import duplicates. LR will find all the photos. Then, using the Library module, select All Photos; or select all photos in the folder called "To be sorted". Under Metadata, look at them by date. You can then make a folder in the appropriate year by month, and drag all the photos from that month into that folder. Rename the folder as you have done in 2018.
Repeat this for all your drives.
This will take some time, so have at it, and have fun.
I don't know how often you will change editing locations - from the desk top to the lap top. The brute force way to do that is to also put the Lightroom Catalog and previews onto the EXHD (as you have done). Then the pc will be transparent and you will keep everything in sync.

But I strongly suggest that you find a way to backup the EXHD. Hard drives will fail - and they won't tell you when it will happen.


I am very excited and beginning to see a light at the end of the vast LR tunnel. Are you saying to import through LR or move them in a drag and drop or copy and paste fashion from the File Explorer area on my computer into "to be sorted?" It's hard to tell how many GB I have of duplicates and I am excited to clean that up. Would you also recommend one of the finding duplicates software as Phil suggested above?

Also another probably dumb question: as I look in each of the drives for that transfer, would a search be the easiest way to find them all in each drive? Many don't seem to be where one would expect them. Like under pictures or something.

As to how often I will change locations, that probably depends on travel schedules and on how long my desktop actually will keep on chugging away. I really needed a new system but decided to go to a laptop.

And I will get a backup for the EHD just as soon as I can! I know it's dangerous. I don't have words for how much I appreciate this. I'm sure I'll have more questions but I do know it is taking time and energy and I don't want to monopolize or for you to know how much it all means to me.
 
Last edited:

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
I am very excited and beginning to see a light at the end of the vast LR tunnel. Are you saying to import through LR or move them in a drag and drop or copy and paste fashion from the File Explorer area on my computer into "to be sorted?" It's hard to tell how many GB I have of duplicates and I am excited to clean that up. Would you also recommend one of the finding duplicates software as Phil suggested above?

Also another probably dumb question: as I look in each of the drives for that transfer, would a search be the easiest way to find them all in each drive? Many don't seem to be where one would expect them. Like under pictures or something.

.....
I am saying to import directly with Lightroom. You don't have to find and move the files by hand. LR will find them. It may take a while, depending on the number of them. Start with one Hard Drive at a time, and start with the obvious upper level folders - like Pictures and Documents. You can select any and all of the folders to look in. When you get to the second and 3rd hard drive, do the same. Again, LR will find them, and ignore duplicates if you tell it to.
When you do the Import, select "Move" files, and move them to the new folder "To be Sorted". And select "Don't import suspected duplicates."
Lightroom will do the work for you.
When you are done with the desktop, you will have to do the laptop the same way.

When you get through with the first drive, feel free to check back in. But I think it will become obvious to you.

We can deal with the organization later.
 
Last edited:

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
I forgot to say above - when selecting a folder, be sure to check the box that says include subfolders.
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Thanks again, Jim.

I have a strange thing going on. I'm moving files into a "to be sorted" folder and it seems to be taking forever. It shows as two tasks in progress: "moving and importing photos" and "building standard previews". It's been engaged for well over an hour. I don't know if my computer or LR have frozen or if it just taking that long to move them. I'm afraid to shut LR down.
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Don't shut it down. Look in the lower left corner of the screen, and see how many it is importing.
It will take quite a while, as the folder you are moving them to is on an external drive. Is it a USB 3 or a USB 2?
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
It's a USB 3. It isn't showing me a breakdown of what's yet to be done. It does show the folders with photos being added. And up on the left top is where it says 2 operations in progress. I may just need to be patient and let it alone? It's moving into "to be sorted."
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Be patient.
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Hi again, Jim. I just thought I'd update you that it has been moving since yesterday! So I"m being patient. The little green box is a little over half done right now. I wish I knew for sure how much needs to transfer yet. My EHD shows 880 GB still free of the 4 TB. I'm praying to the computer gods that my desktop keeps working. I'm itching to get into the photos. :)
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
It looks like you chose a large folder to begin with - must be lots of photos. You are spending a lot of time making the previews and moving them to an external drive. But in the end, it will be worth it.
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Jim (or anyone who has knowledge) Hi again. My transfer continues today. The photo count continues to go up and the available space in my external drive continue to decrease but the % of Lightroom progress has fluctuated. Last night it showed 76% and now it shows 44%. Any further advice? I'm currently just letting it run.
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Is the progress bar just down to one bar? If so, it is probably just making previews.

You could also open up WIndows File Explorer, look at the photo folders on the external HD, and look at properties. That should tell you how many photos are there.
 
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Okay it shows 384 GB with 90K files in the new "to be sorted file". I'm sure I did check no duplicates but that makes me wonder. For a while, I took RAW + JPEG. That could be some of it. I knew it was messed up... :(

The progress bar still shows 2 operations in progress. But there are no new photos shown since 10:27 am.

You could also open up WIndows File Explorer, look at the photo folders on the external HD, and look at properties. That should tell you how many photos are there.[/QUOTE]
 
Last edited:
Joined
Jan 23, 2016
Messages
33
Location
Fishers, suburb of Indy, Indiana, US
Lightroom Experience
Beginner
Lightroom Version
Okay it shows 384 GB with 90K files in the new "to be sorted file". I'm sure I did check no duplicates but that makes me wonder. For a while, I took RAW + JPEG. That could be some of it. I knew it was messed up... :(

The progress bar still shows 2 operations in progress. But there are no new photos shown since 10:27 am.

You could also open up WIndows File Explorer, look at the photo folders on the external HD, and look at properties. That should tell you how many photos are there.
[/QUOTE]
Okay it shows 384 GB with 90K files in the new "to be sorted file". I'm sure I did check no duplicates but that makes me wonder. For a while, I took RAW + JPEG. That could be some of it. I knew it was messed up... :(

Sorry but one change has happened: It has progressed from 44% to 47% on the lower tool bar but no other changes.
 

Jimmsp

Senior Member
Joined
Jan 9, 2011
Messages
1,019
Location
Green Valley, Arizona, USA
Lightroom Experience
Advanced
Lightroom Version
Classic
OK. Sounds like you have 90k of photos to work with, and LR is now working away on previews. LR treats jpegs and raws separately.
Stay the course.
 
Top