Photo Mess and Lightroom

[Alebi]

Hello

Greetings Fellow Lightroomers.
I have read here before, but never had an account.
This place is a great resource for valuable information

Long explanation and a few questions coming up. I am in a bit tricky situation here and need some advice
Will do my best to explain.
Please let me know if anything is unclear


Background:
I have used Lightroom for a few years and do okay there. But always with smaller batches of photos or catalogs using the excuse that I just back it up to my Nas “So I have everything there”
The last 5 days I have had a few attempts trying to sort my Lightroom situation. -Gathering folders ,Backing up , importing folders , sorting folders and sorting duplicates.
I have read articles, seen youtube videos, and tried to clean things up. So far so good.


My NAS and Lightroom Situation :
Now I want to clean things up on my Nas. I already have a 100% external backup, so I am free to go ahead.
The Structure today consists of several folder and backups.Some duplicates, other failed Backups.
The Main folder I have on my Nas ,is called “Private Backup Folder” The size is 1,8TB (Goodbye social life )
I think the realistic size after duplicates and sorting is done should be about half the size.
My main goal is to make an organized version of my library so I can safely remove and delete my duplicates.
That way I can create The new “main backup” and again transfer this to an External Backup drive ,as well as a copy in the clouds.


After making sure that all the folders were on the Nas ,I Started optimistic sorting the folders in finder ,just to get a little better control before importing them to Lightroom.
I have thought about breaking the total amounts of photos into smaller groups ,but then I lose control if they are represented in another folder later in the process.
I imported the folder overnight. It counted 144K photos. (how optimistic)
For some time I actually managed to keep track.”Everything seemed to be there.”
When I did merging of identical folders I noticed some of the folders had different amounts of photos in them ,and after some time I also noticed that Lightroom didn’t import all of the photos.
It would say 165 files in a folder in Lightroom ,and 270 files if I opened it in Finder.

Now it gets a bit advanced:
in my new Catalog I have both deleted or removed some files I was certain at the time was duplicated.
All this in short I lost control so I need to do it again.
I want to delete my newly imported 1,8TB Catalog and the folders they came from . Then Copy my external backup of the 1,8Tb and use that instead.
That way I know that everything I have “sorted” in Lightroom the last days is discarded and I still will have all my photos.

Solution?:
What is the best procedure to do this?
Should I Import in smaller groups?
Can I create a catalog in Lightroom and import to Lightroom with the option to “copy to a new Location and add to catalog” ?
Or do I Import all the folders from my external hard drives and check Dont import suspected duplicates?
When All this is done. How can I compare the new backup with the backup on my external hard drives?

As mentioned before I have seen a lot of videos about merging catalogs ,creating master catalog , finding duplicates etc. But I haven’t found a video where they use an example of how to import a huge mess of a library, and how to sort it all out into a structure.
In the videos I have seen , their duplicate folders are called copy and a decent amount that is possible to sort. Mine have identical names ,and sometimes identical folders have different size.
Is there anywhere I can see a video or read an article that I can relate to that explains how you can sort out when you have messed up and have several backups?

I spent all day trying to formulate this into something nice and short but it ended up totally opposite.
I guess with more details present I`m more likely to get a response and help to sort this totally self inflicted challenge of mine.
If I have forgot anything, let me know and I will follow up

Thanks

 

[rob211]

Sounds horrid.

I might suggest making a pass at some of that stuff in chunks outside of Lr to eliminate some very obvious duplicate files (which BTW aren't the same as duplicate images). Just maybe all the ones with the same filename. Something to pare it down a bit.

Then start importing in stages. Unfortunately if you've let things go probably the only way to truly get rid of dupes is to just cull mercilessly. Is there some reason why you've got that many photos? most individuals just don't need as many photos as there are grains of sand. So just use the opportunity to get rid of half of them.

 

[Jimmsp]

I support the idea of importing in stages, even though I don't have a specific detailed answer to your questions.
All I know is that when I asked Lightroom to do some fairly straightforward, but massive moves, it failed.
I'll go look up some info, but I think massive meant 100 GB or more.
I had to delete everything and start again with smaller chunks. Small chunks worked.
BTW, I didn't want to use the OS for the moves, as it involved a lot of separate folders and I didn't want to spend the extra time finding multiple "lost" folders.
Bad choice.

 

[Alebi]

Hello,
Thanks for input guys!
I have thought about doing it in smaller chunks. Just thought I could sort them all in Lightroom ,eventho it seems a bit hopeless. But I guess It is a bit to much @rob211 : Why I have so many photos- What do you mean by that? I shoot a lot If you meant Why I have so many duplicates I dont have an answer for that. It just happens. Maybe its a cross of procrastination and bad habits. (Which I am trying to change now)
@Jimmsp I can hear the computer work when it moves outside of Lightroom. If I do it inside Lightroom I am not hearing anything.
When photos show up in empty folders etc I get worried.Usually and so far I have done a lot of moving inside Lightroom.Never had any problems with it and actually how I prefer.
Maybe Lightroom works better for moving Photos when its not such a huge amount

 

[rob211]

By a lot of photos I meant that you obviously have more than you can manage. Lr is only a tool for that; it's not a panacea. You said 144k images, which isn't bad, but 1.8TB worth is more like 1.5M 50MB images. That's way more than duplicates. Even quadruplicates. Shooting a lot is fine, but so is having some editing discipline. It makes organization MUCH easier.

 

[Mickey]

It seems that the first step is to figure out if you have duplicates of imported images, but only one copy on your hard drive, or if you truly have duplicates of the images themselves. You mentioned you have multiple catalogs which is why I suspect you have duplicates within Lightroom. Unless you have a really, really good reason to have multiple catalogs, I suggest moving towards only one.

I highly recommend the book by Peter Krogh on Organizing your Photos.

I would also suggest the following high level steps.
1. determine if you have multiple copies of the same image on your harddrives. If you do then it changes things.
2. Start a new fresh catalog to become your master.
3. One by one import your other catalogs into your new master and review them as you go, eliminating the duplicate as you go.

I know its must more complicated than this, but it should help you build the framework you need. Good Luck!

 

[Alebi]

Hey
Thanks for the replies.
I did a sorting into Years ,and Folder wise ,as there are 3-4 backups with same folders in some cases.
Hopefully a step further on the way to sort things up
Will do an import of catalog one by one as recomennded and see if this helps things out.@

Thanks again