Moving files?

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#1
Lightroom Version
8.0
Operating System
Windows 10
Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.

My current folder structure looks like this:

YYYY-MM-DD Shoot Name
x many shoots

I'll be reorganizing to this format:

[2017]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2018]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2019]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.

What's the best way or reorganizing this so that I minimize any downtime with Lightroom? Do I reorganize the files with Windows Explorer and then click a million times in Lightroom to find missing files or is there a better way?
 
Joined
Jan 6, 2018
Messages
130
Lightroom Experience
Power User
Lightroom Version
Classic 7
#2
I woul do it inside Lightroom folders panel
- Create the year folders (2016, 2017, 208, etc.)
- Select all the shoot folders of one year, then, with the mouse move them to the corresponding year folder.

The trick to avoid a lot of manipulation, is to select several folders at a time using the <Shift> key.
It should be very fast, since it won't be a real "move" of the files by the OS, just a change in the disk directory to reflect the new path.
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#3
I woul do it inside Lightroom folders panel
- Create the year folders (2016, 2017, 208, etc.)
- Select all the shoot folders of one year, then, with the mouse move them to the corresponding year folder.

The trick to avoid a lot of manipulation, is to select several folders at a time using the <Shift> key.
It should be very fast, since it won't be a real "move" of the files by the OS, just a change in the disk directory to reflect the new path.
Thanks for the suggestion. That would help avoid the tedious task of clicking "find missing folder" hundreds of times.

Is there any downside to move the folders inside of Lightroom? I have a healthy level of paranoia about the potential for data corruption.
 

LouieSherwin

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 24, 2010
Messages
1,331
Location
Encinitas, CA USA
Lightroom Experience
Advanced
Lightroom Version
Classic 7
#4
Is there any downside to move the folders inside of Lightroom? I have a healthy level of paranoia about the potential for data corruption.
You do not want to move from one hard drive to another hard drive. This is very slow as it requires a copy and delete if each individual image and sidecar if present. Works ok for limited reorganization but will will potentially cause confusion if it is interrupted by say a power failure.

On the other hand when moving folders and/or files in the Folders panel that stay on the same hard drive it is very fast and efficient. Because only the file system folder structure is updated. The individual files are not touched.

In either case it will cause a new backup of every file that is moved. The backup software does not have the smarts to recognize that only the path has changed.

-louie
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#5
You do not want to move from one hard drive to another hard drive. This is very slow as it requires a copy and delete if each individual image and sidecar if present. Works ok for limited reorganization but will will potentially cause confusion if it is interrupted by say a power failure.

On the other hand when moving folders and/or files in the Folders panel that stay on the same hard drive it is very fast and efficient. Because only the file system folder structure is updated. The individual files are not touched.

In either case it will cause a new backup of every file that is moved. The backup software does not have the smarts to recognize that only the path has changed.

-louie
As part of my reorganization, I do want to switch drives (from D to E). Maybe I should reorganize everything outside of Lightroom first?
 

Bristolian

New Member
Joined
Mar 4, 2013
Messages
20
Location
Bristol (UK)
Lightroom Experience
Intermediate
Lightroom Version
#6
I have moved my whole catalogue from one volume, which consisted of two physical hard drives, to another (larger) hard drive from within LR. My catalogue contains around 120,000 images (plus associated xmp files) and although it was a slow process I don't think any of the files got corrupted - at least none of the ones I have opened so far anyway :)
 

LouieSherwin

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 24, 2010
Messages
1,331
Location
Encinitas, CA USA
Lightroom Experience
Advanced
Lightroom Version
Classic 7
#7
First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.

In case there is any confusion I mean a complete backup of your entire filesystem.

I would organize them into the date folders first. That will go pretty quickly. Then once in the new year folders. Use Explorer or another copy tool to move the selected years to the other hard drive. All you will have to reconnect is each year that you move to the other hard drive.

You may want to consider a copy utility that will validate each file as it is copied.

-louie
 

clee01l

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 20, 2009
Messages
13,996
Location
Bellaire, TX USA
Lightroom Experience
Power User
Lightroom Version
Classic 7
#8
As part of my reorganization, I do want to switch drives (from D to E). Maybe I should reorganize everything outside of Lightroom first?
DON'T reorganize outside of LR. When moving folders around inside of LR the file path in the catalog is maintained and updated. If you want to (eventually) move to a different drive. Do your reorganization in LR first on the current drive, then move whole folder structures either in LR or in Windows Explorer. If using Windows Explorer ,you of course need to update the file path using the Find missing file/folder.
 

Zenon

Active Member
Joined
Jan 13, 2017
Messages
981
Lightroom Experience
Advanced
Lightroom Version
Classic 7
#9
I like that advice and is what I do. I always move folders around within LR unless it is a big move like the parent folder structure. I think when moving stuff around using the OS people have good intentions to re-establish the link but forget which causes problems down the road.
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#10
I have moved my whole catalogue from one volume, which consisted of two physical hard drives, to another (larger) hard drive from within LR. My catalogue contains around 120,000 images (plus associated xmp files) and although it was a slow process I don't think any of the files got corrupted - at least none of the ones I have opened so far anyway :)
That's promising. My catalog is substantially larger -- almost 700,000 images. Unless I know for certain that Lightroom's folder moving functionality contains robust error-checking I'm hesitant to move that much data within Lightroom.

Does anyone here know for sure about error checking? I read elsewhere that moving large numbers of files inside of Lightroom has been known to cause corruptions.
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#11
First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.

In case there is any confusion I mean a complete backup of your entire filesystem.

I would organize them into the date folders first. That will go pretty quickly. Then once in the new year folders. Use Explorer or another copy tool to move the selected years to the other hard drive. All you will have to reconnect is each year that you move to the other hard drive.

You may want to consider a copy utility that will validate each file as it is copied.
I have multiple levels of backups (Drobo devices, cloud, etc.). I'm too paranoid not to. :)

And yes, I do use a third-party utility (Teracopy) to move/copy files, which allows for a verification step.

It sounds like Lightroom would let me using the "find missing folder" on, for instance, the 2017 folder so I wouldn't have to find folders for each folder? If so, that should be a lot less onerous than clicking hundreds of folders.
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#12
DON'T reorganize outside of LR. When moving folders around inside of LR the file path in the catalog is maintained and updated. If you want to (eventually) move to a different drive. Do your reorganization in LR first on the current drive, then move whole folder structures either in LR or in Windows Explorer. If using Windows Explorer ,you of course need to update the file path using the Find missing file/folder.
It sounds like I should break this into two steps: 1) reorganize the folders on drive D, 2) copy the data to drive E and use "find missing folder" on that?
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#13
I like that advice and is what I do. I always move folders around within LR unless it is a big move like the parent folder structure. I think when moving stuff around using the OS people have good intentions to re-establish the link but forget which causes problems down the road.
This is a big move. Almost 700,000 photos.
 

clee01l

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 20, 2009
Messages
13,996
Location
Bellaire, TX USA
Lightroom Experience
Power User
Lightroom Version
Classic 7
#14
It sounds like I should break this into two steps: 1) reorganize the folders on drive D, 2) copy the data to drive E and use "find missing folder" on that?
Yes, That is how I would approach it. Re-arranging folders on the same drive simply updates pointers in the Master File Table (MFT). No file blocks get moved. The physical move from one drive to the other is going to take time (over night might be best) because the filesystem uses a Copy function to the target destination , followed by a data integrity check followed by a Delete operation on the source drive. If you do this in the LR Folder panel, LR will automatically update to the new path. If you use Windows Explorer, you will need to use the find missing folder (only on the top level folders). Lightroom will use the same filesystem "Move" function as does Windows Explorer.
 
Last edited:

PhilBurton

Senior Member
Joined
Nov 16, 2015
Messages
1,502
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic 7
#15
First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.
What Louie said.

You may want to consider a copy utility that will validate each file as it is copied.

-louie
Doesn't a copy utility operate OUTSIDE of Lightroom? Isn't it a NO-NO to move files outside of Lightroom?

Phil
 

PhilBurton

Senior Member
Joined
Nov 16, 2015
Messages
1,502
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic 7
#16
Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.

My current folder structure looks like this:

YYYY-MM-DD Shoot Name
x many shoots

I'll be reorganizing to this format:

[2017]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2018]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2019]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.

What's the best way or reorganizing this so that I minimize any downtime with Lightroom? Do I reorganize the files with Windows Explorer and then click a million times in Lightroom to find missing files or is there a better way?
Probably a dumb question.

If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?

My original file structure was: YYYY/YYYY-MM/YYYY-MM-DD. Pretty soon I realized how cumbersome that was, and it added unneeded characters to the file path. So I simplified down to YYYY/MM/DD. Never looked back. Never any urges to go back to the original schema.

Phil
 

clee01l

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 20, 2009
Messages
13,996
Location
Bellaire, TX USA
Lightroom Experience
Power User
Lightroom Version
Classic 7
#17
If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?
Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA. The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).
 

PhilBurton

Senior Member
Joined
Nov 16, 2015
Messages
1,502
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic 7
#18
Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA. The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).
That would also be me, but there is the countervailing force called "common sense." Which sometimes actually gets me to change my ideas. Sometimes. :rofl:
 

msmack

Active Member
Joined
May 19, 2012
Messages
227
Location
Larkspur, CA
Lightroom Experience
Intermediate
Lightroom Version
#19
I have to agree that the YYYY is not necessary as the Parent in that group is the year. I feel the less numbers I have to look at, the easier it is to see something.
 

LouieSherwin

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 24, 2010
Messages
1,331
Location
Encinitas, CA USA
Lightroom Experience
Advanced
Lightroom Version
Classic 7
#20
Doesn't a copy utility operate OUTSIDE of Lightroom? Isn't it a NO-NO to move files outside of Lightroom?

Phil
Sometimes it makes sense to do the copy outside of Lightroom even though you have to go back into Lightroom to tell it what you did. The case in point is that of moving a large collection of images from one hard drive to another. While Lightroom will do this and keep track of where the images are now stored it is faster and I suggest safer to make a large move using an external utility and preferably one that will verify the files as they are copied.

What is being recommended here is to first reorganize the shoot folders in to top level years within Lightroom. Then selective copy some of these top level year folders to the new external hard drive. Once the copy is complete use the "Update Folder Location" to point Lightroom to the folder on the new hard drive.

Where people get into trouble is by going into Finder or Explorer and deciding that they want to move things around. Then when going back to Lightroom it is a big mess and be a real challenge to reconstruct what was done externally.

-louie
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#21
Doesn't a copy utility operate OUTSIDE of Lightroom? Isn't it a NO-NO to move files outside of Lightroom?
It sounds like a split card between people who think you shouldn't and those who think it's okay.
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#22
Probably a dumb question.

If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?

My original file structure was: YYYY/YYYY-MM/YYYY-MM-DD. Pretty soon I realized how cumbersome that was, and it added unneeded characters to the file path. So I simplified down to YYYY/MM/DD. Never looked back. Never any urges to go back to the original schema.
I like having the folders containing photos complete with year. That's how I've been doing it for years and that's worked well for me -- at least until the "bug" with Lightroom. It might be redundant to have YYYY > YYYY-MM-DD but that also makes the folder name independent of high-level folders and more flexible in the event I switch away from Lightroom.
 

frostbytes

Active Member
Joined
Jan 10, 2009
Messages
102
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
#23
Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA. The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).
It wouldn't be too onerous to rename all the folders using a file naming utility but I'd rather not drop the YYYY from the folder name. I'm anal like that too. :)
 

Hal P Anderson

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
May 12, 2011
Messages
2,779
Location
Canada
Lightroom Experience
Advanced
Lightroom Version
Classic 7
#24
It wouldn't be too onerous to rename all the folders using a file naming utility
Renaming in bulk is incredibly hard to recover from unless you do it within Lightroom. Moving a whole directory structure outside of Lightroom can be recovered from with a couple of clicks
 

clee01l

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Jun 20, 2009
Messages
13,996
Location
Bellaire, TX USA
Lightroom Experience
Power User
Lightroom Version
Classic 7
#25
It wouldn't be too onerous to rename all the folders using a file naming utility but I'd rather not drop the YYYY from the folder name. I'm anal like that too. :)
As soon as you rename outside of LR, you then lose the internal reference to the path and have to manually find these folders again in LR. There is no automated renaming utility that works inside Lightroom
 
Top