Moving files?

[frostbytes]

Lightroom Version Number

8.0

Operating System

1. Windows 10

Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.

My current folder structure looks like this:

YYYY-MM-DD Shoot Name
x many shoots

I'll be reorganizing to this format:

[2017]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2018]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2019]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.

What's the best way or reorganizing this so that I minimize any downtime with Lightroom? Do I reorganize the files with Windows Explorer and then click a million times in Lightroom to find missing files or is there a better way?

 

[Philippe Coudé du Foresto]

I woul do it inside Lightroom folders panel
- Create the year folders (2016, 2017, 208, etc.)
- Select all the shoot folders of one year, then, with the mouse move them to the corresponding year folder.

The trick to avoid a lot of manipulation, is to select several folders at a time using the <Shift> key.
It should be very fast, since it won't be a real "move" of the files by the OS, just a change in the disk directory to reflect the new path.

 

[frostbytes]

I woul do it inside Lightroom folders panel
- Create the year folders (2016, 2017, 208, etc.)
- Select all the shoot folders of one year, then, with the mouse move them to the corresponding year folder.

The trick to avoid a lot of manipulation, is to select several folders at a time using the <Shift> key.
It should be very fast, since it won't be a real "move" of the files by the OS, just a change in the disk directory to reflect the new path.

Thanks for the suggestion. That would help avoid the tedious task of clicking "find missing folder" hundreds of times.

Is there any downside to move the folders inside of Lightroom? I have a healthy level of paranoia about the potential for data corruption.

 

[LouieSherwin]

Is there any downside to move the folders inside of Lightroom? I have a healthy level of paranoia about the potential for data corruption.

You do not want to move from one hard drive to another hard drive. This is very slow as it requires a copy and delete if each individual image and sidecar if present. Works ok for limited reorganization but will will potentially cause confusion if it is interrupted by say a power failure.

On the other hand when moving folders and/or files in the Folders panel that stay on the same hard drive it is very fast and efficient. Because only the file system folder structure is updated. The individual files are not touched.

In either case it will cause a new backup of every file that is moved. The backup software does not have the smarts to recognize that only the path has changed.

-louie

 

[frostbytes]

You do not want to move from one hard drive to another hard drive. This is very slow as it requires a copy and delete if each individual image and sidecar if present. Works ok for limited reorganization but will will potentially cause confusion if it is interrupted by say a power failure.

On the other hand when moving folders and/or files in the Folders panel that stay on the same hard drive it is very fast and efficient. Because only the file system folder structure is updated. The individual files are not touched.

In either case it will cause a new backup of every file that is moved. The backup software does not have the smarts to recognize that only the path has changed.

-louie

As part of my reorganization, I do want to switch drives (from D to E). Maybe I should reorganize everything outside of Lightroom first?

 

[Bristolian]

I have moved my whole catalogue from one volume, which consisted of two physical hard drives, to another (larger) hard drive from within LR. My catalogue contains around 120,000 images (plus associated xmp files) and although it was a slow process I don't think any of the files got corrupted - at least none of the ones I have opened so far anyway

 

[LouieSherwin]

First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.

In case there is any confusion I mean a complete backup of your entire filesystem.

I would organize them into the date folders first. That will go pretty quickly. Then once in the new year folders. Use Explorer or another copy tool to move the selected years to the other hard drive. All you will have to reconnect is each year that you move to the other hard drive.

You may want to consider a copy utility that will validate each file as it is copied.

-louie

 

[clee01l]

As part of my reorganization, I do want to switch drives (from D to E). Maybe I should reorganize everything outside of Lightroom first?

DON'T reorganize outside of LR. When moving folders around inside of LR the file path in the catalog is maintained and updated. If you want to (eventually) move to a different drive. Do your reorganization in LR first on the current drive, then move whole folder structures either in LR or in Windows Explorer. If using Windows Explorer ,you of course need to update the file path using the Find missing file/folder.

 

[Zenon]

I like that advice and is what I do. I always move folders around within LR unless it is a big move like the parent folder structure. I think when moving stuff around using the OS people have good intentions to re-establish the link but forget which causes problems down the road.

 

[frostbytes]

I have moved my whole catalogue from one volume, which consisted of two physical hard drives, to another (larger) hard drive from within LR. My catalogue contains around 120,000 images (plus associated xmp files) and although it was a slow process I don't think any of the files got corrupted - at least none of the ones I have opened so far anyway

That's promising. My catalog is substantially larger -- almost 700,000 images. Unless I know for certain that Lightroom's folder moving functionality contains robust error-checking I'm hesitant to move that much data within Lightroom.

Does anyone here know for sure about error checking? I read elsewhere that moving large numbers of files inside of Lightroom has been known to cause corruptions.

 

[frostbytes]

First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.

In case there is any confusion I mean a complete backup of your entire filesystem.

I would organize them into the date folders first. That will go pretty quickly. Then once in the new year folders. Use Explorer or another copy tool to move the selected years to the other hard drive. All you will have to reconnect is each year that you move to the other hard drive.

You may want to consider a copy utility that will validate each file as it is copied.

I have multiple levels of backups (Drobo devices, cloud, etc.). I'm too paranoid not to.

And yes, I do use a third-party utility (Teracopy) to move/copy files, which allows for a verification step.

It sounds like Lightroom would let me using the "find missing folder" on, for instance, the 2017 folder so I wouldn't have to find folders for each folder? If so, that should be a lot less onerous than clicking hundreds of folders.

 

[frostbytes]

DON'T reorganize outside of LR. When moving folders around inside of LR the file path in the catalog is maintained and updated. If you want to (eventually) move to a different drive. Do your reorganization in LR first on the current drive, then move whole folder structures either in LR or in Windows Explorer. If using Windows Explorer ,you of course need to update the file path using the Find missing file/folder.

It sounds like I should break this into two steps: 1) reorganize the folders on drive D, 2) copy the data to drive E and use "find missing folder" on that?

 

[frostbytes]

I like that advice and is what I do. I always move folders around within LR unless it is a big move like the parent folder structure. I think when moving stuff around using the OS people have good intentions to re-establish the link but forget which causes problems down the road.

This is a big move. Almost 700,000 photos.

 

[clee01l]

It sounds like I should break this into two steps: 1) reorganize the folders on drive D, 2) copy the data to drive E and use "find missing folder" on that?

Yes, That is how I would approach it. Re-arranging folders on the same drive simply updates pointers in the Master File Table (MFT). No file blocks get moved. The physical move from one drive to the other is going to take time (over night might be best) because the filesystem uses a Copy function to the target destination , followed by a data integrity check followed by a Delete operation on the source drive. If you do this in the LR Folder panel, LR will automatically update to the new path. If you use Windows Explorer, you will need to use the find missing folder (only on the top level folders). Lightroom will use the same filesystem "Move" function as does Windows Explorer.

 

[PhilBurton]

First backup. Did I mention backup! And don't delete any of your source drives until you have the whole new configuration successfully backed up.

What Louie said.

You may want to consider a copy utility that will validate each file as it is copied.

-louie

Doesn't a copy utility operate OUTSIDE of Lightroom? Isn't it a NO-NO to move files outside of Lightroom?

Phil

 

[PhilBurton]

Because of ongoing problems with Lightroom's folder view (hello Adobe?) I've decided to reorganize the physical folder structure for my hundreds of thousands of photos.

My current folder structure looks like this:

YYYY-MM-DD Shoot Name
x many shoots

I'll be reorganizing to this format:

[2017]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2018]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.
[2019]
- YYYY-MM-DD Shoot Name
- YYYY-MM-DD Shoot Name
- etc.

What's the best way or reorganizing this so that I minimize any downtime with Lightroom? Do I reorganize the files with Windows Explorer and then click a million times in Lightroom to find missing files or is there a better way?

Probably a dumb question.

If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?

My original file structure was: YYYY/YYYY-MM/YYYY-MM-DD. Pretty soon I realized how cumbersome that was, and it added unneeded characters to the file path. So I simplified down to YYYY/MM/DD. Never looked back. Never any urges to go back to the original schema.

Phil

 

[clee01l]

If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?

Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA. The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).

 

[PhilBurton]

Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA. The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).

That would also be me, but there is the countervailing force called "common sense." Which sometimes actually gets me to change my ideas. Sometimes. :rofl:

 

[msmack]

I have to agree that the YYYY is not necessary as the Parent in that group is the year. I feel the less numbers I have to look at, the easier it is to see something.

 

[LouieSherwin]

Doesn't a copy utility operate OUTSIDE of Lightroom? Isn't it a NO-NO to move files outside of Lightroom?

Phil

Sometimes it makes sense to do the copy outside of Lightroom even though you have to go back into Lightroom to tell it what you did. The case in point is that of moving a large collection of images from one hard drive to another. While Lightroom will do this and keep track of where the images are now stored it is faster and I suggest safer to make a large move using an external utility and preferably one that will verify the files as they are copied.

What is being recommended here is to first reorganize the shoot folders in to top level years within Lightroom. Then selective copy some of these top level year folders to the new external hard drive. Once the copy is complete use the "Update Folder Location" to point Lightroom to the folder on the new hard drive.

Where people get into trouble is by going into Finder or Explorer and deciding that they want to move things around. Then when going back to Lightroom it is a big mess and be a real challenge to reconstruct what was done externally.

-louie

 

[frostbytes]

Doesn't a copy utility operate OUTSIDE of Lightroom? Isn't it a NO-NO to move files outside of Lightroom?

It sounds like a split card between people who think you shouldn't and those who think it's okay.

 

[frostbytes]

Probably a dumb question.

If you have top level folders by year, then why do you have second-level folders whose names include YYYY? Why not just MM-DD Shoot Name?

My original file structure was: YYYY/YYYY-MM/YYYY-MM-DD. Pretty soon I realized how cumbersome that was, and it added unneeded characters to the file path. So I simplified down to YYYY/MM/DD. Never looked back. Never any urges to go back to the original schema.

I like having the folders containing photos complete with year. That's how I've been doing it for years and that's worked well for me -- at least until the "bug" with Lightroom. It might be redundant to have YYYY > YYYY-MM-DD but that also makes the folder name independent of high-level folders and more flexible in the event I switch away from Lightroom.

 

[frostbytes]

Going Forward, this is a good suggestion. Having to rename all of the existing folders is a PIA. The 'anal' among us (that would be me) probably could not tolerate the difference between the old (YYYY included) and the new naming schemes (MM-DD not YYYY).

It wouldn't be too onerous to rename all the folders using a file naming utility but I'd rather not drop the YYYY from the folder name. I'm anal like that too.

 

[Hal P Anderson]

It wouldn't be too onerous to rename all the folders using a file naming utility

Renaming in bulk is incredibly hard to recover from unless you do it within Lightroom. Moving a whole directory structure outside of Lightroom can be recovered from with a couple of clicks

 

[clee01l]

It wouldn't be too onerous to rename all the folders using a file naming utility but I'd rather not drop the YYYY from the folder name. I'm anal like that too.

As soon as you rename outside of LR, you then lose the internal reference to the path and have to manually find these folders again in LR. There is no automated renaming utility that works inside Lightroom