Sync PC to Mac, new storage, tidying up, sync issues, want to start fresh

[MikeWhitten]

Lightroom Version

LR: 12.1, Camera raw: 15.1

Operating System

1. macOS 13 Ventura

Y'all,

I'm a long-time Lightroom user migrating from PC to Mac, upgrading my storage, and wanting to do some long-overdue house cleaning.

For years I only used Lightroom on the desktop; I've got about 98,000 photos in that part of my catalog and I'm happy with both the folder structure and the collections. It's looking good. When the iPad Pro first came out I experimented with ingesting photos to Lightroom Mobile while on trips and got hooked! I love the immediacy. So, I switched to that being my only way to take in photos, syncing up to the cloud and back down to the desktop for backups and printing. I've got about 34,000 photos in the cloud. I'm happy with the organization of those photos in the cloud and on my iPad & iPhone.

As part of moving to the Mac and better storage I took a look at the organization (way, way overdue I know) of the cloud files synced down to disk. Ugh. What a mess I've made. For a while I had them syncing to a big puddle without realizing it. Then I tried switching the organizational preference, thinking it'd clean up the old files but didn't check. It's a mess of my own making. Before finding this wonderful site I tried to fix it on my own and it hasn't gone well. The cloud files still look fine in the cloud. The files I imported directly on the desktop still look good. The cloud files that have synced down not so much. I've got some collections with a *lot* of files that show as missing. I'm sure operator error is heavily involved.

I'm at the point I'd like to just start fresh, with the end goal being having a clean catalog that I can trust with the synced-down stuff in the yyyy/mm structure on disk. I don't mind one bit syncing everything down again; totally worth it to me to be sure I've got a clean structure going forward.

I got the book, did some reading, have perused the forums. And... I'm still not totally clear on the best, safest path forward. If there's a cookbook recipe for this in the book or some post I haven't found then please just point me there. Any advice on the best steps appreciated!

Thanks,
Mike

 

[clee01l]

I've switched file system organization several times over the years from when I began in 2007-08. It was about 2009-10 that I switched from a PC to a Mac. For most of the time I have used one of the date named folder schemes. Ultimately I determined that the folder scheme was not really important at all. REAL organization occurs in the effective use of Lightroom Collections and Smart Collections. I would suggest finding a suitable organization scheme that is compatible with the way Lightroom Mobile images are sync'ed into Lightroom Classic. Use this of all new imports. For the moment, ignore the "mess" of the current folder structure and concentrate on Collections and (Lightroom Mobile) Albums.
Later you can work older images into the current folder scheme as time permits. Remember Lightroom Classic does not even care where the master images are stored in the filesystem. In Lightroom (cloudy) the file system is not even used and the file storage is hidden from the user. With that in mind. you can apply the same logic to Lightroom Classic.

 

[MikeWhitten]

Hi Cletus,
Thanks for the reply.

I'm good on collections and organizing my collections into a logical structure. All my photos are in collections and the collections are in logical structures that work for me; this is true on both the Classic and Cloud side. For new shoots I always set up my logical structure and collection ahead of time and import directly to where I want them.

My main issue is after trying to move my data from PC to Mac I've got a broken sync; collections synced down to my Mac are missing images. Checkmark is green, everything looks complete, but Lightroom Classic is showing me that files are missing.

Since I've got to fix that anyway I'd also like to get my on-disk folder structure tidy while I'm doing this move. That's why I'd like to get best steps to get a clean start-from-scratch sync done.

Thanks,
Mike

 

[clee01l]

Checkmark is green, everything looks complete, but Lightroom Classic is showing me that files are missing.

When you had your master catalog file on Windows the absolutePath in the catalog file was listed and included a drive letter. MacOS (nor any other Operating system) does not use drive letters. In your folder panel you have a folder (let's call it "MyFolder") When on Windows it might have been located as "C:\Pictures\MyFolder". When you moved to MacOS, that folder was located on the Primary Disk ("MacIntosh HD") as "/Users/{userID}/Pictures/MyFolder". In Lightroom Classic there is an option to find missing folder. Right click on the top level folder in the Folder panel that is shown as missing. Choose "Find Missing Folder". in the popup context menu and in the system dialog that opens, navigate to the location of that folder in the MacOS Filesystem. Repeat for all top level missing folders. until no more show as missing. A found top level folder will also locate the missing sub folders.

 

[MikeWhitten]

Hi,

Yea, I did that as part of the move process. All the Cloud collections and folder logical structure has synced down. Some collections are totally fine. Some are missing 20% (give or take) of the individual images.

Mike

 

[clee01l]

I seem to be missing something here. Are you saying that some ~20% of the cloud images are not in the computer? Are you sure that all have synced? Did you copy all of the LrC master images from the Windows computer to the Mac or did you let the Adobe Cloud sync these down to LrC on the Mac?

Maybe some LrC Screen shots could better illustrate the issue.


Sent from my iPad using Tapatalk

 

[MikeWhitten]

Hi Cletus,
Hey, again, thanks for your time and engagement.
Good idea about the screenshots. I attached one at random that doesn't show the issue; one that does.

I'm saying that 20% (ish, not exact) of the images from *some* cloud collections are not on the computer. I'm not sure of the overall numbers; 20% is a swag. Lightroom believes all syncing is complete, I think. I attempted to copy over all the images from PC to Mac. Somewhere in there something may have gone wrong; I don't know. Operator error is my first guess; I had a _lot_ of data to migrate, not just Lightroom, and due to me letting my data organization get away from me I can't be sure I didn't miss something.

I did not ask LrC to sync down - yet! It's because of the scattershot nature of the problem that I'm here, hoping to learn how to move the Classic files to a new catalog and then sync it to Lightroom Cloud to have a fresh, clean starting point.

Mike

 

[MikeWhitten]

Ooof, what a mess. Wish I'd found this site and books first.
I found the missing photos; they're in Mobile Downloads.lrdata which I have on disk. Approx 250 photos out of 1500 from one collection are in there.
Putting it where the ebook says and restarting Lightroom didn't help; still missing. I can see the files in the package and I can see then in the cloud.
What's the best way to patch them back in?

 

[GregJ]

I vehemently disagree with Clete about this and have for many years. I wish he would stop teaching that to computer novices and long-term computer users, or anyone struggling with their messy folder and file name organization schemes. The folder/file structure and naming conventions that you should use is absolutely critical in so many areas of your computing life.

Don't rely on LR collections to sort everything out for you and to organize your lifetime of shooting. That is a huge and fundamental mistake. You can do both. Have a sound folder structure that makes sense to you and use collections by building good key words in all of your files in post.

Get a good folder structure that makes sense to you and stop all of that cloud sync mess (at least for now).

Furthermore, unless you are a pro outfit that shoots daily and has huge professional non-personal storage requirements covering hundreds of clients, I would not use a date name folder or file naming scheme. Big mistake if you ask me. But that is another subject.

You have to decide. There are many people out there that teach it like Clete does, and there are thousands of old computer warhorses like me who passionately disagree with that approach.

 

[Johan Elzenga]

For anyone reading this: this is one man’s opinion, even though he writes it as if it were a fact.

 

[GregJ]

Not a fact. Just a very strong, well-founded and educated opinion. In my opinion it is a bad idea to abandon or sacrifice a sensible, logical and clean folder and file naming system just because LR has a powerful collections capability. You can do both.
There is a school of thought out there that I do not agree with but is often pushed on the camera and LR forums and blogs that all you need is LR Collections to do whatever you want to do organizationally, as if to say that you could have one gigantic folder called xyz containing tens of thousands of random file names and just stir it all into one big pot randomly because LR can sort it all out however you want with Collections based on metadata and added key words.
The Collections feature of the relational database (LR) is a powerful tool, but not a reason to not have a sound folder and file structure and system.
Now I'm not saying Clete said not to have a sensible folder structure and file naming system. I bet he has a sensible system that is not random.
And I agree with him and with you (Johan) that LR's Collections capability is a great tool that we could all learn to utilize better as we could all do better with Key Words and our workflow in that regard.
I will say this to the OP. In cases where you want to do extensive surgery (a massive reorg) on your folders and file names and combine them from several sources, and you are dealing with tens of thousands of raw files and hundreds or thousands of folders, I would not do it in LR. In Windows, I would do it in File Explorer and make sure all my raw files were written to with sidecars in LR before I did it. When I finished my extensive reorg effort, I would abandon my old Cat and create a new one and simply import the entire new structure into a new catalog. What would I lose?
I would lose edit history, virtual copies and the Collections. If that is too big a price to pay, then don't do it that way.
But trying to do it in LR Library and keep the cat synced is difficult when you are doing a major reorg.
I use the LR Library module all the time (I stay in LR) to clean up my folder and file organization and it is incremental with mainly just name changes and some cleaning up of the folder stacks.
I'm talking about these recent OPs here who are claiming they are going to do a total reorganization where they are combining the files of several disks and making a new structure. Sometimes it is better to just start over and import into a new cat as long as you have sidecar files with all of your edits.
But yes, otherwise the golden rule is do it in LR.
The other golden rule is don't tell people that they don't have to care about their file organization and folder schemes just because LR has a powerful Collections capability because that impacts their life way outside of LR and forms bad habits.
But that is just my opinion of course. But it is safe to say that this opinion is shared by almost every photographer I know. And I know a lot of photographers! LOL.

 

[MikeWhitten]

For anyone following the original issue:
I did some reading in the MissingFAQ book & learned a lot more about how Lightroom Classic organizes data when synced down from the cloud. A big part of my problem was a lack of understanding and also a lack of keeping that data organized or even knowing for sure where it was all stored. I'm pretty sure I also changed storage schemes a time or two in the last couple years. This led to me making an absolute mess of porting the photo files and catalogs over.

Instead of continuing to burn hours hacking away at trying to put all the pieces back together I decided to do the following:

I made a directory on my NAS called "Lightroom photos". It bifurcates immediately into "From Classic" and "From Mobile". Under each respective directory will live the photo files from each respective domain. Populating the From Classic folder was easy; I just moved my existing data in there and relinked the folder in Lightroom. For those that care it's mostly organized by date with some outliers for a few reasons that suit me. I knew that data was in good order & could be confident of getting it all.

I exported a new catalog based on that folder of images. This gave me a clean catalog of images imported via Lightroom Classic. It did bring along any collections that referenced those photos. I do also make heavy use of collections for doing logical data organization, both in Classic and Mobile apps.

Then I set up my Mobile sync preference to point the download to the new From Mobile folder. Remember - I also knew my photos and collections in the cloud were in good shape. I elected the ByYearAndMonth folder scheme. I turned on sync, agreed to the start-syncing-here prompt, and Lightroom got to work.

As I type this Lightroom is syncing down about 34,000 photos in the background. Just checked - there are 26,000 to go; it's making good time. I made that move around 11:30am my time; about and hour and a half ago.

If all goes according to plan in a few hours, maybe tomorrow morning, I'll have a clean catalog with all my photo files back in good order and all my collections synced.

Thanks to @Paul McFarlane for his extremely rapid review of my proposed solution! It was comforting to have him take a look. Thanks also for the replies here.

I'll report back on the final results.

 

[GregJ]

You guys are brave. Not me....

I'm a prolific photographer who shoots a lot (I didn't say I was good - I said I shoot a lot). LOL.

Anyway, I have 6 TB of hundreds of thousands of raw and TIF images in thousands of folders all on one 8TB internal SSD, backed up to another 8 TB internal SSD, and synced to 4 external 10 TB spinning rust drives (all synced one at a time using GoodSync). I'm also a computer hobbyist and I build a new rig every 3 years for fun and because I can. Let me share with you 5 things I do not do:

1. Use RAID. No way Jose...
2. Use a NAS
3. Use spinning rust drives for my work (except for backup)
4. I do not back up to the cloud or sync with the cloud with anything to do with my photography files. No way. Not with 6 TB and my organization scheme. Not there yet. I do save all my other files of all types to One Drive, but not my image files.
5. I never ever use LR for cloud anything. No way. Not yet. I want my big raw files on my one fast M.2 SSD and LR Classic. I don't want any of the syncing horror I read about here every day. And I don't want LR messing with, changing or adding to my folder structure or file naming system.

Now that is just me. I'm probably doing it wrong. But I don't think so. LR works at warp speed on my rig with 200MB GFX raw files and they all live on a big super-fast M.2 PCIE4 SSD backed up many times to a big slow (but 4 times faster than a spinner) SATA SSD and some 10 TB spinners. I don't want to be on the road on my laptop doing anything LR-wise off of the cloud. I don't want some small cloud-version preview or any of that mess.

Maybe someday.

 

[MikeWhitten]

Hi Greg - I don't consider myself brave. I've only been talking my working photo storage.
My NAS has an external 8TB drive on it used for regular backups. Periodically that gets swapped for the one in the fire safe.
Since I do ingest via Lightroom Mobile my Adobe cloud account has raws for all my files from the last five years or so.
I've got an additional cloud backup company with a full copy of everything, synced nightly.
So three separate, redundant on-site storages and two separate off-site stores.
As a bonus, when I'm on a trip to a place with good internet access by doing daily intakes on the iPad, and the fact that I've got my computer on a UPS backup running, before I get home all raws are. on 1)the memory cards; I carry enough to not erase, 2) the iPad; it has enough storage to hold a trip's worth of raws (to date), 3) Adobe's cloud, and 4) has been synced down to my NAS.
If I don't have internet I take a couple extra 2TB SSD's.
Mike

 

[MikeWhitten]

That's just backups, though. I hope I'm paranoid enough in that area to not lose files.
Syncing has, until this unpleasant experience, been a joy. It's worked, but unknown to me I was building up some data maintenance debt, that I am now paying in spades.
I enjoy editing on the iPad. It surprised me, but I really do. That touchscreen experience is sweet. I do edit some on the desktop and still enjoy making prints; totally the desktop for that.
If I'd stayed on top of my data organization, just by understanding what Lightroom was doing none of this would be necessary. But I didn't, so here we are.

 

[MikeWhitten]

Greg, y'all, to the point about sync pains - sure looks like Lightroom has a memory leak when syncing. I got the message "your machine has run out of application memory" and had to force restart.
I'm not running Lightroom and watching memory usage. It slowly but inexorably keeps ticking up.
I'm on the latest MacOS & latest Lightroom. Not sure where it lies but either Apple or Adobe (or both, I suppose) has a memory leak that's showing up as I re-download these thousands of photos.

 

[clee01l]

…
I found the missing photos; they're in Mobile Downloads.lrdata which I have on disk. Approx 250 photos out of 1500 from one collection are in there….
What's the best way to patch them back in?

.
Apologies for goin MIA. Life intervenes unexpectedly sometimes.
Mobile Downloads.lrdata is the default sync folder when you sync FROM the Cloud. For future, you can designate the folder to receive Cloud images in Preferences. I use date named folders and there is an option in the sync preferences to default to the same folder date named folder scheme as other imports.

For now, if Mobile Downloads.lrdata is not showing in the Folder panel, you can click on the “+” in the Folder panel header and add this folder to the Folder panel. This would allow you to find any marked as missing as this is the current location for the original image files.

How does this get you for solving your present situation?

You can do your folder organization best using the “drag and Drop” function to move files and folders around in the folder panel. Remember I said it does not need to be perfect and complete all at once or even ever since Lightroom does not care where the files reside in the file system just that It has a correct path to all of the files imported.

I think it is best that you choose one “advisor” and stick with their recommendation until you are satisfied that you have a handle on the problem. There are many solutions to your issue. Some work better than others. And many people giving advice on this forum are not as experienced as others. There are about a half dozen “advisors” here that I respect and trust. Most are better than yours truly.


Sent from my iPad using Tapatalk

 

[MikeWhitten]

Hi Cetus, no worries. Life happens to us all.

Post #12 above describes in detail what I decided to do. In brief - I made a new catalog with the images I knew I had correct locally and am re-syncing from the cloud for the ones I'd imported via mobile. I've now got a clean data structure on one volume for both sets of images; I'll not be getting caught in this trap again.

Other than the annoying memory use bug I'm seeing re-syncing is working exactly as it should. As of this typing 13,000 of the 34,000 images have re-synced and the rest are coming down at a steady clip.

I certain I could have gone back and pulled all the image files together again from my local backups. I'm less certain I could have recovered all my edits due to the mishmash I'd made of taking care of the Lightroom data. One positive aspect of this episode is that I've learned about these forums, the FAQ book, and some best practices regarding Lightroom data.

 

[MikeWhitten]

It's done. It finished sometime yesterday afternoon; about 34,000 photos and 500'ish GB synced down from the cloud. Thanks for all the advice and discussion.

Now I'll be paying more attention to how I handle data going forward in this mixed world I like.

 

[GregJ]

It's done. It finished sometime yesterday afternoon; about 34,000 photos and 500'ish GB synced down from the cloud. Thanks for all the advice and discussion.

Now I'll be paying more attention to how I handle data going forward in this mixed world I like.

I think it is best that you choose one “advisor” and stick with their recommendation until you are satisfied that you have a handle on the problem. There are many solutions to your issue. Some work better than others. And many people giving advice on this forum are not as experienced as others. There are about a half dozen “advisors” here that I respect and trust. Most are better than yours truly.

Clete, you are right. Your reply makes me think. Take me for example. I know that every Guru on here (especially you and two others I could name) forgot more about LR yesterday than I will ever know. I think you guys do a great job of not stepping on each other with your advice, especially on a LR setting, library or development issue. But sometimes you drop in little comments about how best to do other things in the computing arena, and I might not agree with some of that....
But I never say never, and I might go LR Cloud sometime in the next couple of years.
Maybe. I doubt it but maybe.... I'm keeping my eye on it.
I have used LR for so long on so many bazillions of raw files that I get comfortable with my way of doing it and may lose sight of the fact that there might be a better way. That is why I skim this forum to learn new tips and tricks which has helped me immensely in LR in the past 6 months. I'm doing things better now than even 6 months ago. Anyway, I never think I can't learn and improve.
This past week went back to a folder I shot 8 years ago in Spain. The folder was named very creatively - Spain 2015. In that Folder it had 2,223 files named Spain 2014 1-2,223. I started looking through the images and was astounded how bad I was at LR in 2014. And I thought I was good! I was making lots of mistakes I would not make now, and LR has improved so much since then (plus I have a much better calibrated 4K monitor now). I'm in the process of reediting them all. I do about 100 a day.
Anyway, I agree that on an issue like this thread it is best to pick an advisor, but also listen to everyone who does this (not from me because I don't cloud sync in LR) and learn about the issue from a variety of angles.
Like workflow on a photography forum. I love to hear avid photographers describe their workflow so I can get tips on how to improve mine, even if I don't do it like them.
It's also hard to separate LR advice from general computer advice because they are so closely tied. Photography too. LR, computers and photography are all linked and the last two (computers and photography) are heated subjects that cause argument and disagreement from "experts" in those areas. One trip to the most popular photography and computer boards would convince you of that.
For example, I was on a gamers PC building board yesterday and 10 guys who all know more than me about building PCs at the top-end-level were all internet screaming at each other arguing over the best way to configure and connect the new Nvidia 4090 GPU to the latest ASUS Rog Maximus Z790 Hero motherboard and the best power supply unit to use and what kind of ram to get and whether or not to overclock it and by how much.... I'm about to do that myself within the next couple of months and am more confused now than before I read all of that. LOL.