• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Are you using the cloud-based Lightroom apps for iOS, Android, Windows, Mac, and Web? Then you'll love my book, Adobe Lightroom - Edit on the Go!

    You'll learn how to use the Lightroom cloud ecosystem to organize, edit and share your photos. You'll also come to understand the thought processes used by professional photographers when they're editing their own photos, so you can transform your photos quickly and easily. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Want to sync existing Classic and Cloudy Libraries....best approach?

Status
Not open for further replies.

mikebore

Active Member
Premium Cloud Member
Joined
Jul 11, 2008
Messages
286
Lightroom Version Number
9.2.1 (Classic) 3.2.1 (Cloudy)
Operating System
  1. macOS 10.15 Catalina
  2. iOS
I currently have all my photos in both Lightroom Cloudy and Classic libraries, but not syncing. I would like to get them syncing. I would be very grateful for any comments and advice on what I think are the three possible approaches. I am prepared to research the detail implementation myself but am looking for guidance about the best major option I should use…..if I any! I may yet decide to stay where I am.

The three major options ( I think) are:

1. Dump the current Cloudy Library and create new one by syncing from Classic.

2. Dump the current Classic
Library and create a new one by syncing from Cloudy.

3. Don't dump either: just turn on sync.


STARTING POINT
-1TB Photography Plan.
-Lightroom Cloudy with 79,000pics/600GB working well on two Macs and four iDevices.
-Lightroom Classic on both Macs with the catalog and masters on a portable Samsung T5 SSD which can be connected to either Mac. For many years this was my main personal working photo library. Sync is not turned on in Classic.
-When I started with Cloudy nearly two years ago I created the Cloudy library by manually exporting year based catalogs and importing them to Cloudy in turn to preserve the Classic structure in Cloudy, creating folders in Cloudy as I went.
-Since then Cloudy has been my main working library because it automatically collects all the photos my wife takes as well as mine in one place and syncs everything to all devices.
-I have kept Classic up to date with monthly manual exports of “original+ settings” from Cloudy and manually importing them to Classic. I have not really made any proper use of Classic, treating it as a kind of backup and thinking I would one day do what I am now asking about.
-The Classic Library has 76,000 photos. The discrepancy with Cloudy is due to duplicates in Cloudy caused by I believe by some initial problems with the auto import settings in Cloudy.

DESIRED END POINT
-Cloudy Library with 76,000 pics automatically syncing to all devices. No duplicates.
-Classic Library with same 76,000 syncing with Cloudy.
-No monthly manual export and imports
-Ability to use the printing and editing tools in Classic for all photos.
-Ability to do culling and rating on iPad, syncing through to Classic.

MORE ABOUT THE OPTIONS:

1. Dump the current Cloudy Library. Assume I would have to clear all existing photos from the account for the fresh start. Then I would create year collections in Classic and turn on sync so these appear in Cloudy. This would have Cloudy using Smart Previews, for the existing pics, and originals for on going, since I expect Cloudy to be my main user tool.

2. Dump the current Classic Library (save it in a drawer). I would create a new Classic library by turning on sync to populate it from the Cloudy. This would have the 3000 dupes in but there are tools to remove dupes in Classic but not for Cloudy. Hopefully deleting the dupes in Classic would propagate to Cloudy? Would all the Cloudy photos appear in Classic with any folder, album or collection structure?

3. Don’t dump either Library, just turn on sync in current Classic. Not sure what the result would be. Probably nothing initially as there are no collections in Classic. Assume I would need to put all 76000 into a whole lot of collections in Classic so that sync would occur? Would it create or eliminate dupes?

Thanks very much for any comment.

PS This thread reminds me of the joke about a local being asked for directions by a tourist, replying "If I wanted to go there I wouldn't have started from here". :)
 
Last edited:
One can always add detail, but I'd just polish up a couple of rules:

8. Videos in Cloudy sync across to Classic and while they are not listed in All Synced Photos, they are included in synced collections and metadata syncs
12. ...in Classic and added to synced collections or directly to ASP are...

Thanks very much John.
So does that mean that videos added in Classic would appear in Cloudy? My list doesn't have item for videos added in Classic but I had thought they would not sync since they don't appear in ASP, but if they are included in sync'd collections sounds like they would sync to Cloudy?
Would it be true to say:

8. Videos added in either Classic or Cloudy are sync'd both ways like photos, but do not appear in ASP in Classic.
 
Thanks very much John.
So does that mean that videos added in Classic would appear in Cloudy? My list doesn't have item for videos added in Classic but I had thought they would not sync since they don't appear in ASP, but if they are included in sync'd collections sounds like they would sync to Cloudy?
Would it be true to say:

8. Videos added in either Classic or Cloudy are sync'd both ways like photos, but do not appear in ASP in Classic.

No, I was trying to split your "Videos in Cloudy sync across to Classic" away from "but are then treated as not sync’d." The second part is not really right. They are treated as synced, just not listed in Classic's ASP, and metadata is synced both ways just like any other synced file.
 
No, I was trying to split your "Videos in Cloudy sync across to Classic" away from "but are then treated as not sync’d." The second part is not really right. They are treated as synced, just not listed in Classic's ASP, and metadata is synced both ways just like any other synced file.

OK thanks. I will use your original suggestion: "8. Videos in Cloudy sync across to Classic and while they are not listed in All Synced Photos, they are included in synced collections and metadata syncs"..... and experiment a bit to understand. what actually happens in practice.
 
Sometime later today my big download should complete, and I can start using the new sync'd Classic.

Bit of a self inflicted delay here. My new Classic had nearly finished downloading, and I was browsing the Classic Catalog Prefs settings. I realised I had not turned on "Automatically write changes to XMP" which I have always had checked in Classic in the past.

I didn't realise that the XMPs then have to be uploaded to Cloud from Classic. Now expecting that to complete later today. They also have to be sync'd down in Cloudy but don't seem to be changing anything.

I realise writing to XMP is controversial and I chose a bad moment to do it!

Classic is showing five pics with sync errors. It has downloaded them. Any suggestions for dealing with sync errors?

Apart from that all is as expected, except that Classic has 79251 pics and Cloudy has 79249 on all devices and the web. Sync Activity in Classic prefs has been showing 2 downloading for a long time with no change to the total. Any suggestions about that? Maybe that 2 will resolve and the total drop from 79251 to 79249 to match Cloudy.

Thanks very much.
 
I didn't realise that the XMPs then have to be uploaded to Cloud from Classic. Now expecting that to complete later today. They also have to be sync'd down in Cloudy but don't seem to be changing anything.

The download of XMP files to Cloudy is only happening on devices where I have the ALL PHOTOS album set to store locally. XMPs only apply to RAW and the numbers tie with the number of RAW files in the library
 
One of the quirks of Classic>Cloudy syncing is that when the original is in the cloud (as in your case), Classic is aware of that fact. Thus if a synced (original) file is physically changed in Classic (such as when updating XMP) the changed file has then to be re-synced to the cloud to replace the original (i.e. in this situation, Classic will sync originals, not just smart previews). For proprietary raw files that's not such a big deal as only the small XMP sidecar is changed. But for all other files types, writing to XMP will invariably mean a new upload of the complete file. That can get to be a royal PITA if you have a lot of large Tiffs and Photo Merge DNGs. And even more of an issue is that some of those Tiffs and Pano/HDR DNGs can easily exceed 200mb in size, and Classic cannot upload any file larger than 200mb in size and so they get "stuck" trying to upload (that might actually explain your 2 stuck uploads). Because of that I've actually dedicated a colour label for >200mb files, to easily warn me not to do an XMP write for such files.
Although I've never been an afficionado of the auto-write to XMP option (it's always disabled) I do occasionally update existing XMP where I want to keep Classic and Cloud metadata fully in sync, so I've learnt the hard way to be really careful about doing that with large files.
 
Thanks very much for the detailed explanation.

I was about to reply that I don't have any files over 200MB, but then looked at the five sync error pics, and they are all TIFFs over 200MB. I don't know if I have other TIFFs over 200MB or if these are the only ones, but might this explain why they are showing as sync errors? I can't be sure but fairly certain that these errors only appeared after I turned on write to XMP.
 
I can't be sure but fairly certain that these errors only appeared after I turned on write to XMP.
That's logical....everything was fine until you turned that option back on, as that updated those files which in turn causes a re-sync (from Classic to the cloud), but because the files are >200mb the sync fails. Nothing you can really do now, other than remove them from Classic (but not deleting them from disk!) and re-import them to cloudy (but you'd have to turn off Auto-Write to XMP in order to stop the issue immediately recurring), or just remove them from ASP in Classic then re-sync them from there as smart previews. Personally, I'd rethink having that Auto-write XMP option always enabled.....

Welcome to the frustrating world of trying to do what Adobe suggests we shouldn't!
 
Thanks again Jim.
I will have a think. There is no good reason for these five to be so large. They can be flattened now. Or I may turn off XMP.

Only mystery now is the 2 downloading. If they succeed in downloading the total in Classic will go up from 79251 to 79253, but Cloudy only has 79249. It would be better if the resolution is that Classic has already wrongly counted them, so that the Classic number goes down by 2 to 79249, same as Cloudy.
 
Good morning! All syncing has finished and just one problem to solve.

The 2 downloading are still stuck. Clicking on the blue entries for these in Sync Status in Prefs shows exactly what the problem is:

"Sync has not finished for file "image010.jpg". Open Lightroom on Lesley iPhone 8"

My wife no longer has the iPhone 8 so the suggested action is not possible. The two photos were not taken on the iPhone 8, they were taken years earlier.

The two photos exist full size in Cloudy with no apparent problem (ie fully sync'd and backed up) and also in All Syn'd Photos in Classic . So far I have done the following:

I exported them from Cloudy and renamed them.
Deleted them from Cloudy and Classic.
Reimported the renamed ones to Cloudy and they have sync'd across to Classic.

So all is well except Sync Status still shows the original ones as "downloading".

I am pretty certain these two are the reason why Classic shows 79240 and Cloudy shows 79238 pictures.

The only thing I can think to do is "Rebuild the Sync Data" from the prefs window. Is this another 10 days syncing and downloading? or is it quicker? Would it solve it?

Can you suggest anything else?

Thanks very much.
 
If you've effectively duplicated them, and synced the duplicates, why not just delete the two stuck originals? Have you checked that they're not already in the Deleted items album in Cloudy? It's quite common for a stuck syncing situation to occur after a file has been deleted in the cloud but which hadn't finished downloading. In which case deleting them completely from the Deleted album usually clears up the problem.

Rebuilding the Sync Data file is doable, and no it doesn't mean another 10 days re-syncing....it just resets the file containing data about current syncing, i.e. those 2 images, and restarts the sync process for those images.
 
Thanks again.
I have already deleted the two stuck originals from Classic and Cloudy, but I hadn't deleted them from the recently Deleted album in Cloudy, which I have now done, but the two miscreants are still showing as downloading. I can't see a recently deleted album in Classic.

Any other thoughts before I try Rebuilding Sync data?
 
Reading your comments about Rebuilding I decided to do it, but it has not solved the problem.

I attach the screenshot I took before Rebuilding, and notice that the Assets Path/Link was a web address, not a file location. I have just been (since rebuilding) to the web page and the two problem files are not there (only their renamed replacements).

Screenshot 2020-06-03 at 09.45.54.png


The sync status window after rebuilding is the second screenshot, no location data.

Screenshot 2020-06-03 at 11.16.25.png
 
I have lost control of this project....close to abandoning.

After the failed sync rebuilding I generated a diagnostic report in case I was able to see any clues, which I wasn't.

As something else to try without much hope I deleted the replacement photos for the two non-downloading ones. It didn't solve it so I reimported them.

As something else to check I clicked Synchronise Folder for the top level, not expecting to find anything but as a check. Something I have done from time to time in Classic in past.

To my surprise it found 12 to import so I clicked OK.

When it had finished importing the 12, the total number of photos in Classic had reduced by 1167 from 79,240 to 78,073. There was no indication in the post synchronise window that it was going to delete 1167 photos and they are not in the trash.

Thinking I didn't want this to propagate to Cloudy I went to my name plate in Classic to pause sync, and found that sync was already paused. Does creating a Diagnostic Report turn off syncing?

Cloudy is still showing the full house 79,238 pictures.

Don't know where I am now! While some the things I have done may not have had much relevance to the problem I was trying to solve, they were not 'risky' things that should cause a problem.

Does anything I have said make sense?

I have a Lightroom created catalog backup from 2 days ago. But I am wary of doing something that will mess up my Cloudy which at this point is all in tact.

Grateful for any advice, but also conscious there is only so much you can do in a forum dialogue.
 
I really don't know what to suggest. Obviously, I wish you hadn't done that folder synchronisation, but that can't be undone. I suspect that the original dialog (which told you there were 10 images to import....but where did they come from?) also told you that it would remove 1167 "missing images". But why would they be missing? I wonder if they are images that you originally synced from Classic, so there would only have been smart previews in the cloud when you created the new catalog.....in that situation, the smart preview IS downloaded, but the original will be flagged as missing. You could check that by selected All Photos in cloudy and use the Filter to select "Sync Status"....does it give you a number for "Synced from Lightroom Classic"? In a perfect world it would tell you that there are 1167....

Let's just focus on that for now, can I suggest you stop "fixing" as there's a risk of making matters worse.
 
Indeed the Folder sync was a mistake, but given that everything in Classic had just been downloaded fresh onto a new Catalog nothing should have happened.

I wonder if they are images that you originally synced from Classic, so there would only have been smart previews in the cloud when you created the new catalog.....

The pre history of this exercise is that two years ago I started with an established but unsync'd Classic and tried to create a Cloudy Library from it. This went wrong so I trashed the Classic which was messed up by the failed attempt and used a Classic library from a backup and never turned on sync.
I created my current Cloudy by exporting catalogs from the restored Classic and importing them to Cloudy.
For two years I have been using this Cloudy independently of Classic without noticing anything left over from the original Classic. Cloudy currently says all 79000 are synd and backed up with originals in the Cloud, none are smart previews.
However when I started this project two weeks ago with a new Classic Lib and I first turned on sync I got a message that Adobe Cloud was currently syncing with "two yrs old trashed lib" ...did I want to use this new one instead?. I didnt think anything of it, but could that cause what you are suggesting. Could 1137 smart previews exist in the cloud without my Cloudy lib seeing them during these two years?

To try what you suggest I assume I will have to turn on sync....isnt there a risk that the 1137 will be deleted from Cloudy?

Can I make any use of the two day old backup?

Thanks for all the help, promise not to try anything else even if it seems safe.
 
Snippet of info....I have just realised I don't need to turn on sync in Classic to check what the Sync Status in Cloudy is. The "Sync'd from Lightroom Classic" number two pics. Which are both ones I resized to bring them under the 200MB limit.
 
OK, scratch that theory then. In which case I have absolutely no idea why those images were removed by the Folder synchronisation....and I'm not even sure of how to proceed. The nuclear option is to start over with another fresh Classic catalog (though that's probably the cleanest and least risky thing to do), but before that you could try using the backup catalog I suppose. Something's just not right, I wish I knew why those images were removed. Trouble is, you're in largely unchartered territory here, and there's no Adobe support likely to be had either.

If it was me, I'd go with a fresh catalog again, but before doing that I'd make sure there's nothing in the cloudy deleted items album, and I'd make sure that all the cloudy apps are seeing the same totals....and I wouldn't do anything while waiting for the sync to complete.

But you've really got to want this method of working, it can get out of control real quick and can be a real time sink when things go wrong. I probably know as much as most people about how it all works, but I got badly burned last year and it took a couple of weeks to recover.

Sorry I can't be more helpful.
 
But you've really got to want this method of working, it can get out of control real quick and can be a real time sink when things go wrong. I probably know as much as most people about how it all works, but I got badly burned last year and it took a couple of weeks to recover.
Sorry I can't be more helpful.

I think you have summarised my current feeling perfectly in that paragraph. It has been a very valuable exercise in which I have learned a lot thanks mainly to you. I think it was Johan who said in a spin off thread from this that when Classic and Cloudy are syncing Classic is the boss. This is not really what I need as Cloudy is where I will do most of my day to day photo management, and there are too many pitfalls for the unwary if they are syncing.

(A small positive PS, if I hadn't done that Folder sync, the underlying problem that caused 1137 photos to disappear would been a ticking time bomb likely to go off at some less convenient point in the future, so with hindsight maybe it was as well I did it!)

Thanks again for all the massive help.
 
If it was me, I'd go with a fresh catalog again, but before doing that I'd make sure there's nothing in the cloudy deleted items album, and I'd make sure that all the cloudy apps are seeing the same totals....and I wouldn't do anything while waiting for the sync to complete.

After a few days recovery I felt I wanted to have one more go, so did as you suggest above.

After another week of syncing I have 79,259 pics downloaded to Classic from the Cloud and all totals line up on all devices, Classic and Cloudy. Hurrah! I have not done a folder synchronisation or turned on XMP.

Prefs>Sync Activity says nothing is happening anymore as expected, so I took some baby steps in using Classic and Cloudy together. I created a new sync'd collection in Classic and it appeared quite quickly in Cloudy. Then I created a new album in Cloudy, but it has not appeared in Classic. Moreover, when I started looking for it in the "From Lightroom" Collection set in Classic, I saw that most of the collections were empty. As far as I can see all the Cloudy Albums are showing as collections but most have 0 as the number of pics. Although Sync Activity shows no activity, Classic is uploading steadily according to Little Snitch network monitor, and it is also using a lot of CPU. Do I just need to wait it out or does something sound wrong to you?

Thanks again for all the help.

PS the two stuck downloading files (reconnect old iPhone....) from first attempt reappeared as I feared, but I was able to delete them on the web interface.
 
Prefs>Sync Activity says nothing is happening anymore as expected, so I took some baby steps in using Classic and Cloudy together. I created a new sync'd collection in Classic and it appeared quite quickly in Cloudy. Then I created a new album in Cloudy, but it has not appeared in Classic. Moreover, when I started looking for it in the "From Lightroom" Collection set in Classic, I saw that most of the collections were empty. As far as I can see all the Cloudy Albums are showing as collections but most have 0 as the number of pics. Although Sync Activity shows no activity, Classic is uploading steadily according to Little Snitch network monitor, and it is also using a lot of CPU. Do I just need to wait it out or does something sound wrong to you?

Cancel that. All has come good overnight. :)
 
Good.....I meant to post last night to the effect of "give it a bit more time"! It can take a while to get itself fully synced after doing a large "catalog restore" from cloud to Classic.
 
Good.....I meant to post last night to the effect of "give it a bit more time"! It can take a while to get itself fully synced after doing a large "catalog restore" from cloud to Classic.

I have tendency not to give things enough time to sort them selves out!

Thanks again for all the help and encouragement.
 
A couple of follow up questions I meant to ask please:

1. This new Classic Catalog and its masters are on an external Samsung T5. Is there any reason why I should not use it attached to my MacBook, and switch between iMac and MacBook? I do this regularly with my old non-syncing Classic without apparent problem. Catalog settings obviously travel with Catalog on the T5 but different Lightroom app settings in Classic on the MacBook might cause a problem perhaps?

2. I am intrigued that you had a problem syncing Classic and Cloudy which took two weeks to sort out. Was the cause of this something you could share and hence help me to avoid?

Thanks
 
Status
Not open for further replies.
Back
Top