Want to sync existing Classic and Cloudy Libraries....best approach?

[mikebore]

Lightroom Version Number

9.2.1 (Classic) 3.2.1 (Cloudy)

Operating System

1. macOS 10.15 Catalina
2. iOS

I currently have all my photos in both Lightroom Cloudy and Classic libraries, but not syncing. I would like to get them syncing. I would be very grateful for any comments and advice on what I think are the three possible approaches. I am prepared to research the detail implementation myself but am looking for guidance about the best major option I should use…..if I any! I may yet decide to stay where I am.

The three major options ( I think) are:

1. Dump the current Cloudy Library and create new one by syncing from Classic.

2. Dump the current Classic Library and create a new one by syncing from Cloudy.

3. Don't dump either: just turn on sync.

STARTING POINT
-1TB Photography Plan.
-Lightroom Cloudy with 79,000pics/600GB working well on two Macs and four iDevices.
-Lightroom Classic on both Macs with the catalog and masters on a portable Samsung T5 SSD which can be connected to either Mac. For many years this was my main personal working photo library. Sync is not turned on in Classic.
-When I started with Cloudy nearly two years ago I created the Cloudy library by manually exporting year based catalogs and importing them to Cloudy in turn to preserve the Classic structure in Cloudy, creating folders in Cloudy as I went.
-Since then Cloudy has been my main working library because it automatically collects all the photos my wife takes as well as mine in one place and syncs everything to all devices.
-I have kept Classic up to date with monthly manual exports of “original+ settings” from Cloudy and manually importing them to Classic. I have not really made any proper use of Classic, treating it as a kind of backup and thinking I would one day do what I am now asking about.
-The Classic Library has 76,000 photos. The discrepancy with Cloudy is due to duplicates in Cloudy caused by I believe by some initial problems with the auto import settings in Cloudy.

DESIRED END POINT
-Cloudy Library with 76,000 pics automatically syncing to all devices. No duplicates.
-Classic Library with same 76,000 syncing with Cloudy.
-No monthly manual export and imports
-Ability to use the printing and editing tools in Classic for all photos.
-Ability to do culling and rating on iPad, syncing through to Classic.

MORE ABOUT THE OPTIONS:

1. Dump the current Cloudy Library. Assume I would have to clear all existing photos from the account for the fresh start. Then I would create year collections in Classic and turn on sync so these appear in Cloudy. This would have Cloudy using Smart Previews, for the existing pics, and originals for on going, since I expect Cloudy to be my main user tool.

2. Dump the current Classic Library (save it in a drawer). I would create a new Classic library by turning on sync to populate it from the Cloudy. This would have the 3000 dupes in but there are tools to remove dupes in Classic but not for Cloudy. Hopefully deleting the dupes in Classic would propagate to Cloudy? Would all the Cloudy photos appear in Classic with any folder, album or collection structure?

3. Don’t dump either Library, just turn on sync in current Classic. Not sure what the result would be. Probably nothing initially as there are no collections in Classic. Assume I would need to put all 76000 into a whole lot of collections in Classic so that sync would occur? Would it create or eliminate dupes?

Thanks very much for any comment.

PS This thread reminds me of the joke about a local being asked for directions by a tourist, replying "If I wanted to go there I wouldn't have started from here".

 

[Jim Wilde]

1. It should work, emphasis on "should". When you try it for the frst time the shstem will either detect that it's the same catalog, in which case syncing should continue to work normally, or it will think it's a different catalog, in which case you would get a warning messsage (do you want to switch sysncing to this "different" catalog) but with the option to cancel.

2. It's a bit of a comedy of errors, but here goes:

At the time I had a couple of collections containing only those photos which weren't in any other collection/album (i.e. "to be organised"), about 11k images in all, two collections in a single collection set. Following the expansion of the Lightroom search function, which included the ability to search All Photos for "Not in any album", those collections and collection set became redundant....so I deleted the Collection Set in Classic. Normally, that would produce a warning message with the option to "leave the photos in All Synced Photographs", which is checked by default. Now it's possible that I unchecked it, though I doubt it, but several days after doing the deletion I discovered that the 11k images had been deleted from the cloud.

But no big deal, as the cloud trash can had recently been introduced, so all 11k images were there.....just restore them and job done, right? Unfortunately not.....the restore worked correctly, so cloudy was back to normal, but sadly there's yet another disconnect between Classic and the cloud in this situation. If you add images to the cloud which already exist in Classic but are not currently synced from Classic (as was the case here), then when the cloud makes them available to Classic for downloading Classic's duplicates detection kicks in and realises that the 11k images are duplicates.....in this situation it doesn't know if the user wants/intends that the downloading duplicates are meant to replace the original non-synced files, so it doesn't. Instead it creates VCs, but that leads to the situation that the VCs are the synced files, the original masters are still unsynced (plus of course the photo counts between Classic and the cloud differs by 11k).

OK, so easy enough to make the VCs the masters (so the masters are then synced) and then remove the unsynced VCs, right? Again, not so simple, as some users here recently discovered you cannot select a bunch of images and use the "Set copy as master" command and have that action done to all the selected images....only the most selected single image gets switched. Even worse, assuming I was prepared to do that 11k times even the "Auto Advance" function isn't available when using the "set copy as master' command. I quickly decided I need an alternative route.

If I had realised that the VC issue was going to occur, I guess I could have removed those 11k images from Classic before restoring them from the deleted items album in Lightroom, but I didn't. So the next thought was to rebuild my Classic catalog completely from the cloud (as you have just done), though that also isn't without its challenges.

Because I'm pretty anal when it comes to keeping Classic and cloud in sync (yes, even keywords and location data are the same in each library) I wanted to keep things that way.....the problem is that doing the catalog restore doesn't bring down the keywords and location data (the Adobe Downloader does, but that doesn't retain the collection/album structure).

Here's where my normal "anal" workflow helped, as I routinely import initially mainly into Classic and I process the images to completion. I can, and do sync them initially as SPs and so can work on them from any app....however, I ensure that keywords and location data are only done in Classic. When the images are "done" I (now) convert any raw files to DNG, then I select all and Cmd+S to write all the metadata to XMP. Then I remove the images from Classic (they're in a "holding" folder outside my normal date-based folder structure), and import them to Cloudy. Cloudy reads the XMP and applies it, so my keywords are then available (though flattened) as well as the location data, and of course all other metadata and edits. These then sync down into Classic, and are automatically stored in the normal date-based folder structure.....but because of storing that XMP into the DNGs before they are imported to the cloud, they are still in the file when it downloads into Classic....and Classic reads that metadata like any normal import, so the keywords and location data are available after the import.

Now, if all my images in the cloud were all either DNG, Jpeg, Tiff, etc., the catalog restore would have been a breeze. But they are not yet (I switched to DNG conversion back in 2017 when I realised the potential of having them in the cloud for situations such as this), all my raw files prior to that are still proprietary....so the catalog restore would lose the originally added keywords and location data. However, the original Classic library DOES have the XMP sidecars, complete with the up-to-date saved metadata, for all the proprietary raw files, and this opens up possibilities. So the restore went like this:

a) In old Classic catalog, make sure current XMP was updated.
b) Create new Classic catalog, change Lightroom Sync prefs to download to a separate drive but using the same date-based scheme as used in the old Classic catalog.
c) Turn on sync and wait for the download to complete. At this point the DNGs, Jpegs, Tiffs etc. would have the correct keywords (including hierarchy) and location data, but the proprietary raw files would not.
d) When it has, and all syncing has finished, use the "Update Folder Location" to switch the new Classic catalog back to the original files, then for the raw files do Metadata>Read Metadata from files....which restores the keywords and location data.
e) When the dust has settled, delete the library from the new drive.

I ran into some unexpected post-recovery issues, relating to re-syncing of files from Classic to cloud, i.e. for reasons I still haven't pinned down (but likely related to the fact that no previews are built automatically by the recovery process) Classic will do that whenever scrolling the library and an image without a preview appears on screen. That's a pretty standard function, but in this particular situation Classic appears to think the metadata has changed when a preview is built and so re-syncs the file.....and when the file in the cloud is an original, Classic DOES sync the original (not a smart preview) if it thinks it has changed. This is where having those DNGs is a disadvantage.....change metadata for a proprietary raw and only the contents of the XMP sidecars are synced, but change metadata for a DNG/Tiff (and when Classic thinks the file has been changed on disk, such as by a CMD+S) the whole file gets re-synced. In the early days after the recovery I would routinely find hundreds of files syncing (very slowly) to the cloud.

I don't know if you'll be affected by that last issue, but something to be aware of if you start to notice some unexpected syncing happening between Classic and the cloud.

 

[mikebore]

Wow! Thanks for taking the trouble to write all that down. I am also somewhat anal about these things, so prone to doing things that may have unexpected consequences when trying to nail down the last .01%. I will file your saga away in case I get in a similar situation. I guess the take away for me is not to make any assumptions about what will happen between Cloudy and Classic, and stick to doing things I am confident about (my 13 basic rules from earlier in the thread). The other take away is backups and recovery options.

As I said at the start one of my objectives in doing this was eliminate duplicates in Classic, hence Cloudy. That is next on the agenda.

 

[Jim Wilde]

One thing I forgot to mention, which may be pertinent to you, is that my method of rebuilding my Classic catalog - in order to retain all my original Classic-generated metadata including hierarchical keywords and all location data (and also face recognition data) - will only work (easily) if you were using a standard date-based folder structure which you can then replicate using the options on the Lightroom Sync tab. If you have your own proprietary named-folder scheme you cannot automatically relink the new Classic catalog back to the original Classic folders.

 

[mikebore]

Thanks Jim, I took on board your earlier advice about letting go of personal structures for masters and am using the standard date based folder structure.

 

[mikebore]

This is all going very well, but I have a question about working practice using them together.

I understand that if I delete a sync'd image in Cloudy will simply unsync it in Classic not delete it. So my standard practice, as you suggested, is to flag as rejected in Cloudy and then delete all the rejected ones in Classic.

Because Cloudy is set to auto import our iPhone Camera Rolls, quite a lot of junk gets into Cloudy, and therefore up to the Adobe Cloud.

Currently I have been opening Classic and letting the Classic sync complete, then deleting in Classic as above.

My question is: Can I delete the junk in Cloudy before before Classic has been opened, without Classic ever seeing them? Or does Classic still see them because they will be in the Cloud in recently deleted for example.

Thanks

 

[Johan Elzenga]

Because Cloudy is set to auto import our iPhone Camera Rolls, quite a lot of junk gets into Cloudy, and therefore up to the Adobe Cloud.

So why don't you turn off that settings, and use the built-in camera in Lightroom Mobile? Depending on the iPhone model, you can shoot in DNG if you use the built-in camera app.

My question is: Can I delete the junk in Cloudy before before Classic has been opened, without Classic ever seeing them? Or does Classic still see them because they will be in the Cloud in recently deleted for example.

Deleted photos do not sync to Classic, so yes this will work.

 

[mikebore]

Thanks very much Johan.

About using the Lightroom camera on the iPhones instead of the Apple one. If this was just me I would probably do this, and aim to use the Apple cam for the junk, (screenshots, note taking etc) and the Lightroom cam for photos that might become keepers. But my wife is something of a technophobe would never get used to this. I could and might do it to number of non keepers to just hers. The Lightroom camera widget makes it nearly as accessible as the Apple one.