Backup deltas, how?

[Ron-Alexander]

Lightroom Version Number

15.6

Operating System

1. macOS 26 Tahoe

I think I have heard/saw several members mention or allude to using 'backup' software for the catalogue and maybe additional files that are a 'delta' type.
I'm only backing up on exit and saving many of the zip files it creates. Is there a tool/step I am missing, or have I totally misunderstood what others are saying?

What about image files? I use Backblaze, Time Machine, and an external drive, plus a current copy on my system drive. Am I missing anything there?

 

[clee01l]

Backblaze, Time Machine etc. have nothing to do with Lightroom. They are critical to any sensible plan for recovering from data disasters such as the inevitable disk failure, dumb user mistakes, or catastrophic events like fire, flood, civil unrest. Every user should have such a plan in place and tested periodically so that when disaster happens, you are not trying to figure out the recovery process.

This is separate from Lightroom's "backup" process which is a snapshot copy of the catalog files (and only the catalog files). As the database is fragile enough that it might get corrupted if not careful. The Catalog backup is essential to rebuilding a corrupt catalog. and AFAICT its only useful purpose.

IOW, you need both. An automated "fire and forget" system backup app for all of your critical user files not limited to the Lightroom images, catalog and settings. And Adobe's process of copying catalog files on exit for recovering from catalog corruption and stupid user mistakes.

I use timeMachine a system backup app that is free from Apple and BackBlaze which satisfies my requirement for offsite disaster recovery.

 

[camner]

I think I have heard/saw several members mention or allude to using 'backup' software for the catalogue and maybe additional files that are a 'delta' type.
I'm only backing up on exit and saving many of the zip files it creates. Is there a tool/step I am missing, or have I totally misunderstood what others are saying?

What about image files? I use Backblaze, Time Machine, and an external drive, plus a current copy on my system drive. Am I missing anything there?

The “delta type” backup you are referring to is for backing up your filesystem (which, of course, includes your LrC catalog files, though as the LrC catalog is active when using LrC the filesystem backup of a catalog open at the time of the backup may or may not be totally up to date). Delta backups can operate at the file level (whenever a file changes in any way, the entire file is backed up again) or at the “block” level (whenever part of a file changes, only the block containing the changed part is backed up again). Cloud backup systems tend to use block level, local backups tend (but not always) to use file level backups.

 

[Ron-Alexander]

The “delta type” backup you are referring to is for backing up your filesystem (which, of course, includes your LrC catalog files, though as the LrC catalog is active when using LrC the filesystem backup of a catalog open at the time of the backup may or may not be totally up to date). Delta backups can operate at the file level (whenever a file changes in any way, the entire file is backed up again) or at the “block” level (whenever part of a file changes, only the block containing the changed part is backed up again). Cloud backup systems tend to use block level, local backups tend (but not always) to use file level backups.

That may be it. Since I am a Mac user, I forget that TM is doing that and BB as well. Since I am now 100% convinced I had a seniour moment, this is closed.

 

[Califdan]

As a Mac user the combination ot TM and BB is a good strategy.

Windows users can substitute the Windows "File History" which comes free with Windows for Time Machine.

However for both platforms, I reccomend ALSO using something that produdes a file strucure on the backup disk that is a replica of what it's backing up (e.g. GoodSync). In many ways this is redundant to TM and FH but unlike those tools the result is directly usable by LrC by just re-linking the higehst level folder(s) rather than having to use TM or FH to rebuild the files on disk from the incremental fragments.

 

[Ron-Alexander]

As a Mac user the combination ot TM and BB is a good strategy.

Windows users can substitute the Windows "File History" which comes free with Windows for Time Machine.

However for both platforms, I reccomend ALSO using something that produdes a file strucure on the backup disk that is a replica of what it's backing up (e.g. GoodSync). In many ways this is redundant to TM and FH but unlike those tools the result is directly usable by LrC by just re-linking the higehst level folder(s) rather than having to use TM or FH to rebuild the files on disk from the incremental fragments.

That might be a good idea. I used to own GoodSynch. I only backed up my home directory and excluded previews and the Arduino folder (which is huge and changes a lot). I think Carbon Copy Cloner does the same thing with deltas. I'll try that when I return from my photo workshop trip next week.

 

[camner]

That might be a good idea. I used to own GoodSynch. I only backed up my home directory and excluded previews and the Arduino folder (which is huge and changes a lot). I think Carbon Copy Cloner does the same thing with deltas. I'll try that when I return from my photo workshop trip next week.

Yes, CCC only copies files that have changed (if you set it to do so). But remember that CCC doesn’t do versioning. It’s purpose is to make the destination volume an exact copy of the source volume at the time the backup job is run, and will add, change, or delete files on the destination to make it so.

 

[Conrad Chavez]

Yes, CCC only copies files that have changed (if you set it to do so). But remember that CCC doesn’t do versioning.

Carbon Copy Cloner can support versions through APFS snapshots, the same mechanism that Time Machine uses for versioning.
I don’t think that messes up the file system clone. I think you can still relink a Lightroom Classic catalog to a CCC backup, unlike Apple Time Machine.

In short, addressing the questions raised in this thread…

Delta backups: If that means incremental backups to save time (only back up changed files) after the initial full backup has been made, that can be done by any good backup software including, on the Mac, Apple Time Machine, Carbon Copy Cloner, SuperDuper!, Chronosync, and others.

Versioning: Time Machine and CCC use APFS snapshots for this, other software may do that or have their own versioning…you have to check.

The backup uses the normal file system, easily relinkable from Lightroom Classic: Time Machine doesn't do that, the other backup software mentioned here does.

 

[Johan Elzenga]

Carbon Copy Cloner can support versions through APFS snapshots, the same mechanism that Time Machine uses for versioning.
I don’t think that messes up the file system clone. I think you can still relink a Lightroom Classic catalog to a CCC backup, unlike Apple Time Machine.

Correct. I use it to make a backup of my entire catalog folder. What CCC does is create an exact clone of that folder, and next to that is a special folder with older copies of changed or deleted files, plus some info for CCC to use that when it needs to restore an older version of the catalog folder.

 

[Ron-Alexander]

Yes, CCC only copies files that have changed (if you set it to do so). But remember that CCC doesn’t do versioning. It’s purpose is to make the destination volume an exact copy of the source volume at the time the backup job is run, and will add, change, or delete files on the destination to make it so.

Ah, that's the word I was tripping over, versions. However, the only file I care about that needs that is the catalog itself and LrC creates it every time I exit LrC.

 

[Ron-Alexander]

I have been sitting here for about 30 mins wondering if/how I should reply. I will, even if it gets me kicked off.
I asked the original question because of the answers I see in response. I am sure there is pertinent advice there, but it is not presented in words/concepts an average user can use. Everyone who can help that is reading this is saying to themselves right now, 'it's perfectly clear what is being said'. IF IT WAS CLEAR, I WOULD NOT ASK!
Let me come at this from a different angle. I will explain what I do, and folks can tell me if I am at risk anywhere.
I have (almost) all my Lightroom Classic data on my Mac's system SSD, under my home dir, in a folder called Lightroom. I say almost because I know one piece of data is missing, which is the (home)/Library/Application Support/Lightroom, which I will either add as a source folder or clone the watcher script I use for preferences to make a copy to my home/Lightroom location, the same as I do for Preferences. Looking at it, the data seems redundant with the Settings in the catalogue folder, but I need to do a detailed comparison to be sure. Screen grab 1 is my home/Lightroom folder and shows the logical chunks of data. Screenshot 2 is the contents of the Catalogues folder (Main is an alias to the current version). Screenshot 3 shows the contents of the primary backup folder, and it uses the same alias trick. A CCC task is watching the entire Lightroom folder. When 100MB of data has changed and 2 minutes elapse, CCC copies the data to the external SSD. See screen grab 4 for the watch list. Screen grab 5 shows the status of my primary backup; right now, it only sees 8Kb modified. When I do an LrC backup on exit, it jumps to over 100MB and runs a CCC run, but only if the backup SSD is plugged in. If I forget and plug it in 5 mins later, the copy will run then. I know many will say backups should go to an EHD first, but if it is not connected, it may never copy; my way at least gets the copy as soon as I plug in the EHD. NOTE: I frequently do NOT use LrC in my home office; it is on my lap, so the primary RAID EHD backup is not connected, but the 2TB SSD I use is credit card-sized, and the backup to it is a Seagate Rugged 2TB, then rubberized and roughly the size of two playing card decks side by side. My travel case contains 2 2TB SSD's for LrC and TM, and two Seagate 2TB Rugged drives as 2nd copies of TM and LrC backups, plus a laptop, extra cables, a BIG charger, and an Amazon Fire HD 8 with manuals, etc.
At some point, I will run another CCC task manually that clones the travel SSD or the home EHD to a second SSD or home EHD. Right now, all my Lightroom files are in 7 locations: travel main LrC, travel main TM, travel backup LrC, travel backup TM, home main LrC, home backup LrC, and Backblaze.
Just a few days ago, I fumbled the transition from 15.5.1 to 15.6, so I had to revert the catalogue to 'fix' my mistake. It worked as intended.
If you can see any scenario where my 7 copies will not get me back up and running with either 100% of the data I had at the last known good point or, at worst, missing the last X LrC sessions, where X is the number of times I either did not do a backup on exit or plug in the primary travel LrC backup. That is my intended worst case, and I can live with that. When I do serious editing (rarely) at my office, I have the EHDs plugged in so I can get as close to 100% data integrity as I can, given the limitations of existing technology.
The reason for my frustration, or worse, is that some of the answers to my question are not helpful for one of a few specific reasons that may or may not have happened in this thread. One is that some very competent members use correct technical words and concepts that a less experienced person will not understand, and in my experience as a teacher, many/most people will not tell you they don't get it! THAT is my biggest complaint about this forum: many responses, especially from a few very senior members, are technically correct but not helpful. Some have zero interpersonal skills.
I need my morning coffee, so I will stop here, but I hope everyone gets my point; if not, I am happy to continue, but not until I get back from my upcoming Algonquin Water Falls photography workshop Mon and Tue.

 

[clee01l]

(home)/Library/Application Support/Lightroom,

There are two Lightroom Settings folders. Global or Local
In Preferences you either choose to store settings locally with the catalog "(home)/Pictures/Lightroom/Master/Lightroom Settings" or Globally "(home)/Library/Application Support/Lightroom". The global location stores settings for ALL catalogs. The local location stores settings for ONLY this catalog. They are mutually exclusive depending on the settings in Preferences. Lightroom Classic only populates one of these automatically.

I understand that you are using timeMachine AND Back Blaze. IMO you do not need any other system backup. CCC is IMO redundant to timeMachine as is Backblaze. BackBlaze offers the additional security of an off-site backup destination In the event that your local systems are destroyed by some catastrophic event.
In describing your set up I see lots of (unnecessary?) redundancy. Perhaps you are overthinking the process. IMO you only need two system backups, timeMachine for a local backup, and BackBlaze for an offsite. If recovery from offsite were not so necessarily slow and tedious, then you could get by with BackBlaze only. The Lightroom Backup folder serves only one purpose, to recover a corrupt catalog. It makes no sense to store the Lightroom backup folder on the same folder as the master catalog. (which it appears that you have done). For when the disk fails, you lose both the master catalog and all backups of that catalog.

If you can recover your master catalog from an earlier state using either timeMachine or BackBlaze, you do not need the Lightroom Backup catalog process at all. I only keep these backup catalogs out of convenience because I have spare disks and it is easier to find the catalog from an earlier time in date named folders than to seek it out in timeMachine. Also timeMachine may back up the Catalog when Classic is active and potentially unstable. At some time in the past that meant a corrupt catalog on recovery, I don't think this is the case today with timeMachine but you can't be certain..
This brings me to another point. As important as it is to have a system backup, it is just as important to periodically test your recovery process. Have you actually tried to recover a random Lightroom catalog or folder to know that you can when you are not under stress and in panic mode? Once you are comfortable with your recovery process being successful, then you will not be in the place you are now.

As for EHDs, I see no reason to ever unplug them. I have a DAS that is always connected. So this is never an issue. However if I need a mobile computer, I can see where I might want to walk away deom externals. Perhaps another reason to backup to a NAS. My portable computer is an Apple Neo. It only backs up to the NAS regularly. If I travel with it, I carry a 1TB 2.5 in HDD which then gets the timeMachine backup.

 

[Ron-Alexander]

There are two Lightroom Settings folders. Global or Local
In Preferences you either choose to store settings locally with the catalog "(home)/Pictures/Lightroom/Master/Lightroom Settings" or Globally "(home)/Library/Application Support/Lightroom". The global location stores settings for ALL catalogs. The local location stores settings for ONLY this catalog. They are mutually exclusive depending on the settings in Preferences. Lightroom Classic only populates one of these automatically.

Ah, I forgot it was an either/or situation. I had it set to global before changing it to local. I changed it to make it easier to back up.

 

[Ron-Alexander]

I understand that you are using timeMachine AND Back Blaze. IMO you do not need any other system backup. CCC is IMO redundant to timeMachine as is Backblaze. BackBlaze offers the additional security of an off-site backup destination In the event that your local systems are destroyed by some catastrophic event.

BackBlaze is for recovery after the house burns down, when all my computers and storage devices are destroyed. Time Machine is for recovering any file, not just Lightroom. In fact, I could exclude the Lightroom folder as it is redundant, but I probably will not.

 

[Ron-Alexander]

IMO you only need two system backups, timeMachine for a local backup, and BackBlaze for an offsite.

First point, the advice I have always adhered to is the 321 backup strategy, 3 copies, 2 media, 1 offsite. Offsite is BB, the 2 media are SSD and HDD, there are at least 3 copies.

 

[Ron-Alexander]

There are two Lightroom Settings folders. Global or Local
In Preferences you either choose to store settings locally with the catalog "(home)/Pictures/Lightroom/Master/Lightroom Settings" or Globally "(home)/Library/Application Support/Lightroom". The global location stores settings for ALL catalogs. The local location stores settings for ONLY this catalog. They are mutually exclusive depending on the settings in Preferences. Lightroom Classic only populates one of these automatically.
View attachment 30544
I understand that you are using timeMachine AND Back Blaze. IMO you do not need any other system backup. CCC is IMO redundant to timeMachine as is Backblaze. BackBlaze offers the additional security of an off-site backup destination In the event that your local systems are destroyed by some catastrophic event.
In describing your set up I see lots of (unnecessary?) redundancy. Perhaps you are overthinking the process. IMO you only need two system backups, timeMachine for a local backup, and BackBlaze for an offsite. If recovery from offsite were not so necessarily slow and tedious, then you could get by with BackBlaze only. The Lightroom Backup folder serves only one purpose, to recover a corrupt catalog. It makes no sense to store the Lightroom backup folder on the same folder as the master catalog. (which it appears that you have done). For when the disk fails, you lose both the master catalog and all backups of that catalog.

If you can recover your master catalog from an earlier state using either timeMachine or BackBlaze, you do not need the Lightroom Backup catalog process at all. I only keep these backup catalogs out of convenience because I have spare disks and it is easier to find the catalog from an earlier time in date named folders than to seek it out in timeMachine. Also timeMachine may back up the Catalog when Classic is active and potentially unstable. At some time in the past that meant a corrupt catalog on recovery, I don't think this is the case today with timeMachine but you can't be certain..
This brings me to another point. As important as it is to have a system backup, it is just as important to periodically test your recovery process. Have you actually tried to recover a random Lightroom catalog or folder to know that you can when you are not under stress and in panic mode? Once you are comfortable with your recovery process being successful, then you will not be in the place you are now.

As for EHDs, I see no reason to ever unplug them. I have a DAS that is always connected. So this is never an issue. However if I need a mobile computer, I can see where I might want to walk away deom externals. Perhaps another reason to backup to a NAS. My portable computer is an Apple Neo. It only backs up to the NAS regularly. If I travel with it, I carry a 1TB 2.5 in HDD which then gets the timeMachine backup.

This isn't about 'system backup'; it's about Lightroom backup. The system backup is covered by BB and TM, the 3rd copy is the Mac HD itself.

 

[Ron-Alexander]

It makes no sense to store the Lightroom backup folder on the same folder as the master catalog. (which it appears that you have done). For when the disk fails, you lose both the master catalog and all backups of that catalog.

Perhaps I didn't explain well. The LrC backups (on exit) are first saved to the same disk as the main catalogue, but within at most 2 minutes, they are copied to an external SSD. Given how critical these files are, the LrC backup SSD is later (usually the same day) copied to a portable rugged EHD. Again, that satisfies the 321 philosophy. As you allude to, there are, in fact, several more copies of the LrC data inside the 3 system backups, but they are a little more difficult to use if/when needed. The SSD and EHD are a single directory that I can copy in whole or just the affected sub-folder. I hope you agree that some failures may be recoverable with only part of the LrC data, so copying everything may not be needed and, at worst, could be counterproductive.
Back to the same disk-for-2-minutes issue you have. If I did lose that backup, going back one version isn't a big deal. I work in short sessions, and I understand you typically leave LrC open all day, if I am remembering correctly. That change in approach does indeed need a different solution. I just happen to like mine over yours.

 

[Ron-Alexander]

As important as it is to have a system backup, it is just as important to periodically test your recovery process. Have you actually tried to recover a random Lightroom catalog or folder to know that you can when you are not under stress and in panic mode? Once you are comfortable with your recovery process being successful, then you will not be in the place you are now.

Yes, I not only tested it, but I also had to use it when I fumbled the 15.6 install, as I said in the post you replied to.

 

[camner]

Correct. I use it to make a backup of my entire catalog folder. What CCC does is create an exact clone of that folder, and next to that is a special folder with older copies of changed or deleted files, plus some info for CCC to use that when it needs to restore an older version of the catalog folder.

I may be wrong, but I don't think that CCC makes that "special folder" with older copies of changed/deleted files by default, though I do think that it can be set to do so.

 

[Johan Elzenga]

I may be wrong, but I don't think that CCC makes that "special folder" with older copies of changed/deleted files by default, though I do think that it can be set to do so.

I never said this was the default (although I think it actually might be). I reacted to Conrad, who said that CCC can be set to use versioning, and added: “I don’t think that messes up the file system clone”. So I confirmed that.

 

[Ron-Alexander]

I may be wrong, but I don't think that CCC makes that "special folder" with older copies of changed/deleted files by default, though I do think that it can be set to do so.

I do not know where they are stored, but in DiskUtility you can see them. This is from my LrC backup managed by CCC. On my system disk I can also see a _CCC SafetyNet folder. Here is the screenshot and the contents of the note.

About the CCC SafetyNet Folder
When CCC updates files on the destination, it places the older versions of modified files, and files that have been deleted from the source since a previous backup, into the _CCC SafetyNet folder. We call this a "safety net" because the alternative would be to immediately delete the older items. The SafetyNet prevents catastrophes — rather than immediately deleting items from the destination, CCC saves these items on the destination as long as space allows.

The SafetyNet folder does not contain the "current" version of your backup. If you're looking for the current backup of your files, don't look inside of this folder, look in the other folders that are adjacent to the _CCC SafetyNet folder.

How do I restore older versions of my files?
The SafetyNet feature was not designed for providing "Restore older versions of a file" functionality, and it has some specific shortcomings that make it unsuitable for that purpose. SafetyNet was designed specifically as a safety mechanism, e.g. if you accidentally selected the wrong volume as the destination to a task, then you could recover content to that volume from the SafetyNet folder.

If you want CCC to retain "backup history" so that you can restore older versions of your files, you should maintain backups on an APFS-formatted volume and enable CCC snapshot support on that volume.

Limiting the growth of the SafetyNet folder
When the SafetyNet feature is enabled for a CCC backup task, CCC will automatically prune the contents of the SafetyNet folder, by default, when the free space on the destination drops below 25GB. You can modify this pruning behavior for each task by clicking the "Use Advanced Settings" button at the bottom of the CCC window. CCC offers pruning based on size of the SafetyNet folder, age of items within the SafetyNet folder, and amount of free space on the destination. If you find that your backup tasks frequently run out of space on the destination, you can modify CCC's SafetyNet pruning limit to make more space available at the beginning of each backup task.

Can I delete items in the "_CCC SafetyNet" folder?
Yes. You can delete any file or folder in the _CCC SafetyNet folder, in fact you can delete the entire folder itself. CCC will recreate this folder as required if you have the SafetyNet feature enabled.

You can learn more about the CCC SafetyNet feature in the documentation:
Protecting data that is already on your destination volume: The Carbon Copy Cloner SafetyNet

 

[clee01l]

This isn't about 'system backup'; it's about Lightroom backup. The system backup is covered by BB and TM, the 3rd copy is the Mac HD itself.

Just think of it this way. There is no Lightroom Backup. Only your system backup.(s) What Adobe calls Lightroom Backup is just a snapshot of the catalog file at a point in time. Nothing else. No settings, no plugins, no preferences. Your System backup is your recovery avenue. Snapshots of the Catalog are to allow you to recover a corrupt catalog by going to an earlier state where it was not corrupt.

 

[clee01l]

Yes, I not only tested it, but I also had to use it when I fumbled the 15.6 install, as I said in the post you replied to.

But how did you test. Did you just grab a copy of the catalog frol the Lightroom Backup folder or did you recover an earlier file from timeMachine of BackBlaze? It is the latter that you need to be comfortable with.

 

[Ron-Alexander]

Just think of it this way. There is no Lightroom Backup. Only your system backup.(s) What Adobe calls Lightroom Backup is just a snapshot of the catalog file at a point in time. Nothing else. No settings, no plugins, no preferences. Your System backup is your recovery avenue. Snapshots of the Catalog are to allow you to recover a corrupt catalog by going to an earlier state where it was not corrupt.

I have all that backed up. See attached, that entire folder is backed up by CCC. BB is for when the house burns down. I have had mixed success with TM so do not rely on it. Perhaps I should see what is on TM that I don't have elsewhere, that's important enough, and that can't be recreated. I just did a space check with DaisyDisk and I only have 65GB that is not Lightroom and Library. Library is the tricky one especially at 135GB. I think I will deal with that when I get back from my trip.

 

[Ron-Alexander]

But how did you test. Did you just grab a copy of the catalog frol the Lightroom Backup folder or did you recover an earlier file from timeMachine of BackBlaze? It is the latter that you need to be comfortable with.

Certainly not BB. I know it's possible, but as I said, it is for a house that burned down, and they ship an HDD. I will simulate a catalogue failure and use TM to recover. A week or so ago, I just renamed the 'bad' backup, unzipped the previous backup, and copied/replaced it in the main catalogue. You are 100% correct; that tested nothing of consequence. Thank you for the reminder.