• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Using Time Machine to back up Lightroom and image files with Synology RT2600ac and external drive

Status
Not open for further replies.

Jeffrey Saldinger

Member
Premium Classic Member
Joined
Mar 25, 2014
Messages
26
Location
Astoria, New York
Lightroom Experience
Intermediate
Lightroom Version
Classic
Lightroom Version Number
Lightroom Classic version 9.3
Operating System
  1. macOS 10.14 Mojave
My actual question is in the last paragraph. The other paragraphs set what I hope will be useful context.

The disk in our Time Capsule has died. I’m using it as only a router for now (ethernet connection to Verizon Fios), and have ordered a Synology RT2600ac router because it can 1) serve as a new router (the Time Machine is about 12 years old) and 2) it can accept an external hard drive that can be used for wireless Time Machine backups from our two MacBooks and one iMac. Therefore, the Synology+external drive should serve as a Time Capsule replacement. (We also use Carbon Copy Cloner each day, so we’re nicely redundant at home.)

I have read that while the Lightroom catalog is in use, it protects itself from being corrupted during a Time Machine backup by somehow preventing Time Machine from backing it up; the backing up happens when Time Machine next runs and Lightroom is closed.

In the early days of setting up the Time Capsule, I excluded altogether my Lightroom catalog and image files from being backed up by Time Machine (I used Carbon Copy Clone instead), but now, as an extra degree of redundancy, I’d like to set up the Synology+external drive to back up everything (I’ll be using a new 2TB SSD) with Time Machine.

Assuming that what I read about Lightroom protecting itself during Time Machine backups is correct, are my image files also safe from corruption (during a Time Machine backup) while I’m working on them? (The files in question are on the MacBooks and iMac; I’m not asking about images files that exist only on external drives, for which I use Carbon Copy Cloner).
 
Remember, LR is non-destructive so it is not changing the actual image file. So Time Machine will be able to back the images up while LR is running. If you are also writing to sidecar XMP files, those will get backed up as long as they aren't open the moment Time Machine attempts to write them, but it will catch them on the next pass.
 
"I have read that while the Lightroom catalog is in use, it protects itself from being corrupted during a Time Machine backup by somehow preventing Time Machine from backing it up"

Hmm, I've never read that and I'm pretty sure it isn't true. Do you have a link?

The issue with Time Machine and LR catalogs is that, when LR is open, Time Machine can grab an inconsistent snapshot of the catalog database and the transaction logs that record updates to the database when LR is open. If you tried to restore a catalog from such an inconsistent snapshot, LR may not be able to open it.

But if you close LR periodically, Time Machine will be able to make a consistent snapshot of the catalog. I generally make sure to exit LR every night, so I know that I'll have a good backup at least once a day, and I don't worry about having LR make it's own backup copies on exit (I turn that off).

And to build on Woodbutcher's reply: The truth about your LR edits is kept in the catalog. So if you have a good backup of your catalog, all your edits and changes to metadata are safe.

If you've enabled the option Automatically Write Changes To XMP, then LR will also save the edits and metadata changes to .xmp sidecars for raws and into the photo files themselves for non-raws. But LR doesn't require consistent point-in-time snapshots of the catalog and the photo files. So it's OK if, after restoring a catalog and photo files from backup, a photo's metadata stored in the catalog doesn't match what's in the photo file. LR will usually update the photo file to have the new data, or in rare cases, tell you that the restored photo file's metadata is in conflict with what's in the catalog, at which point you can tell LR to overwrite the photo file's metadata.

Lots of technical detail here, but the high level is: Make sure you exit LR at the end of the day, and your catalog and photos will be safely captured by Time Machine.
 
It is the Lightroom Classic Catalog file that could be at risk. And even then it is not really at risk, only the Time Machine backup of that master catalog file would be suspect. If Lightroom has this file open for writing it was in an unstable state, then potentially any recovery from one of those backed up copies could be corrupt. Any back up made by Lightroom is a copy of the catalog file that is no longer open for write. So. Time Machine backups of these Lightroom Backups are alway safe to restore from. Depending upon the system backup software being used, some will not backup an open file and will wait until the Lightroom process has closed the file before backing it up.

When Lightroom is running, the catalog file is never in any danger from corruption from the backup process. If the back up process makes a backup of the catalog file when it is in an unstable state, then that backup file might be un the same unstable state when recovered.

I used to exclude the Master catalog file from Time Machine backup and only backup the file that Lightroom makes on the exit backup.

I’ve since not considered the Master catalog state and it now gets included in the Time Machine backup. However, should I need a recovery of the catalog I would first recover from my TimeMachine Backup of my Lightroom Backup file.


Sent from my iPad using Tapatalk
 
Hi,

I just took a quick look at the product info for this router. It looks like an outstanding WiFi router. However, I would be cautious about using TimeMachine. It is almost certainly going to present any attached hard drive as a NAS device to your Mac. While TM will work on a NAS it is implemented using a special disk image called a sparsebundle.

In my experience using a DS1511+ as NAS for Time Machine the sparsebundle would periodically get corrupted such that was not recoverable and I would have start my backups from scratch. I finally gave up and switched to using TM on a locally attached 6TB drive for a fraction of the cost of DS1511 system.

At the time I found numerous reports of problems similar to mine when using TM on a NAS. Perhaps using a single HD as opposed to a RAID as the NAS will be more reliable.

Something to keep in mind as you proceed.

-louie
 
I agree with Louie about the WiFi router. I have an ancient TimeCapsule but I also use a locally attached HDD for an additional Time Machine backup.

My ISP provides me with a Modem/Router and this is my internal network. Both wired (gigaBit) and Wireless. I don’t see the necessity for a separate router through the timeCapsule.

As for TimeMachine backups, I think a locally attached HDD sufficent. If you have more than one machine, then an EHD for each is less than $100USD a piece.


Sent from my iPad using Tapatalk
 
Remember, LR is non-destructive so it is not changing the actual image file. So Time Machine will be able to back the images up while LR is running. If you are also writing to sidecar XMP files, those will get backed up as long as they aren't open the moment Time Machine attempts to write them, but it will catch them on the next pass.
Thank you, Rusty. I do have "Automatically write changes into XMP" checked.

Have you any thoughts on whether a PSD would be in any jeopardy if it is open in Photoshop when a Time Machine back up starts or is opened when one is in progress?
 
"I have read that while the Lightroom catalog is in use, it protects itself from being corrupted during a Time Machine backup by somehow preventing Time Machine from backing it up"

Hmm, I've never read that and I'm pretty sure it isn't true. Do you have a link?

The issue with Time Machine and LR catalogs is that, when LR is open, Time Machine can grab an inconsistent snapshot of the catalog database and the transaction logs that record updates to the database when LR is open. If you tried to restore a catalog from such an inconsistent snapshot, LR may not be able to open it.

But if you close LR periodically, Time Machine will be able to make a consistent snapshot of the catalog. I generally make sure to exit LR every night, so I know that I'll have a good backup at least once a day, and I don't worry about having LR make it's own backup copies on exit (I turn that off).

And to build on Woodbutcher's reply: The truth about your LR edits is kept in the catalog. So if you have a good backup of your catalog, all your edits and changes to metadata are safe.

If you've enabled the option Automatically Write Changes To XMP, then LR will also save the edits and metadata changes to .xmp sidecars for raws and into the photo files themselves for non-raws. But LR doesn't require consistent point-in-time snapshots of the catalog and the photo files. So it's OK if, after restoring a catalog and photo files from backup, a photo's metadata stored in the catalog doesn't match what's in the photo file. LR will usually update the photo file to have the new data, or in rare cases, tell you that the restored photo file's metadata is in conflict with what's in the catalog, at which point you can tell LR to overwrite the photo file's metadata.

Lots of technical detail here, but the high level is: Make sure you exit LR at the end of the day, and your catalog and photos will be safely captured by Time Machine.
Thank you, John. I've attached a PDF of what I read about Time Machine and Lightroom protecting itself (taken with my iPhone from the book itself).

I'll need a little more time to think through the rest of your reply and get back to you with follow-up if appropriate. Thanks again.
 

Attachments

  • Martin Evening on Time Machine and Lightroom IMG_1571.pdf
    260.4 KB · Views: 171
In my experience using a DS1511+ as NAS for Time Machine the sparsebundle would periodically get corrupted such that was not recoverable and I would have start my backups from scratch. I finally gave up and switched to using TM on a locally attached 6TB drive for a fraction of the cost of DS1511 system.

At the time I found numerous reports of problems similar to mine when using TM on a NAS. Perhaps using a single HD as opposed to a RAID as the NAS will be more reliable.

I saw that type of corruption even in Apple TIme Capsule backups, which are like an NAS in that the backups are to a device over the network. I wonder if that has improved (become more reliable) in the more recent versions of macOS that use the much more robust Apple File System (APFS) instead of the rickety old HFS+.

From Time Machine to APFS: How processes have changed by Howard Oakley:
(TMH refers to Time Machine HFS+, and TMA refers to the current snapshot-based Time Machine APFS)
Using snapshots to store backups brings great rewards too. In TMH, the backup store is a single HFS+ volume which, over time, accumulates millions of files, directories, and hard links to both files and directories. This is inevitably prone to failure, and checking and repairing such a huge file system isn’t fast or simple. In TMA, each backup is single file system, which is much smaller and fixed in size once backing up to it is completed. These are checked and repaired individually, although working sequentially through a long series of such snapshots is time-consuming.

TMA backups are therefore inherently more robust and easier to maintain. Their only current disadvantage is that it’s not possible to delete items inside them, although complete backups can be deleted.

Some of Howard’s other articles mention that corruption is still possible with Time Machine APFS, and the cause is often related to SMB (the network protocol), not Time Machine.

The one thing I do know about Time Machine APFS is that it rips through backups much faster than Time Machine HFS+ ever did.
 
Thank you Cletus, Conrad, and Louie for your replies and their pointing me to other consideration related to my situation.



For clarification, my wife and I have two MacBook Pros and one iMac, and we’d like to avoid using one external drive for Time Machine backups for each device -- that approach would of course work for getting the Time Machine backups we want, but it would also have the inconvenience of taking up one port on each device. This would not be an inconvenience with the iMac, but it would be on the MacBooks -- it would occupy a port that might be needed for something else and it would be inconvenient to move the external drive along with the attached MacBook if either of us wanted to take their MacBook and move elsewhere in the house to work.



The Synology RT2600AC approach would allow us to use it with one external drive attached to it and establish that one drive as the wireless destination for Time Machine backups from all of our devices. This is how the configuration appears to be a replacement for the Time Capsule’s functions (i.e., router and wireless Time Machine backup destination for all three devices). As I mentioned in my original post, the Time Capsule’s disc died and I’m now using the Time Capsule only as a router (it was my router all along; I preferred it to my ISP’s router), and would like to replace the Time Capsule altogether with a new router (i.e. the RT2600AC, which I can also use with Time Machine) before the Time Capsule router function also dies.



As for the DS1511+ and related problems with Time Machine, it appears to be an altogether different type of device from the RT2600AC, which Synology’s website says will work with Time Machine. I’ve attached a screenshot from the Synology website. Any thoughts on this with regard to its possibly presenting problems when used with Time Machine?
 

Attachments

  • Screen Shot 2021-11-29 at 10.45.52 AM.png
    Screen Shot 2021-11-29 at 10.45.52 AM.png
    124.1 KB · Views: 129
Since Apple discontinued the TimeCapsule, I’m not sure there is a network solution that I would support. I tried creating a sparsebundle on a NAS but that seemed to preclude its use as a regular file server.

I have 7 devices connected to my TB3 port on my iMac.
OWC makes hubs of the USB-C/TB3 ports. I even have one that has three TB3 out ports.

FWIW, I’m replacing MBPs with iPadPros and not losing any functionality by switching to the iPadPro. (OWC makes Hubs for these too).


Sent from my iPad using Tapatalk
 
Thanks for the continuing help.

My deliberations about what I want to do continue. As of this writing, I’m leaning toward using time machine with only my MacBook, with an attached SSD; continuing to rely on our good Carbon Copy Cloner habits of daily backups (each of us backs up daily to two external drives); and starting to use Backblaze for our two MacBooks (in regular use) and one iMac (which we don’t use very much at all and is now around twelve years old).

I’m also leaning towards buying the current-model router from our internet service provider instead of 1) using the Synology or 2) continuing to use our Time Capsule as a router only (my OP describes how its disk has died). Relatively speaking, our router needs are simple and I’m now inclined to think that using the Synology for Time Machine (i.e., with a USB drive attached) may not be an optimum solution for us.

If, for convenience, I don’t want to keep my Time Machine drive attached all day and prefer instead to do one TM backup once a day, (e.g., when I’m done using my MacBook), then I know I will lose the hourly backups that TM offers. But are there any other considerations I might want to take into account before deciding to go that route? I am aware of the fundamental difference between what CCC does and what TM does, so if I do CCC daily as I have been, would once-a-day TM backups be unnecessarily redundant (as opposed to desirably redundant)?
 
If, for convenience, I don’t want to keep my Time Machine drive attached all day and prefer instead to do one TM backup once a day, (e.g., when I’m done using my MacBook), then I know I will lose the hourly backups that TM offers. But are there any other considerations I might want to take into account before deciding to go that route? I am aware of the fundamental difference between what CCC does and what TM does, so if I do CCC daily as I have been, would once-a-day TM backups be unnecessarily redundant (as opposed to desirably redundant)?

If you don't keep the Time Machine drive attached all day but had set TM to do automatic backups, your Mac may do the hourly backups anyway but use, temporarily, some hidden space on your internal SSD. This can create problems with your available disk space, especially if you forget to attach the TM external disk at the end of the day. On the other hand, if you disable automatic TM backups, you need to remember to run them manually every time you connect the backup disk.
I personally don't use Time Machine because I don't trust it. When it works it works but when it chokes there is little you can do.
You can use CCC or Chronosync or Arq Backup 7 or other apps to continuously backup your LrC Catalog.

If the back up process makes a backup of the catalog file when it is in an unstable state, then that backup file might be un the same unstable state when recovered.
This is, indeed, a a very serious issue. But if, instead TM, you use other backup apps, they can check whether the catalog file is in use, before the backup job is run automatically (say, every 30' - or whenever a disk is attached). Most of these apps can run a script you provide them, before starting a particular backup job: If the value returned from this script is 0, the backup job runs, otherwise the backup job is cancelled (until next time)

I wrote a simple script, that tries to establish if a LrC catalog is in use, by checking whether any <catalog-Name>.lrcat.lock file exists in the catalog folder. When LrC closes the catalog, it deletes this .lrcat.lock file (unless LrC crashes - then you must delete it manually, anyway, or the catalog won't open.) You can put this LRC_LOCKED.sh script in the same folder where the <catalog-Name>.lrcat catalog resides. If you happen to store multiple .lrcat catalog files in the same folder the script will trigger if any of them is in use. Or you can put the script in multiple separate folders that contain a catalog file and use it independently, in different backup jobs. You can download it from: https://1drv.ms/u/s!ApK0vgUzpIFHgqRdRwF4ClRaCm9BBQ?e=VitHxX

Using Carbon Copy Cloner 6.x as an example, you would go to the preflight options of the backup job and set the location of the script there, as I show in this web gallery: https://adobe.ly/3lMFW4T
 
Thank you Y.K.

I've decided that for the time being, I will forgo any backing up during the day and keep to my regular habit of conscientious end-of-the-day (i.e., once I'm finishing working) backing up with Carbon Copy Cloner and Time Machine (with the automatic backup feature unchecked). And I hope to soon be able to get going with Backblaze.
 
Status
Not open for further replies.
Back
Top