• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Testing backup and restore

Status
Not open for further replies.

tspear

Senior Member
Premium Cloud Member
Joined
Sep 23, 2014
Messages
2,247
Location
Waltham MA
Lightroom Experience
Beginner
Lightroom Version
Cloud Service
Lightroom Version Number
Lr Classic 7.1
Operating System
  1. Windows 10
I no longer have spare computers to wipe and rebuild to test my backup processes.
Also, the amount of data is getting a little out of hand.

So what methods are you guys using to test your backup process?

Tim
 
I'm only testing a part of my backup:
The online backup (Crashplan) and the offsite BluRay backups i don't test. The only test i do is the external harddisc copy but i don't do it often enough...
Occasionally i connect the both EHD (a onsite copy and offsite copy) to my bacbook, open the most recent catalog, reconnect the images and do some checks (comparison of numbers) and do a DNG validation.
 
I am debating standing up a virtual desktop on AWS for a few days and test by complete restore flow.

Tim
 
I am doing more and more testing on VM's. All it takes is a lot of storage to be able to test most anything, and Windows 10 Pro (not home) has absolutely terrific VM support.

I also keep an old laptop around, but lack of storage on it makes things harder.

But the main test I do is periodically go after a specific file, notably if I go to work on something for the first time in years, it will remind me to check that that area is still backing up, make sure I can retrieve a copy of that data from backup.
 
I no longer have spare computers to wipe and rebuild to test my backup processes.
Also, the amount of data is getting a little out of hand.

So what methods are you guys using to test your backup process?

Tim
For local backup, I use an application called Retrospect, which is very nice because it backs up multiple versions of a file, instead of overwriting files the way file sync programs work. More often than I would like, I have to restore a file or folder because of "user error." Always works. Retrospect backups go to a physically separate HDD. I purge these backups after three years.

Whenever I do an import into Outlook, Edit: Lightroom, I make a Second Copy, on a different physical HDD. Several times now, again due to user error, I have had to do a re-import from that Second Copy.
 
Last edited:
Including al your photo's, how much data is that?
My Crashplan took month's to finish.

I have just under a TB., it takes about two-three days on average. I have "high speed", as in 750mb sustained up/down.

I am doing more and more testing on VM's. All it takes is a lot of storage to be able to test most anything, and Windows 10 Pro (not home) has absolutely terrific VM support.

I also keep an old laptop around, but lack of storage on it makes things harder.

But the main test I do is periodically go after a specific file, notably if I go to work on something for the first time in years, it will remind me to check that that area is still backing up, make sure I can retrieve a copy of that data from backup.

Yeah, I was thinking the remote VM route since I do not have enough local space. Maybe I should just go get a cheap 2TB hard drive to use for a VM to test restore functionality on.

Tim
 
For local backup, I use an application called Retrospect, which is very nice because it backs up multiple versions of a file, instead of overwriting files the way file sync programs work. More often than I would like, I have to restore a file or folder because of "user error." Always works. Retrospect backups go to a physically separate HDD. I purge these backups after three years>

Whenever I do an import into Outlook, I make a Second Copy, on a different physical HDD. Several times now, again due to user error, I have had to do a re-import from that Second Copy.

Phil,

I have a personal script I have built up over time. So far it has served me well, but I can see the day where I am not gonna bother with it.
At this point, I am more curious about ways people are testing the restore capabilities. Such as you actually installing Lr and making sure the catalog and images are there. What about plugins....

Tim
 
Yeah, I was thinking the remote VM route since I do not have enough local space. Maybe I should just go get a cheap 2TB hard drive to use for a VM to test restore functionality on.
Test USB connected drives first, I'm not sure if they can be used, there are some limits to what connects through, but I think local VM's are a great way to do all sorts of testing. EC instances are similar, but it always bothers me a bit that the meter is running while I'm experimenting.
 
I have just under a TB., it takes about two-three days on average. I have "high speed", as in 750mb sustained up/down.



Yeah, I was thinking the remote VM route since I do not have enough local space. Maybe I should just go get a cheap 2TB hard drive to use for a VM to test restore functionality on.

Tim
Tim,

Hard drives are getting ridicously cheap these days, especially the bigger 3.5" drives. My GO TO brand is HGST, based on drive failure rates reported by Backblaze. Check out HGST DeskStar NAS 3.5" 4TB 7200 RPM 128MB Cache SATA 6.0Gb/s High-Performance Hard Drive for Desktop NAS Systems Retail Packaging 0S04005 - Newegg.com

Throw that bare drive into this case: ORICO Aluminum Alloy & ABS 3.5 inch USB3.0 External Hard Drive Enclosure for 9.5mm .5 inch SATA HDD / SSD 8TB Max Support UASP Windows,XP,Linux,Mac OS 9.1 or above Tool Free -Gray (MD35-U3) - Newegg.com

I have one of these bare drive "readers" and it works great. Computer Parts, PC Components, Laptop Computers, LED LCD TV, Digital Cameras and more - Newegg.com
 
Phil,

I have a personal script I have built up over time. So far it has served me well, but I can see the day where I am not gonna bother with it.
At this point, I am more curious about ways people are testing the restore capabilities. Such as you actually installing Lr and making sure the catalog and images are there. What about plugins....

Tim
Tim,

Retrospect has backup scripts, with more flexibility than I know what to do with. Actually, one of my scripts backs up folder C:\Users\phil\AppData\Roaming\Adobe\Lightroom, which has a subfolder for all plug-ins. Now, I didn't say that Retrospect is exactly cheap.:( it's not, but for me it is totally worth it. :) Retrospect: Windows > Overview
 
@PhilBurton

cool product. If I switch, I am gonna choose something bullet proof for a dumb user that is set and forget. :D
My current script has all sorts of intelligence; which is likely kinda stupid; and I would never do this now. It really is a waste of time, but I have built it up over the years, swapping in/out some technology as things change. A few examples of the crazy stuff I have in the script
-- Local git repository for documents to handle version management that auto checks in local changes and replicates the change repository to my remote sync.
-- Remote sync setup for photography that covers the images/catalog/settings with a date based intelligence. Basically it only deletes from the remote after 60 days.
-- Remote git repositories for source code control, financial systems...
-- Auto prune of Lr catalog backups

Anyways, trying to see who else is paranoid enough to test his/her backup restore solution; and how they did it and verified it.

Tim
 
@PhilBurton that looks worth checking out, I use Cloudberry which seems similar - versions, cloud or hardware based -- but they changed their licensing some time ago to limit the cheap licenses to 1TB, which was really stupid. Otherwise it's a terrific product which has been bug free for me for ages, I've used it on CIFS shares, USB drives, Backblaze B2, Amazon S3 and Glacier all without trouble (their Amazon Cloud Drive was not very robust when I tried it).

I'm a big fan of (paid if needed) 3rd party tools as opposed to a cloud vendor's own -- there have been too many cases of cloud vendors short changing users in some fashion through their tools - throttling, limiting retention periods, etc. Plus they know your encryption keys.
 
So what methods are you guys using to test your backup process?
I run several backup systems. First is Time machine that does incremental backups every hour alternating between our RAID server and an external hard drive. Then I do an automatic bootable backup every night of my main computer. Monthly bootable backups on additional hard drives. One of those is always swapped out to a fire safe in a location about 100 miles away so safe from our expected natural disasters. Another is in another media rated fire proof safe in another building on our farm and a third is in a file cabinet near my machine.

Laptop has Time Machine backing up on a partition on my main computer and an external hard drive. Bootable backups regularly but not automatic.

LambTracker code is in GitHub. A few Scrivener documents are in DropBox and a few thingsare in google drive but otherwise basically no cloud solutions.

Picture files are in a partition on the server. That partition is backed up nightly to an external hard drive. The monthly bootable backups of my main machine also include a partition that is a backup of the server partition with my photos.

I test the backups every month by booting both machines from their latest bootable hard drives and opening at least one or 2 files in my major software packages.

I also have checksum verification on all files running on the daily backups and on my monthly backups to find corrupted and replace corrupted files.
 
@PhilBurton



Anyways, trying to see who else is paranoid enough to test his/her backup restore solution; and how they did it and verified it.

Tim
Yes, I am that paranoid, and for good reason. To me, the main reason for backups is not hardware failure, it's operator error.
 
I also have checksum verification on all files running on the daily backups and on my monthly backups to find corrupted and replace corrupted files.

Oogie, what do you use for checksum verification?
 
Status
Not open for further replies.
Back
Top