• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • It's Lightroom update time again! New cameras (including the Canon R5/R6), lens profiles and bug fixes, and the ability to disable built-in lens corrections for specific new cameras. Here's the usual list

Large External HDD -- What are You Using?

Joined
Sep 3, 2017
Messages
12
Location
So California
Lightroom Experience
Intermediate
Lightroom Version
Classic
I'd appreciate hearing what any Mac users are using for their larger external disk storage needs -- specifically the make/model, as well as "If you had to replace the setup you have today, would you replace it with something similar?"

Background: As a non-photography professional, I have ~16TB of Lightroom library & catalog backups (catalog on primary SSD), multimedia and other files on external storage. I'm presently enjoying single-disk redundancy with my Drobo DAS at Thunderbolt 3 speeds, which makes life super-simple when HDD fail to swap in a replacement, and it looks to macOS like one big drive to simplify everything I do and the apps I use. I have a second Drobo that is kept off most of the time, to create additional sets of 3.5" HDD for backups -- one copy on-site in a fireproof box and to "take-with-me in an emergency", and another set I rotate in/out of my offsite bank safe deposit box. Time Machine handles near real-time incremental backups of my SSD to a small USB HDD, and as a last resort I have Backblaze maintaining a copy of my SSD data (including Lightroom Catalog) as well as my actual Lightroom media library into the cloud. Use of cloud storage as a primary and active data store is not an option for me, given my limited internet speed and monthly data limits where I live. I'm perhaps over-doing my backup strategy given this is for personal use, but having spent 30 years in IT and helping customers with their own disaster planning and actual recovery, well, "it is, what it is" and I try to practice what I preached. ;)
Years ago I outgrew what a single external HDD could handle in those days, and it became complex having 3-4 separate HDD to house my growing data needs especially with some apps I still use today that are insistent upon having all files in a single master directory heirachy. I migrated to an early Drobo before NAS were really a thing in the consumer space. I had too many hardware issues with that first gen Drobo as others did, so ultimately switched to a pair of Synology NAS -- one live, the other kept off most of the time for incremental data backups in a pattern similar to what I'm still doing today. I had and have no need for the other things a NAS can offer (multiuser, data sharing, VM, mail, multimedia server, surveillance, etc.) Compared to just plugging-in external HDDs or a Drobo that can look like a single drive to the OS, as geekie as I can be, I found the whole NAS thing to be a lot of additional work to maintain another OS for just a large data store; It required I use additional backup programs and routines (the NAS can't e.g. make a bootable image copy of my SSD that I maintain); I missed the ease-of-use of what looks like a single (non-networked) external drive with my OS, and even at 1Gb network speeds (max in my home with no LDAP possibility), it sometimes took days e.g. to create even incremental backup copies. So, as crazy as it was, a couple years later, I switched back to Drobo again where I love the hardware reliability of more recent generation HW & SW, the simplicity and Thunderbolt 3 speed. Sadly the Drobo is a noisy beast in my office/spare bedroom that is getting more on my nerves every day with it's ever-running fan. I won't break the warranty to try replacing/soldering a more quiet fan as the net/YouTube suggests is possible. Also, most every time there is an OS change (like is about to happen again with Big Sur), I must be careful to wait for Drobo to update their software to work with the OS, and I've had incremental OS updates cause access to my Drobo fail more than once until Drobo patches their code.​
So, as crazy as it will seem to most of you, here I am again, considering another change of my external HDD needs. My current contenders in priority sequence (at least for now):​
  1. Go back to a new Synology NAS. The question then becomes if I buy a 2nd NAS to create physical backups as I've done before and use all the existing 3.5" 10TB drives I already own, or add another ~$1K into new 12-14TB USB HDD to accomplish the task.
  2. Just bite the bullet and stay with my paid-for (but loud) Drobo setup. I'll sure save some dollars this way.
  3. Switch to couple big external 12TB USB 3.0 drives to house the data I presently have on my Drobo (plus 4 more so I have two backup sets), and loose the physical disk redundancy I presently enjoy (granted, I don't need 7x24 access, but when there is a HDD failure, and I've had it happen 3 times over the years in my HDD, NAS and Drobo days, being able to just put in a new drive and walk away from a Drobo or NAS makes life so much simpler while still being able to access my data.) This solution will initially cost me roughly the same as if I buy 2 NAS and use existing HDD. With data growth though, I'll likely have to expand to a 3rd production drive within 18 months (meaning 3 more drives and then another $1K, vs using 2 of my spare 3.5" HDD I already own), and I'm sorta back to where I began this journey years ago -- lots of drives to contend with.

Thanks in advance with ideas of what you're using, so I can consider them in my decision making.
 
Joined
Sep 3, 2017
Messages
12
Location
So California
Lightroom Experience
Intermediate
Lightroom Version
Classic
OK, while I've not received any feedback --sorry I scared people off with too much background -- I thought I'd close out my own thread with the approach I've decided upon should it benefit someone else one day...

Net is: I'm going to migrate back to a NAS for my bulk storage needs, and use a dual set of 14TB USB drives as my physical backups.

Assuming all goes well with the build and it meets expectations, my end configuration and redundancy/backup approach should look like this:

Mac SSD including Lightroom Catalog, apps & preferences​
  • Continuous
    • Time Machine versioning to an independent always-on USB drive
    • Backblaze cloud backup
  • Monthly (or more often) Backup
    • Bootable disk image to external NAS
    • Bootable disk image to a standalone USB drive maintained with the NAS backup rotation sets
Synology NAS including Lightroom Library, multimedia and other storage​
  • Physical 2-disk drive redundancy
  • Monthly (or more often) Backup
    • Data is logically split across external 14TB USB drives (IOW, total NAS data is greater than any single available drive)
    • Synology DSM (Operating System) and configuration backup to
      • Mac SSD
      • External USB flash drive maintained with the NAS backup rotation sets
External Backup Sets
  • Two backup sets are maintained, each including
    1. Bootable USB Mac SSD disk image
    2. Synology Share (data) backups, separated onto 14TB external USB drives
    3. Synology DSM config on it's own USB flash drive, for physical NAS recovery
  • The current "monthly update" set is maintained in a portable home fireproof box (it's on my emergency take-with-me list), which is then rotated with another set in my offsite safe deposit box every couple of months. The frequency for physical backups and offsite swaps are all a question of my risk tolerance, and has been known to change depending on what's going on (e.g. lots of new data, lots of new or important photos, file reorganization, and OS updates.)
Yes, this is overkill for most non-pros. I get it. OTOH, I'm confident this approach would also work for a professional if cloud-based backups of large data volumes isn't an option for them, as it is for me. This keeps me happy knowing I have physical redundancy if my SSD or any hard disk fails, and there are two other backups of my personal data, multimedia, photos, scans of old family photos and 35mm slides that no longer exist.

Beyond the build and data migration to put the new NAS into production (which will take days to just copy data from the Drobo to NAS), the remaining decision is if I use DSM Hyper Backup or my Mac-based Chronosync backup utility for the NAS Shares (data). I have more research and decisions to make given past experience with both. Each alternative has usability and underlying technical pros and cons. I went back-and-forth on that choice with my previous NAS ownership, and learned a lot what happened when I moved away from the NAS back to DAS I now have. I'm leaning toward using Chronosync on my Mac (with the exception of the DSM config itself that must be performed via the NAS utility.) That allows me to automate creation/update of both bootable macOS images and backup copies of all my data (including photos with XML) readable via Mac or PC without any specialized utility should it ever come to that -- start it, and come back a few hours/days later when it's done. More research is needed while I await delivery of my new gear, and some tests are necessary before I settle on my approach.

In any case, remember to backup your precious Lightroom catalog, photos and data to at least two other places ...just in case!
 
Last edited:

PhilBurton

Lightroom enthusiast (but still learning)
Premium Classic Member
Premium Cloud Member
Joined
Nov 16, 2015
Messages
2,263
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic
I'll jump in here, even though I only skimmed these two posts. (TMI). I use a Western Digital 4 TB Easystore portable drive. https://www.bestbuy.com/site/wd-eas...lash-drive-black-blue/6290668.p?skuId=6290668

I use both a desktop and a laptop (out of the house, or travelling). My desktop has a dedicated 4 TB drive built-in. My laptop's storage is much more limited, so I use the Easystore drive to mirror image that 4 TB internal desktop drive.
 

mcasan

Active Member
Joined
Apr 26, 2015
Messages
544
Location
Atlanta
Lightroom Experience
Intermediate
Lightroom Version
Classic
One Sabrent 2TB Gen 4 SSD for boot, apps, caches...etc.
Two Sabrent 2TB Gen 3 SSDs in RAID 0 for 4TB storage of documents, pictures, videos...etc.
One 8GB HGST HDD for system backup
All drives internal. With many more internal SATA ports open, no need for external drive.
 
Joined
Dec 3, 2016
Messages
244
Location
Milton Keynes, UK
Lightroom Experience
Beginner
Lightroom Version
One Sabrent 2TB Gen 4 SSD for boot, apps, caches...etc.
Two Sabrent 2TB Gen 3 SSDs in RAID 0 for 4TB storage of documents, pictures, videos...etc.
One 8GB HGST HDD for system backup
All drives internal. With many more internal SATA ports open, no need for external drive.
Surely a good system crash, lightning strike etc could potentially take the lot out?
 
Joined
Dec 7, 2007
Messages
2,133
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
For large storage, I use WD external drives. Many of the 8-14 TB EasyStore and Elements drives have relabeled WD Red drives or HGST drives. It is a bit of a lottery, but I have done well with 2 of my last 3 purchases, and the one that was an issue I immediately exchanged so I am now 3 for 3. My preference for drives that are not used daily is external drives as I can unplug them from power and my PC. This reduces my exposure to power or malware issues. I prefer KISS, but then again, YMMV.

--Ken
 
Joined
Jun 24, 2010
Messages
1,685
Location
Encinitas, CA USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Hi Bert,

I have a couple of questions. First do you really need to keep your entire archive spinning all the time? It could make sense to some portion of it to offline status. You can still keep in your catalog for searching and metadata updates and only bring the disks online when and if you need to edit. Also smart previews could be extremely helpful here by allowing you to edit and export versions of the offline images.. You might want to take a look at Peter Kroug's excellent multi-media eBook Organizing Your Photos with Lightroom 5 for ideas and strategies.

Also you don't what physical interfaces you have aside from USB 3.0. I believe that any Mac that has USB3 also has some version of Thunderbolt. Thunderbolt open's up a whole range of possibilities. You can attach a single chassis that has room for 2, 4, 8 or possible more HDs/SSDs in one box with one connection to your computer. Configured as a JBOD (just a bunch of disks) this creates a lot of flexibility. With your archive broken down into smaller chunks each is easier to deal with say for backup or migrating to newer drives.

Just a note on RAID (redundant array of inexpensive disks) used on any NAS. This was initially developed because in the 90's the data sets were rapidly outgrowing the available disk sizes (A really huge HD at the time was 1 GB.) Some inventive folks realized that with software they could string a bunch of disks together to look like one single disk to handle the increasing data set sizes. A nice side effect was that you could spread the data across multiple drives and also increase I/O performance. With today's massively huge 12TB and larger drives and the fact that Lightroom functions just fine with multiple hard drives and obviates the need to put your whole archive in a single RAID partition.

Eschewing RAID and moving back to single drives would, I believe, simplify your disk management and backup strategies. With a single Thunderbolt cabinet you can have all your active date online and one or more empty slots to allow for hot-swapping online data. Just run Time Machine on the drives that have data that is being actively updated and just keep several clones of each of your archived data.

Just some food for thought.

-louie
 
Joined
Jan 18, 2009
Messages
2,325
Location
Fort Myers, FL
Lightroom Experience
Advanced
Lightroom Version
Classic
Just a note on RAID (redundant array of inexpensive disks) used on any NAS. This was initially developed because in the 90's the data sets were rapidly outgrowing the available disk sizes (A really huge HD at the time was 1 GB.) Some inventive folks realized that with software they could string a bunch of disks together to look like one single disk to handle the increasing data set sizes.
Well, to be fair, the "redundant" part was more the initial aim than the "array" looking like one drive. Disk MTBF was awful back in the day.

Certainly your point relative to Raid-0 (i.e. striped, non-Redundant raid) in Lightroom is right on -- Raid-0 adds a lot of complexity, especially if there's ever a failure, that you just don't need for the little performance improvement it brings (and even questionable if in LR usage it would provide any).
 
Joined
Sep 3, 2017
Messages
12
Location
So California
Lightroom Experience
Intermediate
Lightroom Version
Classic
Appreciate the additional input and suggestions from everyone. After nearly 2 weeks of work, I am though approaching the end of my migration from DAS to NAS for my daily environment. I’ve started to wipe some of my previous 3.5” drives and devices in prep for sale.

My NAS now has dual-disk redundancy, with a “little” 300GB Lightroom library that is growing faster all the time, but as importantly to me, over 16TB of movies, TV, music, and other data, that I can access from my Mac, mobile devices and home entertainment systems any time. I have an empty drive slot in the NAS chassis to immediately give me another 12TB+ if needed some day, and can incrementally swap drives with higher capacity versions to gain even more space while still accessing my data and retaining a single physical disk failure while that automatic expansion process is underway. The NAS spins the drives down when not in use, a hibernate possibility to automate turning it on/off based on TOD, and a simple trigger to programmatically wake the device during hibernation if needed. The NAS is somehow far quieter and produces less heat than either of my former dual Thunderbolt 3 Drobos did. I have complete backups of my Mac SSD and the NAS fully automated (including NAS configuration) from the Mac side by starting a single task, with one set maintained at home in a portable fireproof box and a second rotated offsite to my safe deposit box. Time Machine is taking care of near real-time incremental backups of my SSD. As a last measure of defense between physical backups, I have Backblaze backing up my SSD data in near real time, and their B2 service for only my photos (which adds ~$12/year with my current Lightroom library size.)​
Yes, my NAS choice is overkill for most, and certainly more complex than going the multiple-drive DAS route — but given my 30+ years as an IT guy and working with customers on disaster planning and actual recoveries after actual natural disasters, I like the redundancy, simple footprint, and ability to easily expand and resolve individual disk failures that will surely occur again with time in any solution — even for me in my little non-professional home environment. I don’t have to maintain more Lightroom previews on my fixed-size SSD, or split growing live data across physical drives. I did the research and have my NAS locked-down security wise to the network, and use physical device encryption on the little bit of personal data I care about the most, as well as all external backup drives. The greatest problem I anticipated before choosing this alternative, is the NAS transfer rate in my home is 1Gb max — a huge difference from Thunderbolt 3 or even USB 3 speeds. That means it takes a lot longer to do physical monthly backups than it did before, but fortunately with the automation I’ve created using Chronosync, I plug in my 3 USB drives for one of the backup sets, and with a couple of clicks, let the processes run for however many hours (or days) it takes to make incremental changes to the 17TB+ backup set. While backups are *really* slow, I’ve not yet seen any speed problems in a normal working environment (even with the backup working simultaneously), including my interaction with Lightroom that remains as fast as I desire — and I’m really picky in terms of responsiveness.​
Again, thanks to all for your input and advice. I enjoy this helpful community and continue to learn so much. I’ve almost got this data storage thing back under control, and it’s time to go take some pictures!​
 

mcasan

Active Member
Joined
Apr 26, 2015
Messages
544
Location
Atlanta
Lightroom Experience
Intermediate
Lightroom Version
Classic
Surely a good system crash, lightning strike etc could potentially take the lot out?
Yes, a lightning strike could take out the house. But photography is a hobby for me and not an income source. If I wanted to protect an income source I would have backup drives rotated to a bank vault and also use an online storage system.
 
Top