Best option for storing image files - internal or external?

[deekay2308]

Lightroom Version Number

Lightroom Classic version: 8.3.1

Operating System

1. macOS 10.14 Mojave

Currently I have all image files on my internal drive but figure that is not a good long-term storage solution. Is it generally preferred to store image files on an external drive, with the catalog on the internal drive? Any particular tips or issues that I should be aware of if I move to this method?

 

[mcasan]

You did not mention where you do backup. All of this needs to be part of a complete plan.

This is. what I do (your mileage may vary):

• Single internal drive is SSD for max speed and reliability compared to a HDD or a hybrid fusion drive
• Internal SSD is for boot/MacOS/apps/caches (and little else)
• Large external RAID 0 pair of HDD connected via TB3 contains documents, pictures, music, videos...etc.. It also contains my LR catalog backups
• External SSD connected via TB3 contains my Lr catalog along with all the previews....etc.
• Large single drive connected via TB3 is owned by Time Machine and makes backup copies of the entire file system

If I had income based on my catalog and raw images, I would like also have an online backup such as Backblaze and maybe rotate backup drives off site to a bank vault. Each person has to decide how much they value their images and what expense they will incur to protect them.

You will to either let Lr Library move the folders to the new drive or, once they are moved you will have to have Lr find them again on the new drive. Suggest you do the move via LR......only have a full backup.

 

[Califdan]

Storing images in your internal drive is fine if your disk is big enough. However, most folks soon start running low on internal drives at which time they move their images to an external drive. No matter which way you go, you MUST backup those images to a drive other than the one they live on normally AND THE BACKUP YOU CAN CHOOSE WHEN LIGHTROOM EXITS DOES NOT BACK UP THE IMAGES - JUST THE CATALOG.

It is equally important to assure that backups of the catalog are also stored on a disk drive other than the one containing the in-use catalog.

One advantage of having the catalog on the same external drive as the images is that you can just plug that drive into another computer which has LR installed (like a laptop) and everything is there (for the most part). If you do this and take the external drive with you when you travel, assure that you leave a current backup of that drive at home (and in some cloud storage someplace).

 

[deekay2308]

You did not mention where you do backup. All of this needs to be part of a complete plan.

This is. what I do (your mileage may vary):

• Single internal drive is SSD for max speed and reliability compared to a HDD or a hybrid fusion drive
• Internal SSD is for boot/MacOS/apps/caches (and little else)
• Large external RAID 0 pair of HDD connected via TB3 contains documents, pictures, music, videos...etc.. It also contains my LR catalog backups
• External SSD connected via TB3 contains my Lr catalog along with all the previews....etc.
• Large single drive connected via TB3 is owned by Time Machine and makes backup copies of the entire file system

If I had income based on my catalog and raw images, I would like also have an online backup such as Backblaze and maybe rotate backup drives off site to a bank vault. Each person has to decide how much they value their images and what expense they will incur to protect them.

You will to either let Lr Library move the folders to the new drive or, once they are moved you will have to have Lr find them again on the new drive. Suggest you do the move via LR......only have a full backup.

Thanks.

For backup I am using Time Machine, Carbon Copy Cloner and Backblaze (OCD). I think the CCC is redundant and probably not needed.

Questions:
1. You have 3 TB3 outputs?
2. Why do you have the Lr catalog on an external SSD rather than internal?
3. If your large external HDD was - hypothetically - an SSD drive, would you still use a RAID setup?

 

[LouieSherwin]

Hi @deekay2308 ,

I believe that the most important aspect of moving to an external drive is going be your external interface speed. If you have a later model Mac then it should have one or more Thunderbolt ports. I would highly recommend that you then use a Thunderbolt external drive or even perhaps take the next step and get a multiple drive enclosure. With a 4 disk enclosure you can your external drive and your backup drive all in one box with one power supply and cable with room for expansion.

While you can get can connect USB3 or USBC drives to a Thunderbold port you will be better off in the long run sticking with Thunderbolt. USB is point to point so you tie up the port with only one device. Whereas Thunderbolt is a daisy chain where you can run multiple high I/O devices of of one port.

If you only have USB2 or FireWire ports then you will likely notice slowdowns especially on high I/O operations like Import and Export. But a lot of people still use it this way with minimal problems. In this case then you definitely want to keep you catalog files on your high speed internal drive.

BTW I use TimeMachine, Crashplan and Carbon Copy Cloner regularly. I frequently clone my system drive to keep a bootable up to date backup. A couple of years ago my internal SSD just disappeared never to come back. By simply booting from my recent clone and then restoring from TM I was back up running in less than an hour with only about 30 minutes of lost work. In this case since the SSD failed completely there wasn't even a Recovery volume to do a TM restore. Always keep a bootable backup even if it is a month old.

-louie

 

[mcasan]

Questions:
1. You have 3 TB3 outputs?

Nope. Several of my physical drives sit in the same Akitio Thunderbolt 3 X enclosure. But I can add up to 7 more devices on that TB3 chain. The iMac has two TB3 ports. So you could chain a lot of Thunderbolt devices to an iMac.


2. Why do you have the Lr catalog on an external SSD rather than internal?

Because I kept all my 1:1 previews and I want the catalog and all the related files, such as the previews, on the same drive. On a 512 SSD there is not enough room for all the catalog files plus my apps.. Besides, I already had a Samsung EVO 1TB from a couple of year before.


3. If your large external HDD was - hypothetically - an SSD drive, would you still use a RAID setup?

If I had a 8TB SSD that only cost the same as two 4TB HHDs, you bet I would run the SSD. Know where I can find one?

 

[deekay2308]

.... I would highly recommend that you then use a Thunderbolt external drive or even perhaps take the next step and get a multiple drive enclosure. With a 4 disk enclosure you can your external drive and your backup drive all in one box with one power supply and cable with room for expansion.

Thanks Louie.
I will use TB3.
Q re the multiple drive enclosure: Can you provide an example of this as I am not too familiar with it.
What about getting a single large external drive and partitioning it? Would this not achieve the same result?

 

[deekay2308]

If I had a 8TB SSD that only cost the same as two 4TB HHDs, you bet I would run the SSD. Know where I can find one?

I don't, sadly..

 

[Umberto Cocca]

What about getting a single large external drive and partitioning it? Would this not achieve the same result?

No it is not the same.

You have several common failure causes and the fake feeling that you have multiple backups. If the controller fails, you loose it all. If the reading head fails, you loose it all. Power surge? All lost!

With several drives you reduce drastically the probability that a single event can damage them all.

Cloud backup or off-site backup are always recommended to protect your data against theft, fire, flood, earthquake, hurricane: events that makes multiple local copies of the same HDD completely useless.

Sent using Tapatalk

 

[deekay2308]

I am thinking of the following:

Lacie 2big Dock Thunderbolt RAID drive
set up as Raid 1

 

[Conrad Chavez]

Thanks Louie.
I will use TB3.
Q re the multiple drive enclosure: Can you provide an example of this as I am not too familiar with it.
What about getting a single large external drive and partitioning it? Would this not achieve the same result?

Here's an example of a Thunderbolt multiple drive enclosure:
https://eshop.macsales.com/item/OWC/TB3IVKIT0GB/For some, the point of a multiple-drive enclosure is to either provide some uptime protection in the form of a RAID. Partitioning won't get you that. Another big reason is expandability: If you buy a single 8TB hard drive, it's never going to be any bigger than 8TB. But if you buy a 4-bay enclosure and put four 2TB drives in it for 8TB total, and your photos and videos later fill that up, you can buy bigger drives, put them in, and now you have more capacity. You could turn that 4x2TB drive into a 4x4TB (16TB) drive or some capacity in between based on your budget. You can use the drives as normal storage, as some form of RAID, etc. That flexibility is why multi-bay enclosures are often preferred over a single big drive. And that is also why partitioning doesn't get you the same result. There's little need for partitioning these days (and Mac users should now be dividing drives using APFS volumes instead of partitions anyway.)

Either way, it still needs to be backed up.

I decided that USB 3.1 Gen 2 was fast enough for storing original raw files and videos, so I store mine in one of the following type of enclosure, which costs a lot less. I also decided RAID was not necessary (I back them up regularly) so that saved me some money too. About half the cost of the Thunderbolt equivalent.
https://eshop.macsales.com/item/OWC/MEQCTJBT00/A more expensive Thunderbolt solution might still be in my future if I decide that I want to move long-term storage or fast caches onto external SSDs. But for simple photo/video storage on hard drives, USB 3.1 Gen 2 is faster than any hard drive. I have not seen any performance problems related to the storage interface.

 

[LouieSherwin]

Thanks Louie.
I will use TB3.
Q re the multiple drive enclosure: Can you provide an example of this as I am not too familiar with it.
What about getting a single large external drive and partitioning it? Would this not achieve the same result?

Also I think that RAID with the possible exception of RAID 1 (mirroring) is an unnecessary complication that will not give you any real advantage as a photographer. Lightroom handles multiple drives quite easily so your archive can grow bigger than the initial storage device and you can simply add an additional device and keep going without bother or additional expense of building a RAID 5 or RAID 6 array. The only people that can rally make use of desktop RAID these days are video editors where the additional I/O speed really makes a difference when processing large video edits.

I would stay away from the boxes with builtin RAID controllers. You simply don't know their implementation. It may say RAID 1 but in the case that the controller fails can you take one of the drives out and put it in a plain SATA enclosure and continue running. In other words does each mirrored drive look like a plain MacOS HFS+ drive? Hard to tell unless you tear it apart and try, possibly invalidating your warrantee. If you want to use RAID 1 the safest bet is a TB3 enclosure and use MacOS built in RAID 1 that you can setup in the Disk Utilities.

Here is another reliable source for drives and enclosures: www.macgurus.com. I have been using one of their 2 bay eSATA hot swappable enclosures for a long time.

Here are their Thunderbolt3 and their USB3 offerings. The latter page has a brief writeup about the differences between USB3, USB3.1 and ThunderBolt.

-louie

 

[Conrad Chavez]

The latter page has a brief writeup about the differences between USB3, USB3.1 and ThunderBolt.

The second link has USB descriptions that are a little out of date, they don't explain the differences between the current USB 3.1 Gen 1 and Gen 2 labeling, and Thunderbolt is only discussed as far as the USB-C connector it uses. The following link has a more recent explanation of USB 3.1 Gen 1 and Gen 2:
https://www.tomshardware.com/news/difference-between-usb-3.1-gen1-usb-3.1-gen2,36941.html
Short version of everything:

Drive or protocol	Data rate in gigabits (not gigabytes) per second
NAS (Gigabit Ethernet)	1*
Hard drives	1 to 1.5
SSD (2.5" SATA)	4
USB 3.1 Gen 1 (formerly called USB 3.0)	5*
USB 3.1 Gen 2	10*
NAS (10GigE)	10*
Thunderbolt 2 (no longer current)	20*
SSD (NVMe) — used in Mac laptops, some Mac desktops, and better PCs	8-25+
Thunderbolt 3	40*

*Theoretical maximum. Real world throughput is somewhat lower and will vary.

So understand what kind of drives you want to put in, and get a box with an interface fast enough to handle them. And also understand that it's really easy to buy expensive enclosures that are way faster than the drives you can afford to put in them.

 

[clee01l]

I am thinking of the following:

Lacie 2big Dock Thunderbolt RAID drive
set up as Raid 1

As Umberto Cocca Pointed out there are single points of failure that make these options impractical. In the case of the RAID drive, it it the RAID Controller. If you compound this with a Proprietary RAID filesystem, Only a new identical RAID Drive controller is going to read your RAID Drives.
RAID is a business requirement rare needed by the individual. RAID is only practical as a 7X24 uptime requirement. RAID should also nit be confused as a backup. Delete or overwrite a RAID file and the mistake gets replicated everywhere.