• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Catalogs Thunderbolt 3 Justified for Catalog on EXHD?

Status
Not open for further replies.

flyfifer

Graham Goodman
Premium Cloud Member
Joined
Jul 26, 2020
Messages
9
Location
Gloucester, UK
Lightroom Experience
Intermediate
Lightroom Version
Classic
Lightroom Version Number
9.3
Operating System
  1. macOS 10.15 Catalina
In the early stages of looking to upgrade to a new Mac Mini. As part of my cost analysis, I'm looking into the variety of storage options I have in order to try and avoid giving Apple more money than necessary...

If I was to put the Lightroom catalog on an external hard drive, is there any real world benefit in going Thunderbolt 3 over USB-C? If I look at the Samsung X5 against the T5, raw tests suggest a five-fold performance improvement. But this performance improvement comes with a price tag of an extra £220 on the 1TB versions at which point paying Apple £200 to double the internal SSD becomes a viable option. It's not clear to me if Lightroom is performant enough to make use of T3's extra bandwidth.

To be clear, my images are stored on an 18TB NAS. The Mac Mini storage would only be used to store the catalog file (115k images), the previews file, the sync files and any images that are being worked on (my workflow is to import to local drive, work on them and then move them to the NAS once I have finished processing them).
 
TB3 has a speed capability of 40 GBPS while USB3.1 Type C is limited to 10GBPS or about the same sped as the original TB.
A TB3 EHD is about as fast as a buss mounted HD.

I think there are incompatibilities between USB Type-C and TB3 even though they use the same connector. I don’t know what they are, but I have an EHD that has both USB3 TypeC and TB3 Ports


Sent from my iPad using Tapatalk
 
If I was to put the Lightroom catalog on an external hard drive, is there any real world benefit in going Thunderbolt 3 over USB-C? If I look at the Samsung X5 against the T5, raw tests suggest a five-fold performance improvement. But this performance improvement comes with a price tag of an extra £220 on the 1TB versions at which point paying Apple £200 to double the internal SSD becomes a viable option. It's not clear to me if Lightroom is performant enough to make use of T3's extra bandwidth.
There is an informal test you can do with the equipment you already have, especially if you have different kinds of external storage. Open Activity Monitor on your Mac and click the Disk tab. Watch the Data read/sec and Data write/sec fields near the bottom right of the window as you do various things, for example as you try working with copies of your catalog/previews on different internal and external storage. Maybe also choose View > Update Frequency and take it down to 1 second. This should give you an idea of how fast certain activities actually move data.

I think it's important to do that because real world data transfer rates usually don’t come close to the theoretical maximums. I was testing “walking” from image to image quickly in Loupe and Develop, and I am finding it hard to get a R/W speed of over 100MB/sec for an extended period of time. To try and push it harder, I switched to a collection that was all panoramas (for more megapixels per image to transfer), and although I got it to spike above 170MB/sec once or twice, we have to remember that Activity Monitor is reporting storage I/O for the entire system, so something else could have been involved.

To put those numbers into perspective, 100MB/sec is roughly what you get out of a good hard drive, and is about the limit of USB 2.0 (800Mb/sec). 170MB/sec (1.3Gb/sec) is what you get out of a fast hard drive, well within USB 3.0-3.2 Gen 1 (5Gb/sec). However, some of those tests were run with a catalog and set of images that were on my MacBook Pro internal storage that benchmarks at over 2400MB/sec. This led me to keep my catalog on my Mac and my images on external hard drives in an enclosure connected via USB 3.1-3.2 Gen 2 (10Gb/sec) just to be on the safe side. All much cheaper than Thunderbolt 3 externals. Of course, everyone should run their own tests with their own images and catalog, because my cameras are in the low to average megapixel range.

The point is: Many of us already know that the maximum throughput you see is always significantly less than the theoretical maximum, due to protocol overhead and various real world factors. But on top of that, the overhead of what each application does reduces that even further. Even running backups and doing direct copies and data transfers, I rarely see actual data transfer rates coming anywhere close to what Thunderbolt 3 shows on pure benchmarks. It seems like 95% of what photographers do can be covered by midrange (1000MB/sec) NVMe SSDs over USB 3.2 Gen 2 (10Gb/sec), and if on a budget it seems like you would hardly tell the difference between that and a low- end SATA SSD (500MB/sec) over USB 3.2 Gen 1 (5Gb/sec).

It would be interesting to see if others get different results.

(Also, for clarity, how uppercase and lowercase is used is important:
GB & MB = gigabytes & megabytes
Gb & Mb = gigabits& megabits
with 8 bits in a byte)
 
I think it's important to do that because real world data transfer rates usually don’t come close to the theoretical maximums. I was testing “walking” from image to image quickly in Loupe and Develop, and I am finding it hard to get a R/W speed of over 100MB/sec for an extended period of time. To try and push it harder, I switched to a collection that was all panoramas (for more megapixels per image to transfer)
Keep in mind that Lightroom will only use the original image file for export and printing. All other times it will us the much smaller preview files .
 
In most uses, I think LR is constrained by the CPU, not disk bandwidth.

For example, my Macbook Pro builds 1:1 previews at nearly the same speed for raws residing on an external portable USB-3 drive as on the internal SSD. Similarly, it exports raws to JPEGs at about at 1.25 photos/sec, or about 7 MB/sec.

In Develop, consider that a raw file is typically about 25 - 40 MB, so it takes less than half a second for LR to read its bytes even from an external portable drive. And in Library, consider that the pyramid of standard-sized previews for an image are less than 10 MB each (often much smaller), so flipping through Loupe quickly won't stress an external drive. When importing from a memory card, you're constrained as much by the read bandwidth from the card as the write bandwidth of the disk.

Writing large numbers of .xmp sidecars is dependent on the seek times of the disks, not the bandwidth, and USB vs. Thunderbolt won't affect that much.

The SQLite catalog gets mostly cached in memory soon enough, so for reading the catalog, it doesn't really matter which disk it resides on. It might make a modest difference in the rare cases where you update the metadata for thousands of photos all at once.

Backups can definitely benefit from a faster disk, but I don't think most people care much about the speed of backups occurring in the background.
 
I have installed a Thunderbolt 3 drive as an external drive on a laptop. I tested performance and found that the speed came close to the disk i/o specs.

But....

In normal working conditions, I often found a delay between say, selecting a file in any app and the file becoming available. I put the delay down to the latency and design of the drive assembly, as it probably moves to a low energy status when not in use and needs to be ‘woken up’.

So, I am finding for sustained use, such as backups, I am getting close to the advertised disk i/o performance, but for ad-hoc activities I am finding Thunderbolt 3 external drive to be slower than an internal spinning disk.

The key point is there are a lot of factors involved which may mean actual real world performance will be a lot less than the advertised 40gbps maximum.
 
Thanks everyone for the thoughtful and detailed replies.

I have experimented on my iMac with an LaCie Rugged SSD USB 3 disk that I have. The drive is about five years old, so it is definitely not the most performant version that I could get. I'm also not chasing numbers but focusing on a real-world feel to see if the user experience would be acceptable to me.

And I've been impressed at the speed. I suspect that I've been overly nervous about this because of the old and out-dated advice of never keeping your catalogues on an external drive. Nothing felt overly slow to me. Real-time rendering of changes in the Develop module, walking through the images, keyword searches across my 115k images, all were pretty much lag-free. 1:1 previews may be a tad slower but not unacceptably slow given it is something that I don't often do.

So, it's decided. If I go down the route of buying a new Mac Mini, I'm not upgrading the internal SSD and putting my Lightroom catalog on an external USB 3 SSD.

Thanks everyone.
 
Thanks everyone for the thoughtful and detailed replies...

So, it's decided. If I go down the route of buying a new Mac Mini, I'm not upgrading the internal SSD and putting my Lightroom catalog on an external USB 3 SSD.

Thanks everyone.
Before you Jump Some things to consider. USB devices can be connected (with the proper adapter) to TB3 ports.
Don’t lock yourself into the limited technology when future technology might make your computer obsolete before its time.
USB can never connect more than one peripheral in a “chain” to your host computer —imagine using one computer, connecting an external USB hard drive to it, then trying to connect another USB hard drive to that one, and so on. USB can’t do that (unless you use a hub), but Thunderbolt can, with up to six devices. USB is also not bi-directional, which means that you can't transmit and receive data at the same time, while Thunderbolt can. Also, because Thunderbolt uses a PCIe bus, you can even add external graphics cards to Thunderbolt-equipped computers, something you can’t do over a USB bus. New USB Type-C connectors are bi-directional, but that function will only be available with Thunderbolt host ports.


Sent from my iPad using Tapatalk
 
Just to clarify. I store my Catalog on a very fast internal Nvme Ssd drive.

I wanted an external fast drive to store a substantial volume of images and to give me reasonable spare capacity for the next 2 years. It is not possible for me to store these images on external Ssd's , both from a logistical and financial point of view. I am happy I have opted for a Thunderbolt 3 enclosure, which will allow me store up to 24 Terabyte of data. The enclosure operates as fast as the disk drive used internally and substantially faster than my NAS (now used for backups and Music only). I was disappointed to discover that latency for ad hoc disk reads and writes on the external Thunderbolt 3 device, but can live with that.

If you are using a Thunderbird 3 connection to your external SSD then you will have a good option for your Catalog.
 
I agree with Cletus re TB3 ports. My external Thunderbolt device has a second TB3 port, which is extremely useful for daisy chaining other peripherals, such as a second monitor, docking stations, etc.
 
I agree with Cletus re TB3 ports. My external Thunderbolt device has a second TB3 port, which is extremely useful for daisy chaining other peripherals, such as a second monitor, docking stations, etc.

I have two TB3 ports on my iMac One has 4 TB3 EHDs and one USB3 EHD attached. The other has a DisplayPort Ext monitor attached. I have three USB3 ports one with a 4 port Hub. I uses these for things like my i1DisplayPro, recharging my keyboard and trackpad and attaching USB-C card readers and USB3 thumb drives. The USB3 EHD on the TB3 is and must be the last item in the chain.
USB3 hubs split the bandwidth where as the last item on the TB3 chain is as fast as the first.


Sent from my iPad using Tapatalk
 
Before you Jump Some things to consider. USB devices can be connected (with the proper adapter) to TB3 ports.
Don’t lock yourself into the limited technology when future technology might make your computer obsolete before its time.
USB can never connect more than one peripheral in a “chain” to your host computer —imagine using one computer, connecting an external USB hard drive to it, then trying to connect another USB hard drive to that one, and so on. USB can’t do that (unless you use a hub), but Thunderbolt can, with up to six devices. USB is also not bi-directional, which means that you can't transmit and receive data at the same time, while Thunderbolt can. Also, because Thunderbolt uses a PCIe bus, you can even add external graphics cards to Thunderbolt-equipped computers, something you can’t do over a USB bus. New USB Type-C connectors are bi-directional, but that function will only be available with Thunderbolt host ports.

To be clear, I'm only taking about the EXHD that I buy and not the connectivity options of the Mac Mini. The 2020 model of the Mac Mini has four TB3 and two USB3 ports, so I'm happy that I'm not forcing myself down a route of computer obsolescence. Only the EXHD would become obsolete (or, more likely, repurposed) should it no longer meet my requirements.

I'm happy going down the USB3 route now that I've verified USB3 is performant enough for me. The benefits of TB3 are not applicable for me (I'm not peripheral heavy so don't need to daisy chain, for example). Ultimately, the cost of a new USB3 SSD is less than the difference in cost between USB3 and TB3. Therefore, I don't believe that it financially makes sense to go TB3 to protect me from some future requirement that I may never have (and suspect never will). I think it makes more sense to go USB3 and then go TB3 at a later date if I something unexpected changes with my requirements.
 
Status
Not open for further replies.
Back
Top