• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • It's Lightroom update time again. Just some small tweaks as well as cameras/lenses and bugs! See this blog post for Lightroom Classic and this blog post for the Lightroom Cloud Ecosystem changes.

Import Benchmarking LR (in general). Performance of import?

Status
Not open for further replies.

joptimus

Member
Joined
Dec 30, 2015
Messages
41
Lightroom Experience
Beginner
Lightroom Version
Hi guys,

I gave LR Classic CC a try (currently I have LR 6) and since I was curious about performance improvements, I tried to benchmark the import and generation of standard (1440p) + smart previews.

Unfortunately, I cannot get consistent results either on Classic CC nor on LR 6.
What I did:
  1. Create new catalogue
  2. Import 30 RAW pictures and generate previews on import
  3. Take the time when the progress bar disappears
  4. Remove the photos, delete the preview files, reboot (to flush the system memory) and repeat
I wanted to get 3 results and take the average, but they vary just too much. First run about 70 seconds, second 45, third 55...that makes no sense.
I'm on a Macbook Air 2015 - maybe the cpu clock speed varies due to thermal throttling depending on the previous workload.

To conclude:
I cannot determine LR Classic CC performance myself reliably - are there benchmarks out there already?
 
Joined
Jul 2, 2015
Messages
11,997
Location
Netherlands
Lightroom Experience
Power User
I think 30 files is not enough to do a reliable benchmark. You should use several hundreds. Another thing to remember is that the biggest speed increase is the new 'Embedded & sidecar' option during import. If you select that, Lightroom will first use the built-in previews that the camera generated, rather than wait till it generated its own preview. Standard plus smart previews may be a bit faster too, but don't expect spectacular differences with that option.
 
Joined
Feb 1, 2010
Messages
13,540
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
Classic
I agree with Johan, 30 images really isn't enough. I routinely run similar benchmark tests, have been doing that since LR3, but over the years my testing has become more extensive. I now maintain a single "performance catalog" for each major version, each catalog containing the same 1000 x CR2 files from a Canon 5DIII. Those 1k images are sub-divided into two sets of 500, one set remains unedited while the other set gets an identical develop preset applied. The reason for the two sets is that tasks such as exporting and preview building will take longer on developed files, so I'll have an optimum "vanilla" speed, plus a "developed" speed....which can make for more interesting reading, especially if one gets faster and the other slower when the next version is tested.

I also test on two different platforms, OSX and Win10 (two separate systems, supposedly fairly equivalent performance capabilities), and run the whole gamut of preview building tests as well as exports. I have found more inconsistency so far on my Windows desktop, where I don't think the performance improvements have been fully realised yet, but I have been getting fairly consistent (and in some cases, impressive) results on my MBP.

Happy to share some of my results, but do understand they are NOT definitive....in order to ensure as much consistency as possible the tests are run in a refreshed system with nothing else running, which is not how I normally use Lightroom in normal usage. So all I have is an idea of the type of performance that Lightroom version xxx on my system is capable of delivering using the files that come out of my camera, which gives me someplace to look should performance start falling off the cliff, and a benchmark score for comparison with later versions. So while it's good enough to show what performance gains/losses might have been made with the latest version, the "this is on my system only" caveat needs to be understood.

So with that caveat in mind, let me know and I'll dig out my latest test results.
 

joptimus

Member
Joined
Dec 30, 2015
Messages
41
Lightroom Experience
Beginner
Lightroom Version
Thanks for your advice so far.

@Jim Wilde:
I would gladly see your results starting from LR5 to Classic (if you have it).
Honestly, I don't think my 2C/4T machine would show much of a difference between LR6 and Classic CC, but maybe I'll be upgrading next year and things may look different then...
 
Joined
Feb 1, 2010
Messages
13,540
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
Classic
OK, here's my data so far, but with the usual preamble that it's data collected using my specific environment over the years (though when I upgrade I go back and re-benchmark the earlier versions). In that context, it's actually pretty meaningless other than as an indicator as to where I'm seeing performance improvements or degradation. All I can say is that my own results seem to confirm that many improvements have been made as advertised, but it doesn't surprise me that not everyone is seeing the same improvements as @Karayuschij's linked post from the U2U above would confirm. Why that user is getting poor performance is impossible to say, my only observation from a brief glance is that I'd prefer to see larger sample sizes, but that still might not matter of course.

Anyway, OSX first:

OSX.png

All looks very positive, standard previews improvements especially so. The one "slower" category (exports of edited files) can be ignored as it's a tiny difference and I also know that no performance changes were made in that area.

Next, Win10. These are split into two sections, one where Hyper-Threading (HT) is enabled (probably the default for HT-capable systems), the second with HT disabled:

W10HT.png

With HT enabled, results look OK. Not as big an improvement as under OSX, but healthy enough for all that. Standard previews, for some reason, seem blisteringly fast, even faster that OSX.

But note the asterisk against the three tasks on "edited" files. What I was finding when using Win10, and more so with Classic than previous versions, was a definite slowdown the longer the task was running, e.g. the more images in the sample size, the larger the per image time would likely be. Normally, my testing routine would be reboot, open Lightroom, run task 1 for both sets of images consecutively, then reboot, start again with the next task. Using this normal method (which is how I do it under OSX as well) I was getting much longer second runs times per task than I expected, so I modified the approach to do a simple restart of Lightroom between Set1 and Set2, and the "*" indicates that the specific test was following a relaunch. This information has been passed to Adobe, so I'm hoping to see this area improved as the engineers continue to work on Classic performance.

Now, Win10 with HT disabled:

W10noHT.png

Still better than LR6, though it could hardly be worse. I'm not sure how much effort, if any, the engineers are putting into optimising on HT-disabled systems, but it's not really coming through. Some of that (maybe all of it) is likely explained by the "parallelisation" of these tasks now, thus the more logical cores the more concurrent previews can be built. That's a large part of the significant step forward under OSX (8 logical cores) and Win10 with HT (again 8 logical cores).

I've probably got some import timings documented somewhere, if I find them I'll post in an appropriate place.

Just a final word to reiterate that caution needs to be used when examining (or broadcasting) this data. It's not the "gospel according to Jim Wilde", more a (hopefully helpful) indicator that there have been performance changes incorporated into Classic, though it's a pity if not everyone sees them. But's that's been the nature of "Lightroom's performance problems" for years, some tear their hair out in frustration, others wonder what all the fuss is about. I doubt that will ever totally change, though it has been good to see some positive reports on the forum.
 
Status
Not open for further replies.
Top