Benchmark available for testing Lightroom Classic performance on Windows computers

[Roelof Moorlag]

Lightroom Version Number

Classic CC

Operating System

1. Windows 10

Today Puget systems has made available a free benchmark for testing Lightroom Classic performance on Windows computers:
https://www.pugetsystems.com/labs/articles/PugetBench-for-Lightroom-Classic-1571/

 

[Jimmsp]

For the heck of it, I ran the test again.
Fresh download and a fresh unzip. But this time I put the Puget files all on a 2nd SSD that I have dedicated to my usual LR catalogs and my PS scratch drive. The first SSD has the OS and software apps.
The Beta crashed once, but when it ran, it gave me the best result so far.

I was trying to see if I could tell if and where my GPU maxed out - but I missed it. The poor HDR results are still a puzzle.

 

[Roelof Moorlag]

So is anyone interested in sharing the beta results?

Yes, i am! But the test crashed on me.
However Puget published V0.85 beta so i wil try again one of these days.

 

[HobbyJohn]

I have run the benchmark quite a few times making slight tweaks to my system. I haven’t hd any issues getting it to run properly on my new machine, but didn’t have a single success on my old (2014ish) machine. When I’m back on my PC I’ll post my results and observations.
Somewhere it says that you need ~150gb+ free space on the drive or it won’t complete properly. Also shutting down any miscellaneous programs helps, and of course make sure you aren’t using the computer for other things while it is running.

 

[HobbyJohn]

Ok, so here's my current PC build. It is roughly based on Puget's recommendation for LR.

• Gigabyte X570 Aurus Ultra
• AMD Ryzen 9 3900x (stock cooler)
• Crucial Ballistix Sport ddr4-2666 (2x16gb)
• RTX 2060 6gb GPU
• Samsung 870 Evo Plus 500gb m.2 ssd
• Samsung 860 Evo 500gb SSD
• Seagate Barracuda 4tb hdd
• Benchmarks run while using one 27" monitor at 4k resolution

First, note that there will be some variance in performance with each run with the same specs, so one result being slightly higher or lower than the other is probably primarily due to those changes, and not necessarily the hardware/software change. Large differences are what we are looking for.

Overall score is the average of total Active and Passive task scores, times 10.

First I ran the benchmark with a 2gb GT 730 GPU while I waited for the RTX 2060 to be delivered. catalog and files were on the sata SSD. LR GPU Accel set to Auto.

Then I swapped out the GPU and ran again. LR GPU Accel set to Auto.

Then I moved the catalog and files to the m.2 ssd and ran again. LR GPU Accel set to Auto.

Then someone reminded me of the LR GPU Acceleration option. There are four options - OFF, Auto, ON (Display only), ON (Display and image processing)
OFF:

ON (Display Only)

ON (Display and Image Processing)

Then I added another (2) sticks of 16gb ram to see if that made any difference. GPU Accel set to Auto.

Thoughts:

• GPU does not make a significant difference from 2gb to 6gb (even with the faster speeds on the 6gb card). It is documented that LR has primarily been CPU heavy rather than GPU, but there have been some more recent changes to take advantage of GPU
• m.2 vs sata SSD does not yield significant change in results. As noted with using m.2 as a system drive, on paper it should be much faster but in practice it does not "seem" much faster.
• 32gb ram is plenty, and is not a bottleneck
• AUTO setting for GPU acceleration does a good job

At any rate, my new machine is much nicer to work with in LR than my old system. I am very pleased with the build. I reused my case and power supply, but bought everything else new on Amazon. Total cost of new parts was around $1500.

 

[PhilBurton]

Ok, so here's my current PC build. It is roughly based on Puget's recommendation for LR.

• Gigabyte X570 Aurus Ultra
• AMD Ryzen 9 3900x (stock cooler)
• Crucial Ballistix Sport ddr4-2666 (2x16gb)
• RTX 2060 6gb GPU
• Samsung 870 Evo Plus 500gb m.2 ssd
• Samsung 860 Evo 500gb SSD
• Seagate Barracuda 4tb hdd
• Benchmarks run while using one 27" monitor at 4k resolution

First, note that there will be some variance in performance with each run with the same specs, so one result being slightly higher or lower than the other is probably primarily due to those changes, and not necessarily the hardware/software change. Large differences are what we are looking for.

Overall score is the average of total Active and Passive task scores, times 10.

First I ran the benchmark with a 2gb GT 730 GPU while I waited for the RTX 2060 to be delivered. catalog and files were on the sata SSD. LR GPU Accel set to Auto.
View attachment 14009

Then I swapped out the GPU and ran again. LR GPU Accel set to Auto.
View attachment 14010

Then I moved the catalog and files to the m.2 ssd and ran again. LR GPU Accel set to Auto.
View attachment 14011

Then someone reminded me of the LR GPU Acceleration option. There are four options - OFF, Auto, ON (Display only), ON (Display and image processing)
OFF:
View attachment 14012

ON (Display Only)
View attachment 14013

ON (Display and Image Processing)
View attachment 14014

Then I added another (2) sticks of 16gb ram to see if that made any difference. GPU Accel set to Auto.
View attachment 14015

Thoughts:

• GPU does not make a significant difference from 2gb to 6gb (even with the faster speeds on the 6gb card). It is documented that LR has primarily been CPU heavy rather than GPU, but there have been some more recent changes to take advantage of GPU
• m.2 vs sata SSD does not yield significant change in results. As noted with using m.2 as a system drive, on paper it should be much faster but in practice it does not "seem" much faster.
• 32gb ram is plenty, and is not a bottleneck
• AUTO setting for GPU acceleration does a good job

At any rate, my new machine is much nicer to work with in LR than my old system. I am very pleased with the build. I reused my case and power supply, but bought everything else new on Amazon. Total cost of new parts was around $1500.

IF I understand your post, it would seem that replacing the GTX 730 with the RTX 2060 lowered the total score? Did I misread something?

What did you do with the old parts?

 

[HobbyJohn]

IF I understand your post, it would seem that replacing the GTX 730 with the RTX 2060 lowered the total score? Did I misread something?

What did you do with the old parts?

Since it's such a small change, I think that is more from a normal variance from one test to another, and not so much that the new card is "worse". The first test with the new GPU was 28pts less, but then the second test with the new card was 18pts higher, and the test with GPU acceleration partially on was 58pts higher. I'd say that upgrading from the GT 730 to the RTX 2060 did not improve the system, and that the GPU is not a major bottleneck in LR .

Old parts are in my PC parts bin. Reusing ssd and hdd as backup drives. Build was - M5A99FX Pro R2, FX6350 cpu with Hyper 212 cooler, GT 730 2gb GPU, 16gb ddr3, Samsung 850 Evo 250gb SSD, Seagate 2tb hdd.

 

[Paul_DS256]

lowered the total score

You have to decide if the score means anything to begin with

 

[HobbyJohn]

You have to decide if the score means anything to begin with

Yes, and the score is really just a comparison of your system to a “control” system that Puget built. And as with all benchmark tests, they may be a good representation of what you will experience, or they may be way off. Everyone has a different workflow and uses different functions.

I ran these purely out of curiosity, and to see what effect a few different variables that I was already changing might have. Obviously far more benchmark runs would be needed for it to even begin to be valuable data.

For me, it has reassured me that I don’t have a bottleneck in LR that would be cheap and easy to fix. In practice, my new machine is far more efficient and enjoyable to use for all of the software I use for business and for pleasure. Photography is only a hobby for me so the LR performance was not top priority, but happened to align with my other needs.

 

[Paul_DS256]

For me, it has reassured me that I don’t have a bottleneck in LR

Sorry, I don't read that bench mark as being able to show you don't have a bottleneck as it has limited tests and, as you indicate, is not related to your own workflow.

If you want to find bottlenecks, then record your resource consumption over several workflow sessions and see where demand exceeded capacity.

I worked through numerous benchmarks in the past for other software areas. The only ones that provided some insight were those developed by non-hardware organizations and preferably a consortium of experts who debate the individual benchmarks

 

[Linwood Ferguson]

I'd say that upgrading from the GT 730 to the RTX 2060 did not improve the system, and that the GPU is not a major bottleneck in LR .

Did you spend some time with both using it manually and does your LR experience mirror the benchmark?

 

[HobbyJohn]

Sorry, I don't read that bench mark as being able to show you don't have a bottleneck as it has limited tests and, as you indicate, is not related to your own workflow.

If you want to find bottlenecks, then record your resource consumption over several workflow sessions and see where demand exceeded capacity.

I worked through numerous benchmarks in the past for other software areas. The only ones that provided some insight were those developed by non-hardware organizations and preferably a consortium of experts who debate the individual benchmarks

I see where you are coming from. And yes, from the overall score it does not relate to my own workflow. However, I can also look at the processing times for the individual tasks that ARE part of my workflow and can see how those are affected. I'm not at all saying this is a definitive test and that everyone should use it to judge their systems. I ran these for my own personal interest and posted them so that other people can see the results and use them however they please. I have recorded my resource monitor activity (or using other similar tools) and have compared that to these results, but those are of even less use to the public than the benchmark results.

Did you spend some time with both using it manually and does your LR experience mirror the benchmark?

I did not use LR for any meaningful amount of time between the different runs, since I wanted to get it fully up and running quickly.


I'm not claiming to be, nor do I want to become, an expert on this topic. I've seen many people run the benchmark on their system and post the results (here and on other websites and forums), but have not seen many cases where people make incremental changes to a system and run the benchmark at each stage.

 

[Linwood Ferguson]

I'm not claiming to be, nor do I want to become, an expert on this topic. I've seen many people run the benchmark on their system and post the results (here and on other websites and forums), but have not seen many cases where people make incremental changes to a system and run the benchmark at each stage.

My question really related less to the benchmark in general, than just to the question whether you as a human (vs the benchmark) could tell the difference in the moderate vs higher end GPU.

 

[HobbyJohn]

My question really related less to the benchmark in general, than just to the question whether you as a human (vs the benchmark) could tell the difference in the moderate vs higher end GPU.

Ah, sorry. In the little time I used it between the GPU changes, I did not experience any noticeable difference in performance. The work I did in LR with the old GPU was limited to importing a couple dozen images, renaming, and keywording, so I don't think that counts as a good point of reference. I also did not notice any significant difference between m.2 and sata ssds, nor did I notice any difference going from 32 to 64gb ram. I did the most work in LR around the time of the ram change so I can say that for my workflow, that made zero noticeable change to performance. So yes, I'd say the benchmark results suggesting little difference (especially when focused on the individual tasks I perform in my workflow) reflects my experience in using LR.

 

[Linwood Ferguson]

Ah, sorry. In the little time I used it between the GPU changes, I did not experience any noticeable difference in performance. The work I did in LR with the old GPU was limited to importing a couple dozen images, renaming, and keywording, so I don't think that counts as a good point of reference. I also did not notice any significant difference between m.2 and sata ssds, nor did I notice any difference going from 32 to 64gb ram. I did the most work in LR around the time of the ram change so I can say that for my workflow, that made zero noticeable change to performance. So yes, I'd say the benchmark results suggesting little difference (especially when focused on the individual tasks I perform in my workflow) reflects my experience in using LR.

That's fairly consistent with some older testing I did, LR seems to rarely get disk limited (there are exceptions of course), but my only testing for GPU was from a very slow one to a moderately slow one, and Adobe has done a lot since in terms of GPU usage. Was curious what I human would see today, as a new GPU is a pretty cheap upgrade.

I did find that while memory did not help a lot over say 16GB (with some exceptions in big merges), if you over-clock memory you get a mild boost; that's fairly consistent with the idea most things remain CPU limited.

 

[Roelof Moorlag]

But the test crashed on me.
However Puget published V0.85 beta so i wil try again one of these days.

Version 0.85 did crash also but i tried V0.9 today with succes. Here are the results of my 4 year old system: