Develop module Very slow performance on TIFF save

[robashcroftwales]

Lightroom Version Number

10

Operating System

1. Windows 10

I have been using LR Classic for about 6 months, together with Photoshop. I usually edit RAW files in LR then go to PS to do final edits. I convert mode of image to 8 bit file then save as TIFF. It has always been slow to save the image, but recently has become very slow, up to 20 secs or more. I tried altering several parameters in PS itself such as altering the memory allocated to PS (currently 80%) but it made no difference.

I finally went into the preferences for LR (see below) and saw that the 'compression' was set to 'ZIP'. I changed it to NONE yesterday and immediately it fixed the problem. It now files the images in about 2 secs. I'm just not sure why! What is the difference between ZIP and NONE? The file sizes of the resulting TIFFS from both options appear to be about the same size. Changing it to NONE also seems to have speedy up slightly the running of Nik plugins

OS Name Microsoft Windows 10 Home
Version 10.0.19041 Build 19041
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
System Name DESKTOP-BVETVUN
System Manufacturer HP
System Model HP Pavilion Desktop PC 570-p0xx
System Type x64-based PC
System SKU Z7C73EA#ABU
Processor AMD A10-9700 RADEON R7, 10 COMPUTE CORES 4C+6G, 3500 Mhz, 4 Core(s), 4 Logical Processor(s)
BIOS Version/Date AMI F.20, 30/10/2017
SMBIOS Version 3.0
Embedded Controller Version 255.255
BIOS Mode UEFI
BaseBoard Manufacturer HP
BaseBoard Product 82FE
BaseBoard Version 11
Platform Role Desktop
Secure Boot State On
PCR7 Configuration Binding Possible
Windows Directory C:\WINDOWS
System Directory C:\WINDOWS\system32
Boot Device \Device\HarddiskVolume3
Locale United Kingdom
Hardware Abstraction Layer Version = "10.0.19041.488"
Username DESKTOP-BVETVUN\Desktop PC
Time Zone GMT Standard Time
Installed Physical Memory (RAM) 16.0 GB
Total Physical Memory 15.9 GB
Available Physical Memory 8.18 GB
Total Virtual Memory 17.5 GB
Available Virtual Memory 6.62 GB
Page File Space 1.55 GB
Page File C:\pagefile.sys
Kernel DMA Protection Off
Virtualisation-based security Not enabled
Device Encryption Support Reasons for failed automatic device encryption: Hardware Security Test Interface failed and the device is not Modern Standby, Un-allowed DMA-capable bus/device(s) detected
Hyper-V - VM Monitor Mode Extensions Yes
Hyper-V - Second Level Address Translation Extensions Yes
Hyper-V - Virtualisation Enabled in Firmware No
Hyper-V - Data Execution Protection Yes

 

[Klaas]

It's tiff- specific, afaik. You can compress 8-bit-tiffs. But the compression of 16-bit tiffs is not defined in the tiff-specifications. It's not a Lightroom-problem, other software have this problem, too. The scanner-software Vuescan, vor instance. If you compress a 16-bit tiff, each software seems to create a zip-file, but in reality you always get a file size amount nearly of the original, sometimes even a bit more. At least that's my experience.

Klaas

 

[Conrad Chavez]

There are two pieces of background that help explain this.

• Adobe provides multiple compression options for TIFF including None, LZW, ZIP, and JPEG. (Nobody uses JPEG compression on TIFFs because if they want lossy, they just use JPEG. The other options are lossless.) I think the None option saves the fastest, but produces the largest files. LZW has been a popular choice for many years. ZIP is less used and less widely supported; ZIP can create smaller files than LZW at the cost of much longer save time.
• Lightroom Classic has offered ZIP as a TIFF compression option for many years. But somebody noticed that Lightroom Classic was not actually applying the ZIP compression, and apparently had not been doing it all this time. This bug finally got fixed in the October 2020 release of Lightroom Classic.

This is where it gets interesting. After the ZIP bug got fixed, people started complaining all over the place. “How come my TIFF ZIP files from Lightroom Classic suddenly take too long to save?” they asked. The answer: TIFF ZIP is now saving at the expected speed. Before, ZIP compression was not actually being applied, so TIFFs were saving uncompressed, which is relatively fast. But now that ZIP is being applied properly when selected, the longer time to save is not what everyone is used to…but it is now the expected speed of TIFF ZIP compression, unfortunately.

You can test this outside of Lightroom Classic, by comparing to TIFF ZIP files created using an application such as Photoshop. You will find that TIFF with None compression saves very fast, and ZIP saves very slowly. You can also test older TIFFs saved by Lightroom Classic as TIFF ZIP before the bug was fixed. What should happen is:

• TIFF files saved by Lightroom Classic with ZIP compression before the bug was fixed should be much larger than those saved from Lightroom Classic after the October 2020 bug fix release, since they weren’t really being ZIP-compressed.
• TIFF files saved by Lightroom Classic with ZIP compression before the bug was fixed should be much larger than saving the same image as TIFF ZIP from Photoshop.
• TIFF files saved by Lightroom Classic with ZIP compression after the bug was fixed should be similar in file size to saving the same image as TIFF ZIP from Photoshop.

 

[robashcroftwales]

There are two pieces of background that help explain this.

• Adobe provides multiple compression options for TIFF including None, LZW, ZIP, and JPEG. (Nobody uses JPEG compression on TIFFs because if they want lossy, they just use JPEG. The other options are lossless.) I think the None option saves the fastest, but produces the largest files. LZW has been a popular choice for many years. ZIP is less used and less widely supported; ZIP can create smaller files than LZW at the cost of much longer save time.
• Lightroom Classic has offered ZIP as a TIFF compression option for many years. But somebody noticed that Lightroom Classic was not actually applying the ZIP compression, and apparently had not been doing it all this time. This bug finally got fixed in the October 2020 release of Lightroom Classic.

This is where it gets interesting. After the ZIP bug got fixed, people started complaining all over the place. “How come my TIFF ZIP files from Lightroom Classic suddenly take too long to save?” they asked. The answer: TIFF ZIP is now saving at the expected speed. Before, ZIP compression was not actually being applied, so TIFFs were saving uncompressed, which is relatively fast. But now that ZIP is being applied properly when selected, the longer time to save is not what everyone is used to…but it is now the expected speed of TIFF ZIP compression, unfortunately.

You can test this outside of Lightroom Classic, by comparing to TIFF ZIP files created using an application such as Photoshop. You will find that TIFF with None compression saves very fast, and ZIP saves very slowly. You can also test older TIFFs saved by Lightroom Classic as TIFF ZIP before the bug was fixed. What should happen is:

• TIFF files saved by Lightroom Classic with ZIP compression before the bug was fixed should be much larger than those saved from Lightroom Classic after the October 2020 bug fix release, since they weren’t really being ZIP-compressed.
• TIFF files saved by Lightroom Classic with ZIP compression before the bug was fixed should be much larger than saving the same image as TIFF ZIP from Photoshop.
• TIFF files saved by Lightroom Classic with ZIP compression after the bug was fixed should be similar in file size to saving the same image as TIFF ZIP from Photoshop.

Conrad, that was a very good explanation, thank you. I have as a result just discovered something. If you look at my screen grab above the third option down is set to 16bit/component. I was always under the impression that it was best to edit files in 16 bit mode as the dynamic range etc is far better because of the better bit space. But if I set that parameter to 16 bit in the preferences it only allows me to use either NONE or ZIP options for compression. If I change to 8 bit mode I have the options for NONE, LZW, or ZIP. When I set up my LR I must have selected 16 bit and then set ZIP for compression as I thought NONE would result in large files. So it seems the only way you can use LZW is if you opt for 8 bit in the bit depth field - is that correct?

If I file using ZIP it takes about 20 secs to file. If I use LZW in 8 bit mode it's about 3 secs, and NONE in 16 bit mode initially then converted in PS to 8 bit takes about 1.5 secs to file.

 

[Conrad Chavez]

I was always under the impression that it was best to edit files in 16 bit mode as the dynamic range etc is far better because of the better bit space. But if I set that parameter to 16 bit in the preferences it only allows me to use either NONE or ZIP options for compression. If I change to 8 bit mode I have the options for NONE, LZW, or ZIP. When I set up my LR I must have selected 16 bit and then set ZIP for compression as I thought NONE would result in large files. So it seems the only way you can use LZW is if you opt for 8 bit in the bit depth field - is that correct?

I think it’s like what Klaas was talking about regarding how the results of compressing TIFFs aren’t always what we’d expect. I ran a test to make sure. At least on the one file I tried, the 8 bits per channel version behaved as you’d expect: The uncompressed was the biggest, the LZW was smaller, and the ZIP the smallest. But with the 16 bits per channel version, the uncompressed and ZIP were as expected, but the LZW version was larger than the uncompressed version! That’s what I thought I remembered, and that might be the reason you aren’t seeing LZW as a compression option for 16 bits per channel images: LZW isn’t effective with them.

 

[Conrad Chavez]

The scanner-software Vuescan, vor instance. If you compress a 16-bit tiff, each software seems to create a zip-file, but in reality you always get a file size amount nearly of the original, sometimes even a bit more. At least that's my experience.

I checked this one out a little more, since I’m doing a lot of film scanning with VueScan during the pandemic. I have VueScan set to save TIFF with the Compression option enabled. VueScan Help does not mention what kind of TIFF compression is used, so I opened one in MediaInfo which said it was LZW. When opened in Photoshop followed by a Save As TIFF, it also defaults to LZW so I think it’s safe to assume that the VueScan compression option for TIFF applies LZW compression.

I then resaved the same VueScan TIFF LZW file out of Photoshop. The Photoshop TIFF uncompressed is about 15% larger than the VueScan TIFF LZW, and the Photoshop TIFF ZIP is about 11% less than the VueScan TIFF LZW. All of that is more or less as expected, so it looks like when compression is selected for TIFF in VueScan, it is compressed with LZW, and the file size reduction is as expected.

 

[robashcroftwales]

I'm getting rather confused about all of this. I used PS for years. I always shot RAW and controlled files through Bridge, editing first in RAW editor then processing in PS. The files were processed in PS in 16 bit mode, and before I saved them as TIFFs I converted them to 8 bit mode using the IMAGE/MODE menu opt in PS. That route always prompted a save file panel, this one...

It then presented the TIFF options panel (below). From there you can specify the compression method (NON, LZW, ZIP, JPEG)

Since starting to use LR classic this year I have never seen the TIFF options panel. so have no choice at the time of saving what compression to use. When I do a save it just saves the image with no panels presented. I assume that this is bypassed by Adobe as the compression method is already specified in LR Preferences under external editing options.so they assume there is no need to see it. The problem is that you can only specify ZIP or NONE for 16 bit editing from LR to PS. There is no option for LZW. For 8 bit all options are available from LR. But, I want to start off with a 16 bit file for editing then save it as 8 bit (to get smaller file), and I want to also compress it, but using my workflow I can't do that as LR won't let me set LZW for 16 bit (even though I'm going to convert it to 8 bit before saving).

But there is a way around it. The save command in PS when you have come from LR doesn't show the TIFF options panel (so you can change to LZW) but the 'save as' command (CTRL/S) does present it, and allows you to change compression method to LZW, even though you specified NONE or ZIP in LR preferences.

Why is this a problem? Because if I save a file as 8 bit (which started as 16 bit) in ZIP compression it takes too long to file (up to 25 secs). But if I save as LZW (using the 'save as' command in PS) it is about 2-3 secs and still produces a smaller file than no compression. It's not as small as a ZIP but it's OK as far as I'm concerned.

I think this is a fault in LR. You should be able to specify LZW in the LR preferences for 16 bit mode, but you can't. I appreciate LZW might not do much for 16 bit files and that is probably why it is not there, but is for 8 bit mode. But if you convert, as I do, from 16 to 8 bit in PS then it ought to be available. OK, you can get around it by doing a 'save as' and changing the compression on TIFF options, but that is a bit of a delay and hassle.

 

[Paul_DS256]

Since starting to use LR classic this year I have never seen the TIFF options panel.

I don't normally have a need to export TIFF so decided to see what an EXPORT of a TIFF looks like on my LRC 9.4. The TIFF options appear when you select TIFF as the file type but has no where near the same options of what you posted.

This seems to be also reflected in the options for creating TIFF files to pass to external editors in PREFERENCES

Here is the TIFF METADATA of the export. There was a separate panel in the TIFF viewer with 'Byte Order: Intel"

SubFileType (1 Long): Zero
ImageWidth (1 Long): 1400
ImageLength (1 Long): 933
BitsPerSample (3 Short): 16, 16, 16
Compression (1 Short): Uncompressed
Photometric (1 Short): RGB
ImageDescription (37 ASCII): XXXXXXXX
Make (18 ASCII): NIKON CORPORATION
Model (12 ASCII): NIKON D5300
StripOffsets (1 Long): 17518
SamplesPerPixel (1 Short): 3
RowsPerStrip (1 Long): 933
StripByteCounts (1 Long): 7837200
XResolution (1 Rational): 115
YResolution (1 Rational): 115
PlanarConfig (1 Short): Contig
ResolutionUnit (1 Short): Inch
Software (48 ASCII): Adobe Photoshop Lightroom Classic 9.4 (Window...
DateTime (20 ASCII): 2020:11:15 08:45:26
Artist (11 ASCII): XXXXXXX
700 (12586 Byte): 60, 63, 120, 112, 97, 99, 107, 101, 116, 32,...
Copyright (28 ASCII): XXXXXX
RichTiffIPTC (68 Long): 5898524, 1193614083, 540, 470024194,...
Photoshop (42 Byte): 56, 66, 73, 77, 4, 10, 0, 0, 0, 0, 0, 1, 1,...
34665 (1 Long): 16580
ICCProfile (3144 Undefined):
34853 (1 Long): 1738

 

[robashcroftwales]

I don't normally have a need to export TIFF so decided to see what an EXPORT of a TIFF looks like on my LRC 9.4. The TIFF options appear when you select TIFF as the file type but has no where near the same options of what you posted.

View attachment 15522

This seems to be also reflected in the options for creating TIFF files to pass to external editors in PREFERENCES

View attachment 15523

Here is the TIFF METADATA of the export. There was a separate panel in the TIFF viewer with 'Byte Order: Intel"

Paul

You are doing something different - an EXPORT from LR. I'm doing a SAVE from Photoshop itself. They are different. Although I must confess to not knowing exactly what the difference is between the two.

 

[Paul_DS256]

I'm doing a SAVE from Photoshop itself.

Ok, but I was responding to your statement "Since starting to use LR classic this year I have never seen the TIFF options panel. "

 

[Conrad Chavez]

You are doing something different - an EXPORT from LR. I'm doing a SAVE from Photoshop itself. They are different. Although I must confess to not knowing exactly what the difference is between the two.

The difference should not be very much, as far as the file itself if concerned. The main difference is why we have to take different paths in each application.

Photoshop is a traditional application where you open an original document, change things, and save it. The original document is changed. Lightroom is a more modern application where the original is left untouched because edits are recorded as metadata in a database, so Lightroom has no File> Save command. If you want an edited TIFF of whatever you are editing, you must use the Export command. This part explains why you have to export a TIFF from Lightroom while you save it from Photoshop. The actual saving of the TIFF is probably about the same.

Lightroom Classic offers fewer TIFF options than Photoshop, consistent with a general philosophy of doing things more simply than Photoshop when it makes sense. When creating a TIFF, the Lightroom team decided it isn’t important to provide a choice for options such as Pixel Order or Byte Order, they just pick a setting silently. This is fine; you don’t see those options in most other applications today, in part because TIFF reading got a lot more standardized. For example, some Photoshop users have agonized over whether choosing a Byte Order of IBM PC or Macintosh might screw up a cross-platform workflow, but for many years any good TIFF reader on any platform has understood both.

The original post in this thread is about a variation of that workflow. You want a TIFF file out of Lightroom Classic, but not just to drop it in a folder somewhere. You want to send it from Lightroom to Photoshop to save a step or two, and TIFF is just the format option you chose for that.