• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • It's Lightroom update time again. Just some small tweaks as well as cameras/lenses and bugs! See this blog post for Lightroom Classic and this blog post for the Lightroom Cloud Ecosystem changes.

Import RAW, Tiff, DNG, Lightroom and Photolab

Status
Not open for further replies.

walkytalky1000

New Member
Joined
Oct 20, 2014
Messages
8
Location
Northumberland, UK
Lightroom Experience
Intermediate
Lightroom Version
Lightroom Classic 9.0
Operating System
  1. Windows 10
This post reveals my ignorance about a techie side of photography that may well be clear to many.

I understand that LR (and PS) contain a camera raw editor and for years I have been dutifully converting my Canon .CR2s and Panasonic .RW2s into DNGs while importing them into LR.
Q1. Have they been edited?
I have heard good things about the PhotoLabs RAW editor and I like to use the Nik plug-ins. Before I spend any more money ...
Q2. if I want to utilise the PhotoLab processor is it too late for my old photos which are now DNGs?
I note that PL can return the files to LR as DNG or TIFF files - so that means ... er ... what exactly?

Help from the enlightened please.
 
Joined
Jan 18, 2009
Messages
2,502
Location
Fort Myers, FL
Lightroom Experience
Advanced
Lightroom Version
Classic
There are subtle aspects to your question I'm going to try to be careful about, terminology is a bit tough here.

When you imported a CR2 into lightroom and, during the import converted to DNG, that is primarily just a change in file structure, and not a change in the raw data. First caveat: a digital image is more than the actual counted photons that produce it, it is also a ton of metadata that describes how it was taken, including information about that capture's structure. During the DNG conversion that data is changed, and some may be lost. In theory that does not change the actual image data. For lightroom the theory appears true, but if you later tried using a different editor you might not get the same results, but (I believe) because of the meta data, not because of the image data.

Now as it comes into Lightroom, the image produces a preview. That process converts the raw data to a typical color image; it is sometimes called a de-mosaic process. It is fair to call this editing, even if automatic. This must happen to view the image, every raw editor will do it, they all do it differently. This is a non-destructive process at this point however -- if you take the same DNG and go to (say) CaptureOne, then it will ignore the preview, and start with the same raw data (not yet edited) and do it's proprietary conversion to color, which may result in a slightly different look than LR. So long as it can fully process the DNG then the fact you imported it to Lightroom, or even converted to DNG, is moot.

Now if you took that DNG and tried to go to (rumor has it) DxO's editor, it might not work. NOT because it has been edited (in the sense the image has changed) but because the structure of the DNG file is not understood properly by that program. (Note I use this as an example but do not know if it is true, I've simply heard it).

Now if you edited the DNG in Lightroom and exported a TIFF instead, and replaced your DNG with the TIFF -- then yes, absolutely, that's an edited photo and a lot of changes have been "baked in".

So provided PhotoLab can read the DNG's, you can do what you have suggested without worrying that LR has changed them. PROVIDED (and this is important) that PhotoLab is taking the DNG. To complicate this, the way that some plugin editors exchange data with Lightroom varies, some will pass the original raw image, but some will pass a TIFF file invisibly and internally. In that case, the editing done in lightroom may affect what Photolab sees. You need to check how that works (often you have a choice).

Confused?

Just to confuse things a bit further, there's another aspect: During the raw -> DNG conversion you have the option of preserving the raw data. In that case, the raw image as originally taken by the camera, un-converted, is inside the DNG. That preserves your option to essentially un-do the DNG conversion, but only if you selected that up front.

Sorry for a long winded answer. Hope it helped more than hurt. :)

Linwood
 

walkytalky1000

New Member
Joined
Oct 20, 2014
Messages
8
Location
Northumberland, UK
Lightroom Experience
Intermediate
Thanks for your careful reply.
Having dug a little deeper into the details of RAW, TIFF and DNG files I am better informed but more confused.
Never mind. In a more practical vein, having experimented with conversions, exporting and importing, I discover that when viewed in LR to my eyes:
- there is no visible differences between the untouched camera raw file and the LR generated DNG
- Photolab can accept either of these formats
- the returned DNG is visibly different from the original
- a returned TIFF file is different again, perhaps "in the middle"
Not that it makes much difference to me but which of these files is entitled to be called a raw file?
 
Joined
Jan 18, 2009
Messages
2,502
Location
Fort Myers, FL
Lightroom Experience
Advanced
Lightroom Version
Classic
- there is no visible differences between the untouched camera raw file and the LR generated DNG
To be clear, the raw file, and a not-yet-demosaic'd DNG (which is what LR produces) cannot be viewed without conversion. The conversion is internal and automatic using Adobe Camera Raw (built into Lightroom), but it does occur. So that there are no visible difference means both ran through the same raw conversion inside LR before you saw them.

If you take both into Photolab without LR involved (other than to produce the DNG), Photolab should show both identically also to each other, but slightly differently to LR.
- the returned DNG is visibly different from the original
- a returned TIFF file is different again, perhaps "in the middle"
I'm not sure what "returned DNG" vs "returned TIFF" means here. If you mean you are calling Photolab from within Lightroom, and have a choice how Photolab returns the result, and that return makes a difference, it is odd if that is true. Returning an edited photo should be the same regardless of whether it is returned as a DNG or TIFF (and mostly the same as a JPG). At that point the DNG and the TIFF are just containers to hold the data, neither is raw. If they look different afterwards, then the two are going through some different type of either editing or conversion either before leaving Photolab, or in returning to LR. It is possible that LR is importing them differently on return and applying some preset or edit to the result.

It is important as you use two different editors to keep track of which one is applying change (and what type of change), so you do not accidentally have both altering your image.

Note, just to keep this even more confusing, going from one editor to another and back can also have issues with color space. Color space is not directly related to raw conversion, and color space if properly handled has no significant effect on what you see -- but if color space gets mixed up between products you can get anywhere from subtle to dramatic changes in appearance; usually bad ones. Both products likely have controls over color space (on LR they are in the transfer process, either in the plugin setup or under Preferences, External Editing.

While the ability to edit in a "plugin" editor in Lightroom is really cool, it does require both that the product work smoothly with lightroom, and that you really understand the transfer process back and forth to ensure you are not compounding edits in some fashion.

Not that it makes much difference to me but which of these files is entitled to be called a raw file?
Generally speaking a raw file is one that has original sensor data, before the de-mosaic process. Properly speaking this is not a "color" file; it has red/green/blue/green sensors (yes, green twice usually), but these are real color sensor readings, and are linear (200 photos is twice the value of 100 photos).

A color image (commonly speaking) is one that has been changed to pixels, and each pixel has a separate red, green and blue value for that same dot, that yields a blended color for that dot. Each of these pixels is calculated (and not in a linear fashion, but more of a log process) from the sensor readings that surrounded it; each sensor contributes to many pixels. Exactly how this calculation occurs is the "raw conversion algorithm" and is quite different between different manufacturers, and what yields subtly different colors when you do the conversion.

Once converted there is no going back; everything afterwards is editing. People often then say that certain aspects are "baked in".

Lightroom by default does not raw-convert-and-save (even in the import conversion to DNG, that's just format not raw conversion). So all the time, every time you look at an image in LR, it is re-doing the raw conversion, redoing the de-mosaic process. This is partly why it is called a "non-destructive" editor.

When lightroom exports to a TIFF, it is baking in the de-mosaic process; the resulting color image is no longer raw.

Some programs like photoshop, cannot edit raw at all. It must be converted (de-mosaiced) before it can edit. Often this is invisible; photoshop uses the Adobe Raw Converter from Lightroom if you ask it to edit a raw image (and you may or may not actually see that it does this depending on how it is used).

So you could (for example) convert in LR, and edit in Photolab. You could convert in DxO and edit in Lightroom. There are dozens of converters, and dozens of editors. Most people convert and edit in the same tool, but not all.

Just to make things more complicated, the conversion process and the editing process are blended together in many tools, so when you convert in Lightroom some of the "edits" are actually applied during the conversion, not separately -- white balance is a good example. Doing this provides more "leverage" (for want of a better term).

Why does this matter? Consider using Photoshop. If you adjust white balance in lightroom, then send an image to Photoshop to edit, you had a lot more ability to edit the white balance. If on the other hand you leave white balance alone, and then send the image to photoshop (which does a raw conversion on the way), you can change white balance in photoshop but your ability to do so is considerably limited. Small changes work fine, but large ones work much better if done prior to raw conversion.

Sorry... I'm getting long winded again. One last: DNG and TIFF are both really containers, meaning they hold data. They can both hold raw data and both hold converted (demosaic'd) data. NORMALLY though, by the nature of how workflow is designed in these tools, DNG's hold raw data, and TIFF's converted. But that's more like finding milk in a milk carton and water in a water bottle -- it's convention, but nothing keeps you from swapping contents. There are a lot of processes (LR merges, enhance details for example) that return edited non-raw data in a DNG. And there are a few programs out there that convert raw data to TIFF containers for further processing, but they are much more rare and specialized.

And just to confuse things, there are a lot of people who will say that once repackaged in a DNG, raw data is no longer really raw. I get where they come from, but technically that's not correct, any more than putting milk in a water bottle makes it no longer milk. Indeed, there's another similarity -- if you push that analogy a bit further, the labels on the milk carton describe the milk; moving to a different bottle you may lose some description of it, but the real milk liquid is the same. Same with raw data -- converting to a raw DNG you might lose some descriptive data, but it's still raw.

Wow... Victoria is going to start charging me for writing so many words. o_O Sorry... hope that helps...
 
Joined
Jan 18, 2009
Messages
2,502
Location
Fort Myers, FL
Lightroom Experience
Advanced
Lightroom Version
Classic
I'm not very enlightened, but

The dng returned from PL is not a RAW file. It has been "demosaiced" and is referred to as a linear dng. However, no information has been lost.
See, this is where things get complicated. A linear DNG is kind of like being half baked. :cautious:

Some conversion was done with an algorithm that may or may not be the same as other demosaic algorithms, so some aspects are baked in. Other future edits, like white balance, are preserved.

I'm not sure if it is fair to say "no information has been lost" though; is it possible to rebuild the original raw file? If not, then information is lost. Whether it's important information is a different question.
 
Joined
May 9, 2015
Messages
1,066
Location
Palo Alto, CA
Lightroom Experience
Power User
Lightroom Version
Classic
As mentioned this is a complicated topic, fraught with impressive and imprecise terminology.

A true raw file contains "unrendered" data also called "unprocessed" data. In other words the information about the image are not actually pixels (dots of color). What they actually are is less clear but understanding how a sensor works may shed some light (no pun intended). First of all an individual photo-receptor on the sensor is color blind. It can only measure how much energy photons provide, not what wave length (color) those photons have. In other words it is counting photons which gives brightness, but cannot detect their color. This works great for Monochrome images but in order to have a color photo they had to do some trickery. What they did was to place a colored filter over each photo receptor in a mosaic pattern. The most commonly used pattern is called the Bayer Pattern. This consists of a row of filters over the photo-receptors alternating Green and Red. The next row alternates Green and blue. Then back to Green and Red for row 3, etc. By placing a colored filter over each photo-receptor, only photons of that color can get through the filter and be counted by the receptor. A camera RAW file captures this raw data as detected by the photo receptors. So, the numeric value representing any "dot" on the sensor will be restricted to the brightness of only one of those 3 colors.

When a RAW image is rendered (i.e. processed) - meaning converted to pixels - the RAW converter (e.g. Lightoom or Adobe Camera RAW) looks at groups of 4 photo-receptor values in a 2 by 2 square and then calculate the "real" color based on merging those 4 values. It then moves 1 receptor over and does it again. So, the pixel at R1C1 (row 1, column 1) is the average of receptors R1C1, R1C2, R2C1 and R2C2. The color of the pixel at position R1C2 is made up of receptor values from R1C2, R1C3, R2C2,and R2C3. And so on through all the entire sensors's worth of photo-receptors.

But, one camera may use slightly different opacity color filters in their Bayer pattern, or may use a slightly different shade of green or red or blue than another company. So, the Rendering algorithm must be calibrated to match the sensor of each model camera. That's why with LR you have to wait for an update to be able to process RAW images from a new model camera. Adobe, must reverse engineer the RAW data from that model camera using photographs from images of known color subjects (a color target). And, to add to the complexity, Adobe may wind up with a different formula than On-One or any other SW vendor that can render RAW images. In most cases they are quite similar but from time to time the differences are noticeable. In other words a RAW file rendered by Adobe my look a tad different than one rendered by Apple Photo or On-One.

So, what then is a DNG? Well, in one sense it is a RAW file in that it contains 100% of the information from the Camera sensor. However, it may or may not contain 100% of the metadata information from the camera. For example, let's say the Fred's Camera company decides to put a thermometer in their cameras and provide the ambient temperature inside the camera as metadata on each shot. Well, the tool that converts RAW files from Fred's Camera models) will probably ignore that temperature value and it will not be included in the DNG. What I don't know for sure is if the pixel information in a DNG is still in the Camera's Bayer pattern or has been rendered to color pixels by the DNG converter. I strongly suspect that the color values in DNG's are pre-rendered or else DNG's would not be universal.

Moving on, JPG, TIFF, and PSD files among others are "pixel" based file formats, not RAW files. In other words all that interpretation and the rendering (also called "demosaicing") of the Bayer Pattern has already been done by some RAW converter with all that entails. If you are getting JPG's or TIFF's or even DNG's from the camera itself, all that means is that the Rendering of the RAW file was done in the camera and can be considered as just another piece of software similar to LR or ACR or On-one.

So, what does all this mean? Well, it means that images may look different depending on who wrote the rendering software (including rendering SW built into camera's). As to which is "correct" there is no answer other than "the one that gives you the results that you like the best", or "the one that seems to give you the most accurate colors from you camera".

Okay, too much writing and detail but once I started I couldn't stop. Just file this info into the "facts not worth knowing" part of your brain and move on to something fun.
 
Joined
Jan 18, 2009
Messages
2,502
Location
Fort Myers, FL
Lightroom Experience
Advanced
Lightroom Version
Classic
One of the reasons this gets so complicated is that the terms of slippery, and mix up convention with specification.

For example, a DNG is a TIFF. A raw file (most of them anyway) are TIFF files. TIFF is a file structure.

But 99% of the time when you ask a photographer about a TIFF file, they mean one already converted from bayer/raw format to color pixels.

And probably 80% of the time when you ask a photographer about a DNG, they mean one NOT converted to pixels, but just wrapped up in a DNG structure. This used to be probably 99% also, until Adobe started doing its panoramas and other special outputs as DNG files.

All of this is a vast conspiracy by the medical industry to sell more headache cures as people try to figure it out. ;)
 
Joined
Jun 24, 2010
Messages
1,689
Location
Encinitas, CA USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Dan and Linwood,

Great job at writing out thorough explanations of how image sensors collect data. Following the saying, "A picture is worth a thousand words", I will add a link to the following Understanding Digital Camera Sensors.

-louie
 

PhilBurton

Lightroom enthusiast (but still learning)
Premium Classic Member
Joined
Nov 16, 2015
Messages
2,702
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic
One of the reasons this gets so complicated is that the terms of slippery, and mix up convention with specification.

For example, a DNG is a TIFF. A raw file (most of them anyway) are TIFF files. TIFF is a file structure.

But 99% of the time when you ask a photographer about a TIFF file, they mean one already converted from bayer/raw format to color pixels.

And probably 80% of the time when you ask a photographer about a DNG, they mean one NOT converted to pixels, but just wrapped up in a DNG structure. This used to be probably 99% also, until Adobe started doing its panoramas and other special outputs as DNG files.

All of this is a vast conspiracy by the medical industry to sell more headache cures as people try to figure it out. ;)
+1.

And to compound the crime, there are also proprietary formats, such as Adobe's PSD for Photoshop and Apple's new HEIC as a replacement for JPG. And then there are "standard" formats that seem to have about-zero adoption, such as JPEG2000.
 

Zenon

Senior Member
Joined
Jan 13, 2017
Messages
1,905
Lightroom Experience
Intermediate
Lightroom Version
Classic
I found this in a nut shell explanation on a forum long ago.

If you want your color adjustments in DXO to stick then you need to export as TIFF to Lightroom.

If you use DXO for lens correction and noise reduction only and want to do all color work in Lightroom export as DNG.
 

LRList001

Active Member
Joined
Dec 23, 2012
Messages
419
Lightroom Experience
Intermediate
Lightroom Version
6.x
At the risk of causing more confusion, there is a free raw editor called Rawtherapee (Windows, MacOS, Linux all 64 bit). It allows you to choose a number of different de-mosacing algorithms, one of which attempts to show the raw data. It can be revealing in understanding more of what is going on. (I've only used the Windows version.)

Also, if you want to get to grips with the difference between the file container and the data within the file, you might find it informative to read up on the difference between jif and jpeg (and maybe understand why jfif (and others) was developed).

If you go to JPEG - Wikipedia and scroll down to section 4, JPEG files, it gives a clue as to some of the problems of jif.

The key bit is here: "... and are stored in variants of the JIF image format "

A jpeg file is not some nicely defined file format, it is really something of a mess. It is amazing it works as well as it does. You will find this variability happens in many of the older file formats, and pretty well all of the widely used ones. It takes a very single minded purpose (and single owner) to keep a popular file format closely defined.
 
Joined
Jun 24, 2010
Messages
1,689
Location
Encinitas, CA USA
Lightroom Experience
Advanced
Lightroom Version
Classic
Hi,
Let me try to summarize.
I understand that LR (and PS) contain a camera raw editor and for years I have been dutifully converting my Canon .CR2s and Panasonic .RW2s into DNGs while importing them into LR.
Q1. Have they been edited?

No not by the conversion from the original raw (from the camera) to what should be labeled as raw DNG. This is the default mode of Adobe Camera Raw (ARC) in the DNG converter, Lightroom and Photoshop. As stated in the previous posts this form of DNG has the original de-mosaiced camera sensor data. In order to view it has to be fully rendered into a RGB image.

If you have done any editing within Lightroom that will only reside in the Lightroom catalog as metadata and will not be recognized by any other raw processors even if you save the develop settings back to the raw DNG. If you open that raw DNG in another raw processor it will ignore the Adobe settings and start from scratch.

I have heard good things about the PhotoLabs RAW editor and I like to use the Nik plug-ins. Before I spend any more money ...
Q2. if I want to utilise the PhotoLab processor is it too late for my old photos which are now DNGs?
No not too late. I believe that DxO PhotoLab raw processor will read the raw DNG files containing your Canon CR2 raw format and Panasonic RW2 raw format and and allow you to edit them using the DxO editor. But it will ignore any edits made previously with Lightroom per above.
I note that PL can return the files to LR as DNG or TIFF files - so that means ... er ... what exactly?

As stated previously PhotoLab has an option to output a different DNG format called linear DNG. This is a kind of intermediate format which has a lot of the advantages of original raw camera data. But it is demosaiced into RGB, every pixel now has a RGB value but has not been fully rendered. In short it passes along a lot of the advantages of raw data without without the need to demosaic. In this form it can be further processed by another raw processor (ACR, etc.), where it can be mapped to a standard color space, tone mapped, set white balance, etc. and turned into a fully rendered RGB image, TIFF, JPG, PSD, etc.

Adobe ACR both reads and writes linear DNG. So potentially you could do some initial processing of the image in DxO. Save that as a linear DNG and continue to edit it in Lightroom or Photoshop ACR. The advantage of this depends a lot on kinds of adjustments DxO PhotoLab allows you to include and or exclude from the linear DNG.

TIFF files on the other hand are fully rendered RGB data. All the things like color space, tone mapping, white balance are baked int and cannot be as easily adjusted as can with raw DNG or a linear DNG.

I am not sure where the Nic Plugins fit into the PhotoLab flow at this point. The last version I have, the free release from DxO was standalone still required fully rendered RGB data. And the output final result was fully rendered RGB. My best guess based on what these plug-ins are doing is that they still require fully rendered RGB data.

-louie

PS I found the following What is Linear DNG? very enlightening about the differences in DNG.
 

PhilBurton

Lightroom enthusiast (but still learning)
Premium Classic Member
Joined
Nov 16, 2015
Messages
2,702
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic
Some excellent information here. I'm going to bookmark the thread, but is there anywhere this knowledge is all collected together?
Wikipedia?
 

walkytalky1000

New Member
Joined
Oct 20, 2014
Messages
8
Location
Northumberland, UK
Lightroom Experience
Intermediate
The thread has calmed down now so it is time to the thank the gurus and other assorted experts for time devoted to answering my "simple" questions. I imagine I was wrong when I suggested that the answer to my question was clear to many.
Thanks again.
 
Status
Not open for further replies.
Top