Develop module DNG file type generated by Denoise, Lightroom Denoise workflow

[rip]

Lightroom Version Number

12.5

Operating System

1. Windows 10

There appears to be some controversy about what kind of DNG file is produced by Lightroom Denoise.

How can I tell in Lightroom Classic whether a DNG file is a RAW file, or if it has been converted to an RGB DNG?

Specifically regarding Lightroom Denoise, is this statement from Adobe true, or is it false?
"After multiplying and adding up a gazillion numbers, your computer will produce a new raw file in the Digital Negative (DNG) format"
https://blog.adobe.com/en/publish/2023/04/18/denoise-demystified

Also, is the DNG metadata of a DNG file produced by Lightroom Denoise accurate, or is it inaccurate?

The metadata states that the Denoise file contains

:


For some reason the phrase

triggers a 403 forbidden error on this forum.

If this file has been converted to RGB, where can I find the bit depth in Ligthroom?

Does running Lightroom Denoise eliminate the benefits of working with RAW files in Lightroom to adjust Exposure, Whites, Highlights, Blacks, Shadows, Texture, Clarity, Dehaze, Sharpness, HSL, etc?

Is the correct workflow to export the DNG's generated by Lightroom Denoise to Photoshop directly without any Lightroom adjustments?

Thanks!

 

[John R Ellis]

The type of DNG produced by Denoise is a confusing issue, exacerbated by imprecise terminology and a misleading Metadata panel. There's no short answer to your questions, just a long one:

The from-the-camera raw file contains Bayer sensor mosaic (aka color filter array, or CFA) data in a single monochrome channel of 10-16 bits per pixel.

Denoise produces a DNG containing "linear raw" (aka "linear RGB"), where the mosaic sensor data has been transformed (demosaiced) into three RGB channels of 16 bits/channel/pixel.

For nearly all purposes, LR and Camera Raw treat a linear raw the same as a mosaic / CFA raw. In particular, the Develop settings behave identically.

Adding to the confusion, a Denoise DNG also contains the original mosaic / CFA data, which is why the Metadata > DNG panel says "Mosaic Dxta: Yes", even though Develop uses the demosaiced linear-raw data. Adobe has been vague about why they included the original mosaic / CFA data in Denoise DNGs, other than to say it's there in case they add additional processing commands in the future that could leverage it.

In the early days of Denoise, Adobe employees confused matters by imprecisely saying that the Denoise DNG "embedded" the original raw, but this isn't the same kind of embedding that the Adobe DNG converter does, in which all the bits of the original raw file (including its metadata) are copied verbatim into the DNG and which can be extracted later to recover the original raw file. Denoise DNGs do not embed the entire original raw file -- they contain just the original mosaic / CFA data, transformed into the standard representation used by DNGs for such data.

If this file has been converted to RGB, where can I find the bit depth in Ligthroom?

The Bits Per Sample shown in the Metadata > DNG panel indicates the number of bits per pixel recorded by the camera sensor.

More generally, one must be careful when interpreting "bits per channel", "bits per sample", or "bit depth" as displayed by other apps -- it could refer either to the bits per pixel recorded by the camera sensor or to the number of bits per pixel used to record the data in the file format (which might be larger than the sensor's bits per pixel):
https://www.lightroomqueen.com/comm...it-on-14-bit-gfx-100-shots.43767/post-1289852

For example, a camera sensor might produce just 12 bits per pixel, but that could be saved in the file using 16 bits per pixel.

 

[John R Ellis]

I had to edit my full reply above due to the problem typing

into the reply. I kept getting "Oops, a problem occurred", which I guess is what you meant by "a 403 forbidden error on this forum".

 

[rip]

Thanks for the detailed reply.

Yes, if you check the browser console, a 403 forbidden error, is generated every time the phrase "atad ciasoM" is typed.

Is there any software that can edit either the original DNG generated by Lightroom import or the "linear RGB" file generated by Lightroom Denoise?

I'm trying to understand where is the earliest point in my workflow that I can combine areas from two images into a single image if my workflow follows this sequence:
RAW to DNG convesion on import to Lightroom, Lightroom Denoise, Lightroom Preset Adjustments, Photoshop

I tried opening the original DNG conversion in Photoshop and saving it as a 32 bit TIFF, but
1. Lightroom can't run Denoise on this file type and
2. When open in Lightroom the Histogram is altered, the image is washed out and overexposed.

If Photoshop can't do it, is there some other software that can combine areas from two DNG files into a file type that Lightroom Denoise can read or a type that preserves the headroom for Lightroom Presets afforded by DNG RGB?

Thanks again.

 

[clee01l]

The DNG file format contains a data block for the data that you can view. There are other data blocks than can contain thumbnails and even full lossy compressed JPEG data. It is the metadata that determines what is in the Main data block There is an indicator in the metadata that describes the main data block as RAW photosite values or RGB Pixels. The RGB values can be the same values found in a Lossless TIFF File or in the case of a lossy DNG, Lossy RGB values

A DNG created by a Camera as RAW will have Photosite data in the main data block and a metadata tag indicating that the Main data block contains RAW photosite values. Similarly A Proprietary RAW file copied to a DNG will have RAW photosite values copied from the Proprietary File and indicated as such in the metadata description for the main data block.

The DNG files produced by the DNG process are not fully RGB and not fully RAW. The metadata description for their main data block should indicate this. I do not fully understand the data type produced by the NR process It seems to be what might be described and an inter mediate between fully RAW photos sites and RGB pixels. As I understand the ACR process to convert RAW files. The first step is to Demosaic the data. That is to take the Photosite values and apply an algorithm used to reconstruct a full color image from the incomplete color samples output from an image sensor overlaid with a color filter array (CFA) such as a Bayer filter. In this sense I guess they would be considered “Mosaic Data”. The next step in the ACR process would be to apply tone corrections, sharpening and some noise reductions.

It. After this Mosaic process the Lightroom DeNoise would be applied rather than the standard NR applied to all RAW files as well as tone corrections and sharpening


Sent from my iPad using Tapatalk

 

[John R Ellis]

I'm trying to understand where is the earliest point in my workflow that I can combine areas from two images into a single image if my workflow follows this sequence:
RAW to DNG conversion on import to Lightroom, Lightroom Denoise, Lightroom Preset Adjustments, Photoshop

LR and Camera Raw can't in general combine areas from two images into a single image, in the way that Photoshop can. They can do HDR and panorama merges, but not combine arbitrary areas (e.g. copy eyes from one and paste them into another). So the earliest in that workflow you could combe general areas from two photos is the last stage, editing in Photoshop.

You can't put Photoshop before Denoise, since Denoise requires a raw with mosaic/color-filter-array data, and Photoshop can't edit mosaic/CFA data -- you have to convert to a RGB format first.

 

[Jimmsp]

From a workflow point of view, I import my raw files as that - I don't import/copy them as .dng, just maintain the original raw. I don't know if there is any advantage or disadvantage either way. I just figured it took less time for a large number of files that I hadn't culled through yet.
But then if I have shot high ISO, I try to run the LRC Enhance Denoise first (if I remember) before I do anything else. Occasionally I have forgotten, and if I have already made tonal adjustments, the end result still looks ok to my eye. Then if I "edit in" Photoshop, I have LRC set to export a tiff. And as John said, that is the first time I can actually combine parts of images if I so choose. But those are on my partially processed tiffs.
I don't see any reason yet to change this workflow; though I could be missing something.
I sometimes run Topaz Photo AI before I go to PS, sometimes in PS, sometimes afterward - but in all cases it is working on tiffs.

 

[karl.nyhus]

This thread is filled with confusion and difficult-to-explain terms and concepts. I blame Adobe for sharing processes whose design is still very much in flux.

Adobe started with the idea that a DNG file would be Adobe's standard version of a raw file that could replace the chaos of incompatible proprietary raw files. So far, so good. But it is no longer as simple as that.

DNG files these days can be much more than a raw file replacement. Adobe now uses DNG for all sorts of intermediate results, like what comes out of its Denoise processing.

Tellingly, it is obvious that even Adobe doesn't completely trust its raw-to-DNG conversion to include all needed data as it allows an original proprietary raw file to be embedded in the various types of DNG files (or now, apparently, just the sensor mosaic/color-filter-array portion of the raw file).

 

[clee01l]

Adobe started with the idea that a DNG file would be Adobe's standard version of a raw file that could replace the chaos of incompatible proprietary raw files. So far, so good. But it is no longer as simple as that.

DNG files these days can be much more than a raw file replacement. Adobe now uses DNG for all sorts of intermediate results, like what comes out of its Denoise processing.

The DNG file type is an open file type and as designed by Adobe is extensible to include any type of data in the main data block. It was never intended to be exclusively a RAW file container. DNG is part of the TIFF/EP6 family of files. The confusion lies, I think, in the misconception that DNG files are RAW files. The truth is "They Can Be RAW files but never exclusively RAW".

About the best summary of what DNG is can be found on Wikipedia https://en.wikipedia.org/wiki/Digital_Negative

 

[karl.nyhus]

The confusion lies, I think, in the misconception that DNG files are RAW files. The truth is "They Can Be RAW files but never exclusively RAW".

Exactly!

 

[karl.nyhus]

The DNG file type is an open file type and as designed by Adobe is extensible to include any type of data in the main data block.

But that's where the confusion comes in. The DNG file type was initially pitched as a way to reduce confusion caused by each camera maker having their own raw file. Now DNG is its own source of confusion as the DNG file extension on the end of a file name gradually comes to mean "anything and nothing" as Adobe "extends" its uses.

 

[clee01l]

But that's where the confusion comes in. The DNG file type was initially pitched as a way to reduce confusion caused by each camera maker having their own raw file. Now DNG is its own source of confusion as the DNG file extension on the end of a file name gradually comes to mean "anything and nothing" as Adobe "extends" its uses.

I like to think of DNG as a universal file type. The problem, if there is one ( and I don’t think there is one), is that you need to read the metadata to find out what’s in the main data block. The issue is similar to the TIFF format. All you can say about the TIFF file is that it is RGB. Is it 8 bit, 16 bit, or 32 bit? Is it compressed or uncompressed? If Compressed is it Lossy compressed or losslessly compressed?


Sent from my iPad using Tapatalk

 

[karl.nyhus]

Cletus, that's all very well for Adobe, but have you or anyone else answered all of the OP's questions from the beginning of this thread?

Six questions were posed that did a fair job of illustrating the current confusion around DNG files.

 

[clee01l]

Cletus, that's all very well for Adobe, but have you or anyone else answered all of the OP's questions from the beginning of this thread?

Six questions were posed that did a fair job of illustrating the current confusion around DNG files.

I gave it my best shot. As did John Ellis.


Sent from my iPad using Tapatalk

 

[karl.nyhus]

I gave it my best shot.

And I thank you for the information you and John shared. But confusion remains and that should be laid squarely at Adobe's (design) doorstep.

 

[clee01l]

And I thank you for the information you and John shared. But confusion remains and that should be laid squarely at Adobe's (design) doorstep.

I don’t think there needs to be confusion. Certainly, the Adobe Design is not flawed. I do agree that Adobes marketing and naming conventions could be a lot better, with the best example being the “Lightroom CC” confusion released nearly 10 years ago and still persisting to this day.


Sent from my iPad using Tapatalk

 

[Jimmsp]

Does running Lightroom Denoise eliminate the benefits of working with RAW files in Lightroom to adjust Exposure, Whites, Highlights, Blacks, Shadows, Texture, Clarity, Dehaze, Sharpness, HSL, etc?

Is the correct workflow to export the DNG's generated by Lightroom Denoise to Photoshop directly without any Lightroom adjustments?

I have been following this thread, and for me the discussions about bit depth are almost irrelevant - as I am pretty well locked into the Canon CR3 raw files, Lightroom Classic and Photoshop. They are what they are.

The two questions I quoted above are for me, and I suspect others , the most important ones.

I think the answer to the first one is "no" , but I only say that by observation of my results on my high noise shots I sometimes take in low light. And it appears to be irrelevant to the order, though I try to follow the original Adobe advice to Denoise first, then work on the output dng.

Likewise, to question 2, if the answer to the first question is "no", then when you export to Photoshop should also be irrelevant. I for one try to as much as I can before I export a tiff to Photoshop.

I have no technical basis for saying this, just experimental observation.

 

[PhilBurton]

Let's say that you have a RAW image from 2016. You have already edited it and produced JPGs and maybe events. Now you want to DeNoise the original RAW file Is there a way that you can carry over the exiting develop history and apply it easily to the denoised DNG?

And I agree that the expanding definition of DNG can be confusing.

For marketing geeks like me, Adobe needs to improve its "message architecture."

 

[clee01l]

Let's say that you have a RAW image from 2016. You have already edited it and produced JPGs and maybe events. Now you want to DeNoise the original RAW file Is there a way that you can carry over the exiting develop history and apply it easily to the denoised DNG?

And I agree that the expanding definition of DNG can be confusing.

For marketing geeks like me, Adobe needs to improve its "message architecture."

I understand that a RAW file can be edited in LrC without going through the DeNoise. The recommended workflow is to begin with a RAW file and apply DeNoise, then further adjustments. However I have found that if you have an adjusted RAW file you can take that RAW file into the DeNoise process and when it comes out the RAW edits applies to the raw file will be applied to the DNG. This makes sense since a) you can’t apply edits to a RAW file only to its RGB derivative and b) Adobe determines the order that edits get applied.


Sent from my iPad using Tapatalk

 

[John R Ellis]

I've observed the same thing as Cletus. I've denoised old raws and used Sync Settings to copy the edits from the raw to the DNG, and generally get good results (even though the edits were originally created on a non-denoised raw, and Eric Chan recommends doing Denoise first).

 

[PhilBurton]

I understand that a RAW file can be edited in LrC without going through the DeNoise. The recommended workflow is to begin with a RAW file and apply DeNoise, then further adjustments. However I have found that if you have an adjusted RAW file you can take that RAW file into the DeNoise process and when it comes out the RAW edits applies to the raw file will be applied to the DNG. This makes sense since a) you can’t apply edits to a RAW file only to its RGB derivative and b) Adobe determines the order that edits get applied.


Sent from my iPad using Tapatalk

That's the best possible answer. Seems like Adobe really though this feature. +1 for the Adobe Lightroom team here.

Phil

 

[vbnut]

I've observed the same thing as Cletus. I've denoised old raws and used Sync Settings to copy the edits from the raw to the DNG, and generally get good results (even though the edits were originally created on a non-denoised raw, and Eric Chan recommends doing Denoise first).

I thought I read somewhere (I can't find it now) that if I run Denoise on a Raw file that already has some adjustments, those adjustments will automatically be incorporated into the DNG created by Denoise. I assumed that meant there was no reason to sync the adjustments from the Raw to the DNG. Did I remember incorrectly (or interpret what I read incorrectly)?

 

[John R Ellis]

if I run Denoise on a Raw file that already has some adjustments, those adjustments will automatically be incorporated into the DNG created by Denoise. I assumed that meant there was no reason to sync the adjustments from the Raw to the DNG.

That's correct -- no need to copy the settings. Senior moment on my part.

 

[Victoria Bampton]

For some reason the phrase View attachment 23289 triggers a 403 forbidden error on this forum.

That is a mystery! Without the colon, I can write Mosaic Data just fine, but even I get stopped if I try to add the colon. There are 2 layers of security software running to protect both us and you, and it's not showing up in any of the logs. Since it seems to be a rarity, I'm inclined to ignore it unless it starts happening more frequently.

 

[clee01l]

I think the

That is a mystery! Without the colon, I can write Mosaic Data just fine, but even I get stopped if I try to add the colon. There are 2 layers of security software running to protect both us and you, and it's not showing up in any of the logs. Since it seems to be a rarity, I'm inclined to ignore it unless it starts happening more frequently.

I think the Security Software is trying to prevent the injection of malicious code in the message. "Data(no space):" is the trigger that malicious code can follow. I had to separate the Colon from the word "data" to get this to post.