DNG Confusion from an Experienced User

[GregJ]

Lightroom Version Number

Latest Classic

Operating System

1. Windows 11

I have used LR almost daily for 12 years and am accustomed to working with Fuji, Canon and Nikon raw files which all have sidecars that I write all edits and metadata to (besides just to the catalog). I have shot the Leica Q2 a lot lately and just shot it every day for 7 weeks in Rome and Sicily. Leica uses DNG for their raw files.

I have a folder on my base PC with the 2,400 edited DNG raw files shot on our recent Sicily trip (imported to LR from the laptop when I got home). As always, I had “Automatically write changes to XMP.” Since DNG files have no sidecar XMP, those changes are written directly to the DNG file itself.

There are advantages and disadvantages to that. I hate it because I sync to external drives as backup using GoodSync, and any edit in LR causes the whole DNG file to resync on a GoodSync job later on when I backup my big 6TB raw image folder to several other single 8TB drives. So instead of just a tiny sidecar file syncing, the whole DNG file has to overwrite the older version of the file on the GoodSync job. That is annoying and slow, but not my question – just background.

A week after returning from Italy, I decided to skim through the images again in LR on the big 32- inch 4k mini-LED IPS pro monitor. You see things on that fantastic monitor that you don’t see on the 4k 15-inch laptop on the road. So I did some more fine tuning edits. On the next GoodSync job, it said “No changes.” But there were changes. I edited 400 of those files in LR that day and all 400 should have been picked up by GoodSync when I ran a sync job. But it said no change. My changes were not being written to the DNG, despite having checked “automatically write changes to XMP.” So I went into the Photo menu to “Update DNG Previews and Metadata.” I selected all 2,400 DNG files and clicked on that, thus starting the longest LR project in history. What was happening? All the metadata changes were written in (but should have been already there). It was a lot slower than generating 1:1 previews at import. I have very fast gaming PC with 64 ram, M.2 SSD for Adobe and the system and 3080 GPU.

I create 1:1 previews on import, and that goes pretty fast. My CPU maxes out and my fans sound like a jet talking off. But when I use “Update DNG Previews and Metadata” the update moves at a snail’s pace and the CPU only goes to about 20% usage. What the heck is going on with that “update DNG Preview & Metadata”?

So here are my questions if anyone knows the answers:

- What is the difference between “Automatically write changes to XMP” and “Update DNG Previews and Metadata” when working with DNG files?

- Does “Update DNG Previews and Metadata” redo the 1:1 preview file stored in the .lrcat file that I generate at import or is it updating the embedded jpeg in the raw DNG file? When is best to do this? Is there a way to make it automatic or do you have to select the images and run the update?

- After an editing session, should I always highlight all the files and click on “Update DNG Previews and Metadata”? When is it best to do that?

- What is the difference in the Metadata that writes to the file in those two menu items? “Automatically write changes to XMP” writes all edits and metadata changes to the file. When does that writing to the DNG file happen? Instantly after edits or at some specified time on exit? Then, metadata is also written to the file if you execute “Update DNG Previews and Metadata.”

I read all the help files and I don’t think Adobe makes this clear. I might have missed something.

 

[GregJ]

Further Note: I just did some experiments with the latest LR Classic update and noticed that the exports fly way faster and better. I exported 200 full-size jpegs to a test folder on the desktop, so it was all happening on a fast M.2 SSD.
The export went way faster than it used to. My CAM software reported the CPU running at over 5000 MHz for a brief second on each file export but at only 20% load, whereas before, the CPU maxed out 100% load the whole time and the GPU was little or not used on exports. Now both CPU and GPU are working the export and it is much faster and quieter as the fans don't ratchet up on my fan profiles because the CPU is much cooler on that big export job. It is amazing really. It is true that I just installed a new RTX 3080 Ti GPU now that the prices are finally coming down. But it's not that I don't think. I think LR just makes better use of the GPU on more LR actions now. I could be wrong.
I need to see how generating about 300 1:1 preview files goes. I bet its faster and uses both CPU and GPU better now if it is anything like exporting full-size jpegs from the raw.

 

[Califdan]

Not clear if LrC is failing to updat the DNG's or if GoodSync is failing to detect that those DNG's had been updated. Do an experimanet. Make some changes in LrC to one of the DNG's. Then open the DNG directly from photoshop (not through Lightroom) and see if the changes are there. That should tell you if LrC is writing the changes to the DNG. Don't save from Photoshop but if the changes were seen in PS, see in GoodSync picks up the file. If it says "no changes", then test that statement by resorting the file from the GoodSync destination and open the restored file in PS to see if the changes are there. If they are then the GoodSync message of "no changes" was bogus.

 

[johnbeardy]

So here are my questions if anyone knows the answers:

- What is the difference between “Automatically write changes to XMP” and “Update DNG Previews and Metadata” when working with DNG files?

"Automatically write" only saves xmp data to the files. This xmp only contains some of your LR work, not all, and is mainly intended for exchanding common metadata with other apps or users.

"Update DNG previews...." updates the previews as well.

- Does “Update DNG Previews and Metadata” redo the 1:1 preview file stored in the .lrcat file that I generate at import or is it updating the embedded jpeg in the raw DNG file? When is best to do this? Is there a way to make it automatic or do you have to select the images and run the update?

It updates the embedded JPEG. It does happen automatically when you use ACR in Bridge/Photoshop, but in LR there's no automatic settings. You choose to run the updating, or not.

- After an editing session, should I always highlight all the files and click on “Update DNG Previews and Metadata”? When is it best to do that?

When the DNG needs to be viewed in another app or elsewhere. Otherwise, rarely if ever.

The ability to update the DNG embedded preview is a bit of a hangover from the early, pre-Lightroom days of the DNG file format. Before Aperture and LR many people would manage photos with a cataloguing program which relies on the embedded preview, and correct them with another such as ACR which didn't change the embedded preview (Nikon Capture did update them but was an outlier). You'd be managing photos without seeing their corrected appearance.

Once DNG came along, its updated previews meant you could manage photos and see correct WB, dust spots, brightness and colour, and that was a big advantage that drove many people to adopt the format. But once Aperture and Lightroom arrived, managing and adjusting were combined in one application, and the need for the DNG embedded previews was greatly diminished.

But imagine you are sending raw file to someone who doesn't use LR or Bridge to view the pictures - eg PhotoMechanic, Explorer or Finder. That's hen the updated preview might be handy. How often is that?

- What is the difference in the Metadata that writes to the file in those two menu items? “Automatically write changes to XMP” writes all edits and metadata changes to the file. When does that writing to the DNG file happen? Instantly after edits or at some specified time on exit? Then, metadata is also written to the file if you execute “Update DNG Previews and Metadata.”

It's the same metadata.

I've always been sceptical of “Automatically write changes to XMP” with DNGs as it can trigger backup tools to make multiple backups of the DNG. This can bloat the storage requirements, especially with cloud backup, but it also encourages a false sense of security because writing XMP is mainly designed for data exchange, not backup, and omits a range of LR metadata. So if you do enable that automatic setting, you need to fine tune your backup procedures so that DNGs are backed up once when they are new and your Lightroom work is backed up routinely by backing up the catalogue.

 

[BobT]

If an experienced user is confused, how do you think I feel? I don't even want to think about it. I import, convert to DNG, edit and if necessary, export as JPG and that's it.

 

[johnbeardy]

"Experienced" has a variety of meanings

My summary would be:

• Back up DNGs when they are new + don't keep backing them up
• Back up your catalogue routinely
• 50-50 on enabling automatic saving
• Only update the preview for 3rd party requirements
• Occasionally run the DNG validation

 

[clee01l]

Greg has asked me to comment on this thread. I only say that I agree completely with John Beardsworth's comments.
Greg has also stated that

Anyway, I'm starting not to like DNG.

I think you might be more of a victim of the false impression that you need to write metadata out separate from the metadata contained in the Lightroom Catalog.
If you treat the Leica RAW files like any other RAW file, then what is there not to like? You do not need to backup any original RAW File more than once. If ALL of the metadata is in the LrC catalog file and your system backup is backing up that along with any Lightroom Classic Backup copy, then you are well protected from ever losing your metadata changes.
IMO the original image file RAW or otherwise should never be accessed by any app other than the DAM tool that is managing it (LrC). When you create an export derivative file in Arc, you are not going to use the Metadata from the original file but you will use all of the metadata that you stored in the catalog file to create that export.
Remember, the DNG file specification is just a container like any other image file format. It can contain a RAW Photosite data block (as in the Leica camera file) and it always contains at least one RGB data block. DNG has been developed by Adobe because it is so flexible in regard to the data and data types that it can contain. Proprietary RAW filetypes are not flexible because the Header block containing the metadata is fixed is size and structure and the RAW data block characteristics are rigidly determined by the proprietary information contains in the proprietary RAW Header block.

 

[GregJ]

I've always avoided DNG because of the lack of sidecars, and was amazed that my fellow Canon, Nikon and Fuji shooters would convert to DNG. No way would I ever do that. But now I'm shooting two wonderful little Leica Q2 cameras (regular and mono) so have thus entered the Alternate Universe of Leica, and thus am forced into the dark world of DNG. Now I like DNG even less than before.

John, I want to thank you for the DNG background information as well as the explanation of the two menu items I asked about concerning previews, embedded jpegs in the DNG and writing metadata and edits to the DNG directly because of no sidecars

Also thanks for the GoodSync explanation. Goodsync is my long-sought (after many years of other alternatives) solution for backing up my image folder which resides on a single 8TB internal Sata SSD in my PC and contains thousands of folders and many hundreds of thousands of raw files, as well as 25,000 TIFF scans of all my previous decades of film shooting. It is a single folder with almost everything I have shot in my life, and I'm 65 and been shooting since 15, so that's 50 years. I don't so RAID (for many I think good reasons), and I have about 5.5 TB of files. That means they fit on a single 8TB SATA SSD and can be backed up with GoodSync (duplicated exactly) to several single 8TB spinning drives. That means If my 8TB SSD fails, I can switch instantly to one of my external drives, buy a new SSD, copy back to it, connect the new drive back to LR and I'm good. That has happened only once, and I dream of the day that my files are all on an internal 12 TB M.23 PCIE SSD. In fact, an 8TB M.2 PCIe Gen 4 drive just hit the market for 1500 dollars and I'm getting it as my main data drive. The 8TB SATA SSD will become the internal backup drive for my image file folder.

Anyway, thanks because I have discovered that GoodSync is not picking up this small edit changes that are supposed to be written to file (to the DNG) after a small edit in LR. Sometimes it does, but usually it doesn't. That is a disaster for me and makes me lose confidence in GoodSync. I have to rely on it to produce an exact copy of my image folder to those other drives (I do it 5 times - 5 single stand-alone drives that contain exact duplicates of the image folder on my main PC 8TB SSD. Now I'm worried that when I run a sync job with GoodSync that it might be missing an update I made to a file - especially DNG files in LR. That is not good. (I ran your test. Bridge has the small correction I make in LR, but then a GoodSync session says no change. Not good!

Now, about your comment concerning "automatically writing changes to XMP". I don't post much here, but a few years ago there was a thread I ran about importing the catalog into your home base desktop from road-trip shooting on a laptop. I'm a travel shooter and edit files every night on my laptop on the road. When I get home its already all done. The culling, developing and naming of the files (titles, captions, file names, etc...) is all done. I had a conversation with the Queen about this and my odd way of doing it was deemed OK for me, but not taught in any LR class or book. The bottom line is that the XMP sidecar file contains all of the edits I do in LR and all the metadata. The only think I do that is not contained in the sidecar is any flags or ratings. Literally everything else I ever do in LKR is in the sidecar file. All slider edits, masks, file names, captions, time changes, title - you name it - it is in the sidecar file. Why is that important to me? It is vital. Because I don't import the catalog when I get home. I simply copy the files from that trip with their sidecar files from the laptop to whatever folder I want in my PC. Then I import that folder into my desktop LR catalog. Done. Every edit and all metadata is there - except for flags. I can't find anything else that is not there. Why do I do it that way? Because a few years ago I tried importiung the catalog techniques and i had problems. It was a mess. It created folders and time stamps I didn't want and it messed with my intricately devised folder system. I felt like I lost control and weird stuff happened. I'm sure I could learn how to do that cat import from my laptop to my PC at home better, but just copying the files with their sidecars from my external SSD (which is what I backup to from my laptop on the road each night) directly to a folder I name in my desktop is super-fast and simple. Then it takes about ten seconds to import that folder into the desktop main LR catalog. What did I lose? Nothing.

Now the DNG files from my recent Q2 shoots. Let's use this last 7-week Sicily trip as an example. My wife and I returned home with 2,400 DNG files in my laptop, fully edited in LR and copied to the external SSD. We walked in the door of my San Antonio condo after being gone 7 weeks. I walked over to my PC and fired it up so it would start updating Windows, Adobe, Norton, MS Office and all my many other programs. About an hour later after my PC was updated after a 7-week absence, I plugged in the SSD which had an exact copy of the 2,4000 DNG files that were on my laptop, and I copied them to a folder called Sicily 2022 in my desktop. That took about 4 minutes. Then I imported that folder into the LR Catalog. That took 15 seconds. Done. The next day I looked through those images on my desktop PC and that awesome monitor. Wow. Those Q2 DNG files were almost as good as my Fuji GFX Medium Format work. Almost. Closer than I would have thought. Anyway, all my edits (including masks) were there. All my metadata changes were there (titles, captions, time changes, camera data - everything as far as I could tell. What was not there? I believe you when you say the write to DNG might not be as thorough as the write to XMP sidecars when using the proprietary raw files of the major brands.

Any comments on my weird way to get my road laptop work into my desktop base home system? I have only one catalog, and that is on my PC. All files get erased from my laptop after every shoot.

 

[GregJ]

Greg has asked me to comment on this thread. I only say that I agree completely with John Beardsworth's comments.
Greg has also stated that

I think you might be more of a victim of the false impression that you need to write metadata out separate from the metadata contained in the Lightroom Catalog.
If you treat the Leica RAW files like any other RAW file, then what is there not to like? You do not need to backup any original RAW File more than once. If ALL of the metadata is in the LrC catalog file and your system backup is backing up that along with any Lightroom Classic Backup copy, then you are well protected from ever losing your metadata changes.
IMO the original image file RAW or otherwise should never be accessed by any app other than the DAM tool that is managing it (LrC). When you create an export derivative file in Arc, you are not going to use the Metadata from the original file but you will use all of the metadata that you stored in the catalog file to create that export.
Remember, the DNG file specification is just a container like any other image file format. It can contain a RAW Photosite data block (as in the Leica camera file) and it always contains at least one RGB data block. DNG has been developed by Adobe because it is so flexible in regard to the data and data types that it can contain. Proprietary RAW filetypes are not flexible because the Header block containing the metadata is fixed is size and structure and the RAW data block characteristics are rigidly determined by the proprietary information contains in the proprietary RAW Header block.

Clee (Clete right?) - Thanks for the comment because you probably don't remember, but back in my early days of LR use over a decade ago, you helped nurse me through the growing pains. I've never liked DNG but like it even less now that I have to use it (because of Leica). Anyway, I hear you. I could turn off write to file once my images are off my laptop and onto my home PC. All further edits would be held in the catalog, of course. But old habits die hard. Your solution is the right one. That way, GoodSync won't have to rewrite the whole DNG file on every syn job after I make minor edits in LR.
You might remember my weird way of getting my files from the road laptop into my main LR catalog and home PC. I don't import the laptop Cat - just the raw files w sidecars and then import to the Cat. Maybe I'll try the standard way next time and work my way through the pollution to my folder names and levels that importing the catalog caused me the few times I tried it in the past. .

 

[GregJ]

"Experienced" has a variety of meanings

My summary would be:

• Back up DNGs when they are new + don't keep backing them up
• Back up your catalogue routinely
• 50-50 on enabling automatic saving
• Only update the preview for 3rd party requirements
• Occasionally run the DNG validation

Hey John, you are right. I'm very experienced in photography but in LR? I have probably used LR more than most people who use it but my knowledge is nowhere near any of the gurus, which is why I come here occasionally to ask questions. We get stuck in our workflows and sometimes aren't aware of better practices. I should hang out here more and read the threads and learn.
Question: You said "Update DNG previews.... updates the previews as well."
So it updates the 1:1 previews I generate at import? Or it updates the embedded jpeg? I think you said the embedded jpeg, not the preview.
By the way, when I generate the 1:1 previews at import, and then do further edits, the preview is still the base version and would not show further edits. I have always wondered about this. LR uses the previews ion both the library view and the development panel now. My understanding is that LR onlu used it in the libray panel previously and had to generate a different one when you went to the develop panel. So if the preview is of the original import of the raw, how does it show the edits without regenerating the 1:1 preview? If it does generate a new preview after every edit, what is the value of creating the original 1:1 preview to begin with? Does every edit update the preview? The embedded jpeg in the DNG is not updated unless you tell it to, as you explained. But the 1:1 preview in the .lrcat - when is that updated and when you edit files in the develop module, does it update the preview and then overwrite it on my disk?

You know what? I might delete all previews and stop generating them. My computer is so fast that it generates a 1:1 in about 2 seconds anyway.

Also, I wish DNGs and other raw formats had no embedded jpegs. And I wish all of my cameras produced no JPEGs and had zero jpeg functions on the menus. I wish what we saw in the live view and EVFs was not a generated jpeg but the actual raw file and an actual raw histogram. One can dream. The only jpegs I need are to export from the raw at full size in LR, post them to Flickr and then erase them. I don't keep jpegs on my disk ever.

 

[clee01l]

Clee (Clete right?) - Thanks for the comment because you probably don't remember, but back in my early days of LR use over a decade ago, you helped nurse me through the growing pains. I've never liked DNG but like it even less now that I have to use it (because of Leica). Anyway, I hear you. I could turn off write to file once my images are off my laptop and onto my home PC. All further edits would be held in the catalog, of course. But old habits die hard. Your solution is the right one. That way, GoodSync won't have to rewrite the whole DNG file on every syn job after I make minor edits in LR.
You might remember my weird way of getting my files from the road laptop into my main LR catalog and home PC. I don't import the laptop Cat - just the raw files w sidecars and then import to the Cat. Maybe I'll try the standard way next time and work my way through the pollution to my folder names and levels that importing the catalog caused me the few times I tried it in the past. .

Actually, “clee” is short login name for “C(letus) Lee”. It was the first Login ID I was ever assigned and I have used it where possible ever since. ‘Clete’ is a nick name for ‘Cletus’ and ‘Cletis’ and the feminine ‘Cleta’. I’ll answer to any of these.

I’m sorry I don’t remember our earlier exchanges.

For years I used a travel catalog as an import to my master catalog Not only does it transfer all of the edit adjustments, but it also includes Collection membership and all of the other things not contained in the XML file. For the last year or so, I have abandoned the Laptop as being too bulky to carry on trips. Opting instead on a 12.9” M1 iPadPro. Instead of using Lightroom Classic in the field, I now use the Cloud based Lightroom and for extra security, store my images on the Adobe Cloud in addition to my iPadPro. As long as I have a decent internet connection (most hotels), I get nightly sync back to my master catalog on my iMac running 7X24. It is not the perfect solution as Lightroom and Lightroom Classic treat Keywords differently. So I don’t do keywords in Lightroom but instead wait until the image has synced to the master Catalog and keyword on the iMac.

With Lightroom, I have the added advantage of creating Trip Albums to be shared and to sync with Adobe’s Portfolio.

I can see where your XML solution instead of a travel catalog can work for you. If you can live with the issues it presents then all is good. Like my Lightroom (cloud) solution, I can live with its shortcomings. At some point I hope that Lightroom is as robust as Lightroom Classic and can move entirely to Lightroom. Until this I am happy with my workflow.


Sent from my iPad using Tapatalk

 

[GregJ]

Thanks Cletus. Really interesting.

 

[johnbeardy]

Question: You said "Update DNG previews.... updates the previews as well."

Ah, you need to associate the "as well" with the paragraph immediately before that quote. It referred to saving the xmp metadata, so the Update DNG Previews updates the embedded previews as well as saving the xmp metadata to the files.

 

[GregJ]

Ah, you need to associate the "as well" with the paragraph immediately before that quote. It referred to saving the xmp metadata, so the Update DNG Previews updates the embedded previews as well as saving the xmp metadata to the files.

Thanks John. That is useful info.

 

[GregJ]

By the way ... not that I want them, but what are those badges everyone has? I have had the latest version of every Adobe product for the past 23 years. I want me some badges man! LOL.

 

[Hal P Anderson]

For the "Premium Member" badges, you need to buy the appropriate book from Victoria.

 

[GregJ]

For the "Premium Member" badges, you need to buy the appropriate book from Victoria.

I'll do it. It would be funny if her book was full of the DNG and preview stuff I've been asking. LOL.... I assume she keeps it updated on the latest releases.

 

[Hal P Anderson]

I assume she keeps it updated on the latest releases.

Oh, yeah!

 

[Victoria Bampton]

I'll do it. It would be funny if her book was full of the DNG and preview stuff I've been asking.

Yup, there's diagrams and everything! Most of the preview stuff is in the Improving Performance chapter of the Missing FAQ, and DNG is in The Geeky Bits chapter, and writings settings to the files is in Further Editing in Other Programs chapter.

eBooks are updated for release day for any release that has new features. Paperbacks go back to press every couple of years.

 

[BobT]

Yup, there's diagrams and everything! Most of the preview stuff is in the Improving Performance chapter of the Missing FAQ, and DNG is in The Geeky Bits chapter, and writings settings to the files is in Further Editing in Other Programs chapter.

eBooks are updated for release day for any release that has new features. Paperbacks go back to press every couple of years.

A picture is worth a thousand words. Maybe I'll get it now.

 

[HFogel]

Hi, I have a slightly different question, but still centered on DNG. So, I convert my RAW captures to DNGs, and find it works great. I'm an advanced user, so well-versed on the pros and cons. Ok, so most folks import RAW captures and convert to a Post=Production DNG, with all the benefits of that format.

However, I also shoot with Ricoh GR cameras that create native DNGs, which are a different beast, without the features of a Post-Production DNG. So, the question is, how does LR Classic deal with native DNGs? Does the "Automatically Write Changes into XMP" feature in Catalog Settings apply to native DNGs? Does it apply to Post-Production DNGs?

Do you have to run the "Update DNG Preview & Metadata” command to all native DNGs?

Finally, is there a way to easily convert native DNGs to Post-Production versions? I haven't found the "Convert to DNG" function to work on DNGs to do this.

Thanks!

 

[clee01l]

Hi, I have a slightly different question, but still centered on DNG. So, I convert my RAW captures to DNGs, and find it works great. I'm an advanced user, so well-versed on the pros and cons. Ok, so most folks import RAW captures and convert to a Post=Production DNG, with all the benefits of that format.

However, I also shoot with Ricoh GR cameras that create native DNGs, which are a different beast, without the features of a Post-Production DNG. So, the question is, how does LR Classic deal with native DNGs? Does the "Automatically Write Changes into XMP" feature in Catalog Settings apply to native DNGs? Does it apply to Post-Production DNGs?

Do you have to run the "Update DNG Preview & Metadata” command to all native DNGs?

Finally, is there a way to easily convert native DNGs to Post-Production versions? I haven't found the "Convert to DNG" function to work on DNGs to do this.

Thanks!

The DNG file format is a container file that describes its own contents. It is quite flexible. Converting proprietary RAW files to a DNG on import creates a DNG file that has RAW data in one of the main data blocks. The Ricoh camera creates a RAW file and stores the RAW photo site Data in one of the main data Blocks. RAW data is photo site values stored in a regular order in a data block whose contents have been delineated in the DNG file Header. This is the same whether the file is a Nikon NEF, Canon CR2 or a Ricoh, Pentax, or Leica DNG. In camera processors create the RGB view as seen on the camera back and also stored in another data block as a JPEG thumbnail. This is the image that you first see when you open the RAW file in any image viewer. The RAW data is not in image form as the photo site values are just numbers measuring the intensity of light at the photo site and not RGB pixels .
What you have termed Post Production is I surmise a processed image . Raw files no matter their origin have to be demosaic'd and converted to RGB pixels before any photos application can read them . In Lightroom, this is done using Adobe Camera RAW. Once the raw Photosite values has been converted to RGB pixels it can be manipulated by an image processor like Lightroom. Since Lightroom is classed as a non destructive editor, none of the changes it makes are ever applied to the original RAW file . To preserve the Lightroom processed image, the Lightroom process "Export" is used to create a derivative image. Most often the file format used to store this derivative RGB image is either a JPEG or TIFF. Other file formats are options in the Export process. One of these is the DNG format. When selected, the processed RGB image data is copied to a DNG file type container and the main data block contains these RGB pixel values. You could just as easily create a TIFF file on export and the main data block would contain the same RGB pixels. The reason is that the DNG file format is based upon the TIFF/EP6 standard just as the TIFF file specification. TIFF, DNG and PSDs are public file formats created and managed by Adobe.
To summarize, RAW files preserve and store the original photo site values. They can be stored in a proprietary file format or an open public file format. DNG is the open Public file format used. Not all DNG files contain RAW photo site values. The DNG file format is a container file that describes its own contents. If it describes the main data block as containing raw Photo site values, the DNG is a RAW file. If it describes the main data block as containing RGB Pixels, then the file is nor considered RAW and is subject to the same contraints as a JPEG, TIFF pro other file format containing RGB pixel data.

 

[GregJ]

The DNG file format is a container file that describes its own contents. It is quite flexible. Converting proprietary RAW files to a DNG on import creates a DNG file that has RAW data in one of the main data blocks. The Ricoh camera creates a RAW file and stores the RAW photo site Data in one of the main data Blocks. RAW data is photo site values stored in a regular order in a data block whose contents have been delineated in the DNG file Header. This is the same whether the file is a Nikon NEF, Canon CR2 or a Ricoh, Pentax, or Leica DNG. In camera processors create the RGB view as seen on the camera back and also stored in another data block as a JPEG thumbnail. This is the image that you first see when you open the RAW file in any image viewer. The RAW data is not in image form as the photo site values are just numbers measuring the intensity of light at the photo site and not RGB pixels .
What you have termed Post Production is I surmise a processed image . Raw files no matter their origin have to be demosaic'd and converted to RGB pixels before any photos application can read them . In Lightroom, this is done using Adobe Camera RAW. Once the raw Photosite values has been converted to RGB pixels it can be manipulated by an image processor like Lightroom. Since Lightroom is classed as a non destructive editor, none of the changes it makes are ever applied to the original RAW file . To preserve the Lightroom processed image, the Lightroom process "Export" is used to create a derivative image. Most often the file format used to store this derivative RGB image is either a JPEG or TIFF. Other file formats are options in the Export process. One of these is the DNG format. When selected, the processed RGB image data is copied to a DNG file type container and the main data block contains these RGB pixel values. You could just as easily create a TIFF file on export and the main data block would contain the same RGB pixels. The reason is that the DNG file format is based upon the TIFF/EP6 standard just as the TIFF file specification. TIFF, DNG and PSDs are public file formats created and managed by Adobe.
To summarize, RAW files preserve and store the original photo site values. They can be stored in a proprietary file format or an open public file format. DNG is the open Public file format used. Not all DNG files contain RAW photo site values. The DNG file format is a container file that describes its own contents. If it describes the main data block as containing raw Photo site values, the DNG is a RAW file. If it describes the main data block as containing RGB Pixels, then the file is nor considered RAW and is subject to the same contraints as a JPEG, TIFF pro other file format containing RGB pixel data.

Thanks Clete. I knew 95% of that, but the way you put it all together was really nice and educational for all. In fact, I'm going to copy it into my permanent camera notes. Great write-up
I remember a couple of years ago I asked Jim Kasson about this. He gave me a lecture similar to this when I asked about how Helicon Focus working with LR managed to take (for example) 90 of my raw files that I converted to DNG, focus stack them and produce another huge DNG file. He explained that the DNG file that Helicon produced from all that work it does was demosaice'd and thus different than the base DNG raw files that were input. It was no longer a true raw file as we think of if but was a demosaic'd DNG - really more like a TIFF file.
I probably screwed up that explanation and Jim would probably spank me for the way I explained that.
But you haven't lived unless you take a focus bracketed stack with a GFX 100 of (for example) 90 RAF raw images, then export to Helicon focus from LR the 90 Fuji RAF files as DNG - then Helicon Focus does its thing and sends one giant DNG file back to LR with everything in the frame in sharp, mind-numbing, glorious focus.
But I still say - The only thing I dislike about shooting a Leica Q2 is the fact that Leica uses DNG files (which means no sidecar files), and I do not like that.

 

[BobT]

The DNG file format is a container file that describes its own contents. It is quite flexible. Converting proprietary RAW files to a DNG on import creates a DNG file that has RAW data in one of the main data blocks. The Ricoh camera creates a RAW file and stores the RAW photo site Data in one of the main data Blocks. RAW data is photo site values stored in a regular order in a data block whose contents have been delineated in the DNG file Header. This is the same whether the file is a Nikon NEF, Canon CR2 or a Ricoh, Pentax, or Leica DNG. In camera processors create the RGB view as seen on the camera back and also stored in another data block as a JPEG thumbnail. This is the image that you first see when you open the RAW file in any image viewer. The RAW data is not in image form as the photo site values are just numbers measuring the intensity of light at the photo site and not RGB pixels .
What you have termed Post Production is I surmise a processed image . Raw files no matter their origin have to be demosaic'd and converted to RGB pixels before any photos application can read them . In Lightroom, this is done using Adobe Camera RAW. Once the raw Photosite values has been converted to RGB pixels it can be manipulated by an image processor like Lightroom. Since Lightroom is classed as a non destructive editor, none of the changes it makes are ever applied to the original RAW file . To preserve the Lightroom processed image, the Lightroom process "Export" is used to create a derivative image. Most often the file format used to store this derivative RGB image is either a JPEG or TIFF. Other file formats are options in the Export process. One of these is the DNG format. When selected, the processed RGB image data is copied to a DNG file type container and the main data block contains these RGB pixel values. You could just as easily create a TIFF file on export and the main data block would contain the same RGB pixels. The reason is that the DNG file format is based upon the TIFF/EP6 standard just as the TIFF file specification. TIFF, DNG and PSDs are public file formats created and managed by Adobe.
To summarize, RAW files preserve and store the original photo site values. They can be stored in a proprietary file format or an open public file format. DNG is the open Public file format used. Not all DNG files contain RAW photo site values. The DNG file format is a container file that describes its own contents. If it describes the main data block as containing raw Photo site values, the DNG is a RAW file. If it describes the main data block as containing RGB Pixels, then the file is nor considered RAW and is subject to the same contraints as a JPEG, TIFF pro other file format containing RGB pixel data.

Cletus,
I've been dutyfully converting all my RAWs to DNG thinking an open source file format has a better chance of surviving to posterity than a proprietary format. Now you tell me that DNG is just a container. Also, there are DNGs and DNGs depending on what camera produces them and to top it all off, a DNG is not even necessarily RAW. Have I been deluding myself? It makes one wonder then what is the purpose of an open source so called "file format" anyway?

 

[Rob_Cullen]

A DNG created /converted from a proprietary raw image file remains as raw.

A proprietary raw (eg. NEF CR3) might contain a structure like this-
IMAGE DATA + METADATA D,A,F,G,B,E,C.
The DNG converter simply re-arranges the metadata to be more compatible with a wider range of software-
IMAGE DATA + METADATA A,B,C,D,E,F,G.
Notice the difference?

OTOH, yes there is other software that places info into a DNG 'wrapper' that does not necessarily contain sensor 'raw' Image Data.

And to get very technical there is the PDF-
https://helpx.adobe.com/content/dam/help/en/photoshop/pdf/dng_spec_1_6_0_0.pdf