• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Yet another DNG question

Status
Not open for further replies.

Replytoken

Senior Member
Lightroom Guru
Premium Classic Member
Premium Cloud Member
Joined
Dec 7, 2007
Messages
3,066
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
I generally try to avoid posting questions about topics like this as the discussion often head into the direction of Coke vs. Pepsi, Canon vs. Nikon, Ford vs. Chevy, etc. I understand why folks do and do not care for DNG, and and I am fine with that as everybody needs to do what works best for their own work flow. Having said that, I am considering a move from DNG files back to raw files for use within Lightroom, but before making any decision, I would like to consider some of the pros and cons of switching as it relates to my particular work flow.

I presently maintain two backup copies of both my raw files as well as their DNG counterparts (without an embedded original). I have been doing this since I started shooting raw in 2006, but I suspect that I am not taking advantage of some of the features that DNG files offer. And, in light of an issue that I am currently having with v.8.6 of Adobe's stand-alone DNG converter, making a switch is now more of a possible reality than an consideration.

As I do not save my adjustments back to my files and rely on my catalog (and backups of my catalog) to maintain my post processing work, my present work flow really just relies on the base files being available to LR, in DNG or raw format. And, while DNG has offered the ability to verify files, that is also a feature that I have not taken advantage of to date. So, I am wondering what DNG features I will be giving up by importing raw files into LR instead of converted DNG's?

I know I will save drive space by not converting, but that is not really a big issue for me. On the other hand, my importing process will be substantially sped up as I will no longer need to convert files.

I do not have a dislike for DNG, so I am not looking for reasons to discontinue using it as much as I am wondering if I am taking advantage of what it offers for my work flow, and making sure I understand what features I would give up if I stopped converting files. Any advice would be greatly appreciated, and let's try to keep the fervor to a minimum as there is no right or wrong answer.

Thanks,

--Ken
 
The biggest issue for me would be the ability to of software to convert raw files. It is only a matter of time before Adobe and others start to drop early raw formats.
As for the rest of the benefits, there is know way to know without knowing what is your workflow.

Tim
 
The biggest issue for me would be the ability to of software to convert raw files. It is only a matter of time before Adobe and others start to drop early raw formats.
As for the rest of the benefits, there is know way to know without knowing what is your workflow.

Tim
The effort to develop a proprietary RAW conversion takes all of the resources, There is no resources required to continue to include the proprietary RAW format conversion in future versions of ACR. I think you will be hard pressed to name any proprietary RAW file format supported by ACR in the past that is no longer supported. Dennis is also correct that if support is ever dropped for a legacy Proprietary file format, there will be plenty of opportunity to convert to a supported file format so no data is lost. Proprietary file formats are supported by the camera maker and I think it is more likely that the camera mfg will develop advanced image processing technology unique to their RAW format. They won't be supporting DNG conversions. For that reason. the file to archive is the proprietary RAW (which LR reads fine btw). If you create a DNG conversion of the proprietary RAW file, then there are two formats that must be archived.

Like Ken, I like DNG. I wish the big name mfgs would adopt it as an in camera option like Pentax, Leica, Hasselblad and some others. Until that happens, I'm going to hold on to my NEFs and not slow down my imports creating DNGs that I need to manage along with the NEF.
 

I am familiar with Peter's work. That is what led me, in part, to LR and the DNG format. But, my current work flow does not really take advantage of many of DNG's features, and in light of the issue I am having with v.8.6 of the converter, I think a re-evaluation seems to be in order.

--Ken
 
...If you create a DNG conversion of the proprietary RAW file, then there are two formats that must be archived.

Like Ken, I like DNG. I wish the big name mfgs would adopt it as an in camera option like Pentax, Leica, Hasselblad and some others. Until that happens, I'm going to hold on to my NEFs and not slow down my imports creating DNGs that I need to manage along with the NEF.

I have been hedging my bets by archiving bot the raw file and the DNG, and in light of how Nikon has been treating older versions of Capture NX, it is not clear that either format is more secure than the other, and as long as there are conversion tools available, then we should be fine. Upon further thinking about one of my biggest reasons for continuing to use DNG, data validation, even seems to have diminished. If a raw file goes bad, I have two other copies of it, so it can be replaced. If they are also bad, then there really is not much that I can do about the situation, so I am not really sure how valuable that feature is for me. Given my workflow, I am more inclined to start importing raw files unless there is something else I am not considering that I should.

--Ken
 
Last edited:
Not trying to influence you in any way, Ken (like Cletus I don't use DNG and I also don't routinely save XMP)....but if the issue with the 8.6 DNG Converter is that develop icon thing from your recent other thread, have you tried the 8.7RC converter? I happened to try it yesterday for a colleague on another forum to test DNG conversion of a D750 file, and on import there was no develop icon on the thumbnail.
 
The effort to develop a proprietary RAW conversion takes all of the resources, There is no resources required to continue to include the proprietary RAW format conversion in future versions of ACR. I think you will be hard pressed to name any proprietary RAW file format supported by ACR in the past that is no longer supported. Dennis is also correct that if support is ever dropped for a legacy Proprietary file format, there will be plenty of opportunity to convert to a supported file format so no data is lost. Proprietary file formats are supported by the camera maker and I think it is more likely that the camera mfg will develop advanced image processing technology unique to their RAW format. They won't be supporting DNG conversions. For that reason. the file to archive is the proprietary RAW (which LR reads fine btw). If you create a DNG conversion of the proprietary RAW file, then there are two formats that must be archived.

Like Ken, I like DNG. I wish the big name mfgs would adopt it as an in camera option like Pentax, Leica, Hasselblad and some others. Until that happens, I'm going to hold on to my NEFs and not slow down my imports creating DNGs that I need to manage along with the NEF.

I had an early Pentax Camera which had a proprietary RAW and also output JPEG. Pentax was the only company which provided the software to read the raw files, and they ended support for the software with no notice and a Windows software update rendered the software unusable. So, I might be jaded because of this experience and many others outside of the digital photo area.
In addition, as a software developer, I can attest to the fact that maintaining mode for older formats does add to the project cost. Not a lot, but some. At some point, this will now longer be a trivial amount and Adobe will end support for older raw formats. When? I would guess when we have a core architecture change to either Windows or Mac. And you will notice that both are coming due for a core architecture change soon to address new hardware and interface requirements. As a result, I would guess somewhere within the next three to five years Adobe will start to drop support for older raw formats.

The reason I am not concerned about DNG being left behind is when you look at the core specification, it is designed as an extensible format. The result, a well written viewer for the 1.0 specification of DNG will kick out errors of information which is being ignored but is perfectly capable of reading the current specification. This kind of extensibility is not guaranteed or even implied with raw formats. In fact, from what I have read the Canon and Nikon formats have gone through many versions and derivations; yet each version still shares the same file extensions. So no software which only supports a single version can read another version of the same companies specification file.

As a result, I think DNG will be supported longer. But I do hedge my bets, I keep the raw on a backup and use DNG in the main photo repository for the working copy.

All else fails; you can also switch to rawtherpy or DarkTable to convert the oldest raw image formats, they will likely support older formats longer then Adobe.

Tim
 
I have been hedging my bets by archiving bot the raw file and the DNG, and in light of how Nikon has been treating older versions of Capture NX, it is not clear that either format is more secure than the other, and as long as there are conversion tools available, then we should be fine. Upon further thinking about one of my biggest reasons for continuing to use DNG, data validation, even seem to have diminished. If a raw files goes bad, I have two other copies of it, so it can be replaced. If they are also bad, then there really is not much that I can do about the situation, so I am not really sure how valuable that feature is for me. Given my workflow, I am more inclined to start importing raw files unless there is something else I am not considering that I should.

--Ken

Ken,

There are a few aspects where data validation comes into play.
One is at the hardware level, second is the OS, third is the application. Each layer builds certain assumptions into how the validation and guarantees are performed. As long as the computer runs perfectly and never crashes, and Lightroom never makes a mistake, these assumptions and validations will hold and you never have an issue.

Every time an application opens a file for read/write access there is a chance of corruption. The goal of each layer I listed above is to reduce the chance of corruption in case of failure at any level. That is why DNG incorporates the validation.
For me, I have my catalog and images replicated to the "cloud" for access on my other computers and also my backup software. The result is I have many processes which touch the files, in theory the vast majority after initial import should be read only and should never corrupt the file, but you never know. As such the built in validation in DNG should allow me to find problems sooner then later and also allow me to recover from the problem easier before it affects more files.

Tim
 
Not trying to influence you in any way, Ken (like Cletus I don't use DNG and I also don't routinely save XMP)....but if the issue with the 8.6 DNG Converter is that develop icon thing from your recent other thread, have you tried the 8.7RC converter? I happened to try it yesterday for a colleague on another forum to test DNG conversion of a D750 file, and on import there was no develop icon on the thumbnail.

Hi Jim,

I had switched back to 8.3 (as it was an earlier version that supported my cameras) before I saw that 8.7 was available as an RC. I have made no decision yet on changing, and will most likely wait until 8.7 is a full release and try it out. In the mean time, I spent the evening correcting many an image. I believe that all is now "right" in my main catalog. Your help was very key, so I want to thank you again!

--Ken
 
As such the built in validation in DNG should allow me to find problems sooner then later and also allow me to recover from the problem easier before it affects more files.

Tim

Hi Tim,

This is true, and I am still considering it as a useful feature. I just need to get a better handle on the time commitment to validating images, so I can determine how often a validation can be run.

Thanks,

--Ken
 
I'm not sure what Pentax you had, but When I started with the D10 Pentax had both PEF and DNG as an in camera file format options. And if Pentax no longer supports software for the legacy proprietary file format, I'll bet that ACR will still read it in version 8.7. Pentax did not develop in house post processing capability. Instead, they used Ichikawa soft laboratory Co., Ltd. (SilkyPix) to provide software to support the Pentax RAW post processing requirements. Until recently, Nikon used Nix Software to produce Capture NX. When Nix Software was purchased by Google the relations ship was dissolved and Nikon had to replace CaptureNX2 with CaptureNX-D which perhaps not coincidently is developed by Ichikawa soft laboratory Co., Ltd.

I base my statement that the programmer cost to continue support for legacy proprietary file formats is trivial because I too was a program developer.
CR2, NEF and perhaps other proprietary RAW formats are based upon the TIFF/EP6 standard, the same extensible standard as DNG.

RAW Therapee & DarkTable do not have deep pockets like Adobe. Which do you think will survive the longest? And why would you think they would have resources to support legacy RAW formats should Adobe abandon them? RAW Therapee & DarkTable in terms of Software development studios are very similar to BibblePro. BibblePro was an excellent product. RAW image rendering was on par with ACR. However it suffered from lack of development resources and was hard pressed to put out timely RAW format updates to support new camera releases. Bibble Labs was acquired by Corel in 2012. Being acquired is probably the ultimate fate of RAW Therapee & DarkTable.

As Denis de Gannes, has astutely pointed out, there will be plenty of time to convert to DNG if proprietary RAW support for any format is discontinued.
 
Upon further thinking about one of my biggest reasons for continuing to use DNG, data validation, even seem to have diminished. If a raw files goes bad, I have two other copies of it, so it can be replaced.

Data validation is one of the reasons I use DNG too. Of course you can replace a corrupted file with one of your other copies, but how would you find out it was corrupted?

In my case, DNG files not validating acted as the canary in the coal mine, letting me know that my drive was dying. That meant I had time to pull the rest of my data off that drive before it became corrupted. Had it not been for the DNG validation failing, some of my other files could have become corrupted, and my working backups would also have eventually been updated with corrupted copies.
 
Data validation is one of the reasons I use DNG too. Of course you can replace a corrupted file with one of your other copies, but how would you find out it was corrupted?

In my case, DNG files not validating acted as the canary in the coal mine, letting me know that my drive was dying. That meant I had time to pull the rest of my data off that drive before it became corrupted. Had it not been for the DNG validation failing, some of my other files could have become corrupted, and my working backups would also have eventually been updated with corrupted copies.

I has similar thoughts after I wrote this, Victoria. My concern is how long it takes LR to run a validation check on DNG files. Is this a quick process, or something that takes hours? If it is the latter, then I suspect that it may not be run often due to time constraints. Any experience with validating catalogs that you want to share?

--Ken
 
clee,

Pentax Optio 430. Man that was a long time ago in the digital world. A business acquaintance got a few of them prerelease, the final release only supported JPEG . So maybe my expectations were misplaced in terms of support :D

By RT and DT, I was commenting on the fact you will more likely be able to have access to conversion tools if you get left in a lurch by a commercial vendor. Not that they would be able to match the Adobe support.

Tim
 
It does take a while, depending on the number of photos. I run it on all photos before I update my offline backup set every 6 months or so, and just do recent photos more frequently. I leave it running overnight, so it doesn't interfere with anything.
 
I just ran it on my Two year old MBP. SSD drive, 16GB Ram 2.8Ghz I7.
Just under 14000 DNG files. 7 Minutes to validate. So about 2000 DNG files a minute on my machine.

Tim
 
Status
Not open for further replies.
Back
Top