• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom 6 questions. All the information you need is in Adobe Lightroom 6 - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. When you upgrade to subscription, there's also a Lightroom Classic version available.
  • It's Lightroom update time again! Color Grading and improved zoom across the whole Lightroom family, plus graphical watermarks for the cloud-based Lightroom apps, as well as the usual new cameras, lens profiles and bug fixes. See this blog post for Lightroom Classic and this blog post for the Lightroom Cloud Ecosystem changes.

Yet another DNG question

Joined
Dec 7, 2007
Messages
2,135
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
I generally try to avoid posting questions about topics like this as the discussion often head into the direction of Coke vs. Pepsi, Canon vs. Nikon, Ford vs. Chevy, etc. I understand why folks do and do not care for DNG, and and I am fine with that as everybody needs to do what works best for their own work flow. Having said that, I am considering a move from DNG files back to raw files for use within Lightroom, but before making any decision, I would like to consider some of the pros and cons of switching as it relates to my particular work flow.

I presently maintain two backup copies of both my raw files as well as their DNG counterparts (without an embedded original). I have been doing this since I started shooting raw in 2006, but I suspect that I am not taking advantage of some of the features that DNG files offer. And, in light of an issue that I am currently having with v.8.6 of Adobe's stand-alone DNG converter, making a switch is now more of a possible reality than an consideration.

As I do not save my adjustments back to my files and rely on my catalog (and backups of my catalog) to maintain my post processing work, my present work flow really just relies on the base files being available to LR, in DNG or raw format. And, while DNG has offered the ability to verify files, that is also a feature that I have not taken advantage of to date. So, I am wondering what DNG features I will be giving up by importing raw files into LR instead of converted DNG's?

I know I will save drive space by not converting, but that is not really a big issue for me. On the other hand, my importing process will be substantially sped up as I will no longer need to convert files.

I do not have a dislike for DNG, so I am not looking for reasons to discontinue using it as much as I am wondering if I am taking advantage of what it offers for my work flow, and making sure I understand what features I would give up if I stopped converting files. Any advice would be greatly appreciated, and let's try to keep the fervor to a minimum as there is no right or wrong answer.

Thanks,

--Ken
 

tspear

Senior Member
Joined
Sep 23, 2014
Messages
1,799
Location
Waltham MA
Lightroom Experience
Beginner
Lightroom Version
The biggest issue for me would be the ability to of software to convert raw files. It is only a matter of time before Adobe and others start to drop early raw formats.
As for the rest of the benefits, there is know way to know without knowing what is your workflow.

Tim
 
Joined
Jun 20, 2009
Messages
16,604
Location
Houston, TX USA
Lightroom Experience
Power User
Lightroom Version
Cloud Service
The biggest issue for me would be the ability to of software to convert raw files. It is only a matter of time before Adobe and others start to drop early raw formats.
As for the rest of the benefits, there is know way to know without knowing what is your workflow.

Tim
The effort to develop a proprietary RAW conversion takes all of the resources, There is no resources required to continue to include the proprietary RAW format conversion in future versions of ACR. I think you will be hard pressed to name any proprietary RAW file format supported by ACR in the past that is no longer supported. Dennis is also correct that if support is ever dropped for a legacy Proprietary file format, there will be plenty of opportunity to convert to a supported file format so no data is lost. Proprietary file formats are supported by the camera maker and I think it is more likely that the camera mfg will develop advanced image processing technology unique to their RAW format. They won't be supporting DNG conversions. For that reason. the file to archive is the proprietary RAW (which LR reads fine btw). If you create a DNG conversion of the proprietary RAW file, then there are two formats that must be archived.

Like Ken, I like DNG. I wish the big name mfgs would adopt it as an in camera option like Pentax, Leica, Hasselblad and some others. Until that happens, I'm going to hold on to my NEFs and not slow down my imports creating DNGs that I need to manage along with the NEF.
 
Joined
Dec 7, 2007
Messages
2,135
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
I am familiar with Peter's work. That is what led me, in part, to LR and the DNG format. But, my current work flow does not really take advantage of many of DNG's features, and in light of the issue I am having with v.8.6 of the converter, I think a re-evaluation seems to be in order.

--Ken
 
Joined
Dec 7, 2007
Messages
2,135
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
...If you create a DNG conversion of the proprietary RAW file, then there are two formats that must be archived.

Like Ken, I like DNG. I wish the big name mfgs would adopt it as an in camera option like Pentax, Leica, Hasselblad and some others. Until that happens, I'm going to hold on to my NEFs and not slow down my imports creating DNGs that I need to manage along with the NEF.
I have been hedging my bets by archiving bot the raw file and the DNG, and in light of how Nikon has been treating older versions of Capture NX, it is not clear that either format is more secure than the other, and as long as there are conversion tools available, then we should be fine. Upon further thinking about one of my biggest reasons for continuing to use DNG, data validation, even seems to have diminished. If a raw file goes bad, I have two other copies of it, so it can be replaced. If they are also bad, then there really is not much that I can do about the situation, so I am not really sure how valuable that feature is for me. Given my workflow, I am more inclined to start importing raw files unless there is something else I am not considering that I should.

--Ken
 
Last edited:
Joined
Feb 1, 2010
Messages
12,520
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
Not trying to influence you in any way, Ken (like Cletus I don't use DNG and I also don't routinely save XMP)....but if the issue with the 8.6 DNG Converter is that develop icon thing from your recent other thread, have you tried the 8.7RC converter? I happened to try it yesterday for a colleague on another forum to test DNG conversion of a D750 file, and on import there was no develop icon on the thumbnail.
 

tspear

Senior Member
Joined
Sep 23, 2014
Messages
1,799
Location
Waltham MA
Lightroom Experience
Beginner
Lightroom Version
The effort to develop a proprietary RAW conversion takes all of the resources, There is no resources required to continue to include the proprietary RAW format conversion in future versions of ACR. I think you will be hard pressed to name any proprietary RAW file format supported by ACR in the past that is no longer supported. Dennis is also correct that if support is ever dropped for a legacy Proprietary file format, there will be plenty of opportunity to convert to a supported file format so no data is lost. Proprietary file formats are supported by the camera maker and I think it is more likely that the camera mfg will develop advanced image processing technology unique to their RAW format. They won't be supporting DNG conversions. For that reason. the file to archive is the proprietary RAW (which LR reads fine btw). If you create a DNG conversion of the proprietary RAW file, then there are two formats that must be archived.

Like Ken, I like DNG. I wish the big name mfgs would adopt it as an in camera option like Pentax, Leica, Hasselblad and some others. Until that happens, I'm going to hold on to my NEFs and not slow down my imports creating DNGs that I need to manage along with the NEF.
I had an early Pentax Camera which had a proprietary RAW and also output JPEG. Pentax was the only company which provided the software to read the raw files, and they ended support for the software with no notice and a Windows software update rendered the software unusable. So, I might be jaded because of this experience and many others outside of the digital photo area.
In addition, as a software developer, I can attest to the fact that maintaining mode for older formats does add to the project cost. Not a lot, but some. At some point, this will now longer be a trivial amount and Adobe will end support for older raw formats. When? I would guess when we have a core architecture change to either Windows or Mac. And you will notice that both are coming due for a core architecture change soon to address new hardware and interface requirements. As a result, I would guess somewhere within the next three to five years Adobe will start to drop support for older raw formats.

The reason I am not concerned about DNG being left behind is when you look at the core specification, it is designed as an extensible format. The result, a well written viewer for the 1.0 specification of DNG will kick out errors of information which is being ignored but is perfectly capable of reading the current specification. This kind of extensibility is not guaranteed or even implied with raw formats. In fact, from what I have read the Canon and Nikon formats have gone through many versions and derivations; yet each version still shares the same file extensions. So no software which only supports a single version can read another version of the same companies specification file.

As a result, I think DNG will be supported longer. But I do hedge my bets, I keep the raw on a backup and use DNG in the main photo repository for the working copy.

All else fails; you can also switch to rawtherpy or DarkTable to convert the oldest raw image formats, they will likely support older formats longer then Adobe.

Tim
 

tspear

Senior Member
Joined
Sep 23, 2014
Messages
1,799
Location
Waltham MA
Lightroom Experience
Beginner
Lightroom Version
I have been hedging my bets by archiving bot the raw file and the DNG, and in light of how Nikon has been treating older versions of Capture NX, it is not clear that either format is more secure than the other, and as long as there are conversion tools available, then we should be fine. Upon further thinking about one of my biggest reasons for continuing to use DNG, data validation, even seem to have diminished. If a raw files goes bad, I have two other copies of it, so it can be replaced. If they are also bad, then there really is not much that I can do about the situation, so I am not really sure how valuable that feature is for me. Given my workflow, I am more inclined to start importing raw files unless there is something else I am not considering that I should.

--Ken
Ken,

There are a few aspects where data validation comes into play.
One is at the hardware level, second is the OS, third is the application. Each layer builds certain assumptions into how the validation and guarantees are performed. As long as the computer runs perfectly and never crashes, and Lightroom never makes a mistake, these assumptions and validations will hold and you never have an issue.

Every time an application opens a file for read/write access there is a chance of corruption. The goal of each layer I listed above is to reduce the chance of corruption in case of failure at any level. That is why DNG incorporates the validation.
For me, I have my catalog and images replicated to the "cloud" for access on my other computers and also my backup software. The result is I have many processes which touch the files, in theory the vast majority after initial import should be read only and should never corrupt the file, but you never know. As such the built in validation in DNG should allow me to find problems sooner then later and also allow me to recover from the problem easier before it affects more files.

Tim
 
Joined
Dec 7, 2007
Messages
2,135
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
Not trying to influence you in any way, Ken (like Cletus I don't use DNG and I also don't routinely save XMP)....but if the issue with the 8.6 DNG Converter is that develop icon thing from your recent other thread, have you tried the 8.7RC converter? I happened to try it yesterday for a colleague on another forum to test DNG conversion of a D750 file, and on import there was no develop icon on the thumbnail.
Hi Jim,

I had switched back to 8.3 (as it was an earlier version that supported my cameras) before I saw that 8.7 was available as an RC. I have made no decision yet on changing, and will most likely wait until 8.7 is a full release and try it out. In the mean time, I spent the evening correcting many an image. I believe that all is now "right" in my main catalog. Your help was very key, so I want to thank you again!

--Ken
 
Joined
Dec 7, 2007
Messages
2,135
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
As such the built in validation in DNG should allow me to find problems sooner then later and also allow me to recover from the problem easier before it affects more files.

Tim
Hi Tim,

This is true, and I am still considering it as a useful feature. I just need to get a better handle on the time commitment to validating images, so I can determine how often a validation can be run.

Thanks,

--Ken
 
Joined
Jun 20, 2009
Messages
16,604
Location
Houston, TX USA
Lightroom Experience
Power User
Lightroom Version
Cloud Service
I'm not sure what Pentax you had, but When I started with the D10 Pentax had both PEF and DNG as an in camera file format options. And if Pentax no longer supports software for the legacy proprietary file format, I'll bet that ACR will still read it in version 8.7. Pentax did not develop in house post processing capability. Instead, they used Ichikawa soft laboratory Co., Ltd. (SilkyPix) to provide software to support the Pentax RAW post processing requirements. Until recently, Nikon used Nix Software to produce Capture NX. When Nix Software was purchased by Google the relations ship was dissolved and Nikon had to replace CaptureNX2 with CaptureNX-D which perhaps not coincidently is developed by Ichikawa soft laboratory Co., Ltd.

I base my statement that the programmer cost to continue support for legacy proprietary file formats is trivial because I too was a program developer.
CR2, NEF and perhaps other proprietary RAW formats are based upon the TIFF/EP6 standard, the same extensible standard as DNG.

RAW Therapee & DarkTable do not have deep pockets like Adobe. Which do you think will survive the longest? And why would you think they would have resources to support legacy RAW formats should Adobe abandon them? RAW Therapee & DarkTable in terms of Software development studios are very similar to BibblePro. BibblePro was an excellent product. RAW image rendering was on par with ACR. However it suffered from lack of development resources and was hard pressed to put out timely RAW format updates to support new camera releases. Bibble Labs was acquired by Corel in 2012. Being acquired is probably the ultimate fate of RAW Therapee & DarkTable.

As Denis de Gannes, has astutely pointed out, there will be plenty of time to convert to DNG if proprietary RAW support for any format is discontinued.
 
Joined
Sep 29, 2007
Messages
22,669
Location
Isle of Wight, UK
Lightroom Experience
Power User
Lightroom Version
Cloud Service
Upon further thinking about one of my biggest reasons for continuing to use DNG, data validation, even seem to have diminished. If a raw files goes bad, I have two other copies of it, so it can be replaced.
Data validation is one of the reasons I use DNG too. Of course you can replace a corrupted file with one of your other copies, but how would you find out it was corrupted?

In my case, DNG files not validating acted as the canary in the coal mine, letting me know that my drive was dying. That meant I had time to pull the rest of my data off that drive before it became corrupted. Had it not been for the DNG validation failing, some of my other files could have become corrupted, and my working backups would also have eventually been updated with corrupted copies.
 
Joined
Dec 7, 2007
Messages
2,135
Location
Puget Sound
Lightroom Experience
Intermediate
Lightroom Version
Classic
Data validation is one of the reasons I use DNG too. Of course you can replace a corrupted file with one of your other copies, but how would you find out it was corrupted?

In my case, DNG files not validating acted as the canary in the coal mine, letting me know that my drive was dying. That meant I had time to pull the rest of my data off that drive before it became corrupted. Had it not been for the DNG validation failing, some of my other files could have become corrupted, and my working backups would also have eventually been updated with corrupted copies.
I has similar thoughts after I wrote this, Victoria. My concern is how long it takes LR to run a validation check on DNG files. Is this a quick process, or something that takes hours? If it is the latter, then I suspect that it may not be run often due to time constraints. Any experience with validating catalogs that you want to share?

--Ken
 

tspear

Senior Member
Joined
Sep 23, 2014
Messages
1,799
Location
Waltham MA
Lightroom Experience
Beginner
Lightroom Version
clee,

Pentax Optio 430. Man that was a long time ago in the digital world. A business acquaintance got a few of them prerelease, the final release only supported JPEG . So maybe my expectations were misplaced in terms of support :D

By RT and DT, I was commenting on the fact you will more likely be able to have access to conversion tools if you get left in a lurch by a commercial vendor. Not that they would be able to match the Adobe support.

Tim
 
Joined
Sep 29, 2007
Messages
22,669
Location
Isle of Wight, UK
Lightroom Experience
Power User
Lightroom Version
Cloud Service
It does take a while, depending on the number of photos. I run it on all photos before I update my offline backup set every 6 months or so, and just do recent photos more frequently. I leave it running overnight, so it doesn't interfere with anything.
 

tspear

Senior Member
Joined
Sep 23, 2014
Messages
1,799
Location
Waltham MA
Lightroom Experience
Beginner
Lightroom Version
I just ran it on my Two year old MBP. SSD drive, 16GB Ram 2.8Ghz I7.
Just under 14000 DNG files. 7 Minutes to validate. So about 2000 DNG files a minute on my machine.

Tim
 
Top