• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

Lightroom STILL doesn't make images unique

Status
Not open for further replies.

dkperez

Active Member
Joined
Mar 6, 2008
Messages
148
Lightroom Experience
Intermediate
Lightroom Version
Lightroom Version Number
9.4
Operating System
  1. Windows 10
I've asked this over in the Lightroom Forum in the past, and I"ve never gotten a useful answer....

I take a set of images and run them through Helicon Focus. I render them with a specific setting and get back a .dng file with a unique name. I render a second time with a DIFFERENT set of parameters and get bad a DIFFERENT .dng file with a unique name. They're not the same, they don't look the same.

In Lightroom, in Library or in Develop, they show up THE SAME. At 1:1 they show up the same. Does anyone here have a way to make Lightroom display them as the actually are? It doesn't happen when I save as jpegs, and it doesn't happen when I save as tiffs, but with dng files I keep having to open them in Adobe Bridge to see the differences...
 
FWIW I am on Windows 10 Pro also. Not to mention I do not see how that would affect what LR sees in loupe.

I'm sorry. I am at a loss.
I have suggested earlier that somewhere in Windows there is an image caching offering. I have no proof for this except that I know that this occurs in the browsers to speed loading of the same file over and over.
 
I"ve considered STAMP COLLECTING instead of macrophotography!

No smart previews.

Yes, I’ve exported from Helicon and move them so Lightroom couldn’t import them, then imported them. Still the same.

I don’t know of any way to stop between Helicon and Lightroom other than to save the Helicon images and move them so Lightroom can’t import them.

Opening the images in Photoshop before I let Lightroom import them shows them as different in Photoshop. I haven’t seen a way to save a .dng file from Photoshop?

I’ve already done the “blow away the whole catalog and create a new one” thing… Didn’t matter.

But, at this point I have two people who use Helicon Focus, and three computers that are all displaying very different images as identical.
 
I have suggested earlier that somewhere in Windows there is an image caching offering. I have no proof for this except that I know that this occurs in the browsers to speed loading of the same file over and over.

But one that LR would be using in Develop? Reusing despite all the hoops that have been jumped through?

Opening the images in Photoshop before I let Lightroom import them shows them as different in Photoshop. I haven’t seen a way to save a .dng file from Photoshop?
Sorry about that, I just thought it could and did not try, just tried and you are of course correct.

I actually had Helicon on my system, I thought it had expired, but it says 30 days left. I rendered three ways, so I had three at the bottom, and did a Save All (is that how you are doing it?). It offered to save as TIFF. I switched to DNG and it says "Some of the results3) cannot be saved into DNG format". I changed to TIFF and specified a folder (different) and got an "Unable to Export" that said "The follow files were not imported because they could not be read or the destination folder is not writeable".

The three TIFFs were in the folder I specified, I ADDed them with an import session, and they are three different photos.

How are you outputting DNG's?
 
Just by accident I was reading a book at breakfast after your first note today and the quotation in it applies:

"When you have exhausted all possibilities, remember this: You haven't" - Thomas Edison
 
I can't follow all the twists and turns of this, but a good general philosophy for problem solving is to divide the universe of possibilities into two roughly equal-sized chunks and rule one chunk out. Progressive iterations narrow down the problem.

Is it correct that the problem lies in how Helicon Focus communicates its output to LR? If that's not correct, can you rule it out? I don't use Helicon, so pardon me if the following is naive.

1. Have you tried re-installing Helicon Focus?
2. Does Helicon have options for outputting its files? If so, have you fiddled with them?
3. Is it possible that Helicon is somehow attaching an image, such as a preview, to its output file?
4. If your real goal is to produce focus-stacked images, is there a reason not to give Zerene Stacker a try? Some problems aren't worth solving, and you may be getting into this territory with Helicon.
 
5. As I understand it, several Helicon output files that are known to be different (e.g. by viewing in Photoshop) look the same in LR. What happens if you go into Develop and make some kind of minor change, such as a small tweak to texture or vibrance or clarity? I gather from previous threads on this forum that you may be presented with a preview when first opening a file, but even a slight change will cause LR to render the image anew.
 
1. Have you tried re-installing Helicon Focus?
2. Does Helicon have options for outputting its files? If so, have you fiddled with them?
3. Is it possible that Helicon is somehow attaching an image, such as a preview, to its output file?
As others have checked the files and do see the differences, I do not see how any of these three suggestions could solve the problem.
 
1 Some time back I had a problem very like yours. Somehow LR was looking at the same image, I was convinced I was looking at what should have been different versions, but they all looked the same. From my somewhat unreliable memory, it was my fault, I got LR/me confused (rename the files to be very different) 2 LR's caching is less than perfect and sometimes (rarely) it doesn't update as quickly as it should do (minutes of displaying the wrong thing, LR needs 'poking' eg change the zoom ratio). What have you got for your graphics card acceleration? Which ever it is, turn it the other way. 3 The third observation is that you mention that the LR display changes from a correct different view to the same view. That is utterly typical of LR loading a jpeg thumbnail that then gets replaced with the LR rendered version. If that is happening, it suggests that in some way LR is ignoring the processed file and is reverting to using the raw original. The DNG format isn't so much a single data format, but is a container for different formats, including the possibility of both the original raw file and other versions of it (at the same time). How big is the DNG compared to the original NEF file? Given that others are seeing the differences and you are not, I suggest the first two points to look at, the third one is clutching at straws but deleting your preferences (just rename the prefs file and restart LR as an experiment) might be worth trying.
 
Last edited:
I don't know if this will clarify or muddy the waters further. I was curious to see whether the photos look the same on my computer (Win 10, running LrC 9.2.1) or different as most folks have reported. Sure enough, they looked somewhat different. But then I noticed that a develop preset I often use had been applied, even though I didn't apply it on import. Eventually I realized that in my Preferences file I told LR to apply this preset to RAW files. When I changed that to "Adobe Default" and imported the photos to a different folder, they all look the same to me.
 
Edit to add: I checked whether the photos would look different if I applied my usual preset to them. One of them looked different; then I removed the preset and the changes persisted. When I looked at the other two, I got what I think is the OP's experience, namely that the proper photo would appear for a second or two, then the altered version of the previous one replaced it. But now when I go back to verify this behavior, it has gone away and the unadjusted version appears. Further testing seems to give variable behavior; sometimes the edited version sticks, other times it reverts.

I also see that sometimes when I apply this preset to one of the images, the image looks very strange with haloes around the stems; usually it doesn't have that effect.

I don't understand it, but it makes me wonder whether there is something strange about these files, which is why I suggested re-installing Helicon before.
 
John, welcome to my world!!!!!!!!!! Bizarre.

BTW: I now have TWO other people who are running Windows 10 Pro on PCs with very different configurations, and both are seeing exactly what I do when the .dng files are displayed in loupe view. One of the two ALSO does focus stacking with a completely different tool for creating the stacks and a different version of Helicon Focus for rendering. He downloaded my 3 images and found them to identical in both loupe and Develop, then ran one of HIS stacks through HF, generated multiple different outputs, ran multiple HF sessions, and even did some other fiddling with output images. Every time, his outputs looked IDENTICAL... I'm not sure what's different between the 4 systems we're using and what y'all are doing, but he contacted Helicon Support and THEY have now reproduced the problem in Lightroom.

Needless to say, their first response, and the one that continues to today, is - it's not their fault.. It's Lightroom.

Yesterday morning I opened two .dng files in Word. You can read the “header” xmp stuff. I took a 19 image stack and a 7 image subset and did a file compare on the xmp part. They were done in TWO different sessions so I’d expect SOME differences. The ONLY three fields that differed were the date at the top, the xmp:ModifyDate and the xmp:MetadataDate. One is 19:44:09, the other is 19:45:30.

As near as I (and the compare) can tell, EVERYTHING ELSE in there is identical. In Lightroom they look the same.

I took a .NEF file, in Lightroom, changed the exposure to about -3 and exported as a .dng. Changed the exposure to about +4 and exported a .dng. Different names. Opened THOSE and compared:
The three dates are different (expected)… BUT, there are OTHER differences:
xmpMM:DocumentID
xmpMM:InstanceID
crs:Exposure2012
stEvt:InstanceID
stEvt:when

And when imported into Lightroom, they look VERY different.
The Exposure is -3.xx vs +4.xx so crs:Exposure2012 isn’t important. I suspect stEvt:when isn’t either.
xmpMM:DocumentID, xmpMM:InstanceID and stEvt:InstanceID ALL CONTAIN THE SAME VALUE.

Back to Helicon…

I ran a NEW stack. Ran 16 images in method B, then ran the first 4 images in method B. Saved each as it was created. Exited from Helicon then exported the first 8 images from Lightroom and ran them method B, and saved. So, it’s a different time, a different number of images, and all the filenames are different.

In Lightroom loupe, ALL THREE images look identical. Comparing the .dng files, the 16 image run and the 4 image subset are identical – as near as I can tell there are NO fields in the xmp text that differ. There’s nothing I know of to tell Lightroom these are anything but two identical files with different names. PRESUMING Lightroom is using something in this xmp data to decide what constitutes a "different" image.

Then I compared the 16 image and the SEPARATE 8 image subset renders. This time the three dates differed, and the xmpMM:DocumentID, xmpMM:InstanceID, stEvt:InstanceID, stEvt:when differed as they did in the two .dng files I created from the .NEF. Further down in the xmpMM:History section there are repeated occurrences of the stEvt:instanceID that repeat earlier values, but there are TWO different values of stEvt:instanceID in various places in the text… All differ from the values in the stack of the 8 image subset.

So, there is data in the .dng that SEEMS to be unique when I run a SEPARATE session, yet in Lightroom, ALL 3 .dng files, 16 images, 4 image subset, 8 image subset, all look the same.

What DOES appear to be the same in all of the .dng images from these runs is the xmpMM:OriginalDocumentID, and in the “xmpMMDerivedFrom” section, the stRef:documentID, and the stRef:eek:riginalDocumentID, which match the xmpMM:OriginalDocumentID earlier in the text.

What is LIGHTROOM using to decide there are two different files? If it’s InstanceID, that only appears to change if a different Helicon Focus session is run. But, if it’s stRef:documentID or stRef:eek:riginalDocumentID, those are the same in all 3 .dng files.

I took a subset of images from the middle of the stack, and ran those. All the above differ, PLUS the crs:RawFileName, aux:ImageNumber and the stRef:documentID, and stRef:eek:riginalDocumentID that were the same previously. This render output, that started with an image in the middle of the stack rather than the first image in the stack, IS DIFFERENT in Lightroom...

From all this fiddling around, I have more data, but have no idea what is and isn’t meaningful. The question “How does Lightroom determine two .dng files are the same (or different)?” hasn’t been answered. As far as I can tell, Helicon doesn’t do it’s own .dng conversion. It uses the Adobe .dng converter – when you install HF, it appears to install an Adobe .dng converter. So, is it Lightroom, Adobe dng converter, or Helicon that’s not doing what’s needed to make Lightroom treat the .dng files as unique?
 
If you have tried this my apologies but...

It sounds like with experience you know what to expect. Specifically, if you had images A, B and C can you predict in advance which image will be the one displayed in LR for all three? Let's pretend it is going to be B.

If you can, can you interfere with the import on exit of Helicon so that B cannot be imported at all? I.e. so that LR never even gets to see the image that otherwise would be used on all three?

What do you see then?

Because if you then see B, it seems to imply that the DNG's all somehow, somewhere contain image B.

I'm starting to wonder if the DNG's are more complicated than we think, and Helicon is putting all the data for all the images in all three.

By the way... I still don't see how you are getting DNG's from Helicon. I installed it and I get TIFF's back in Lightroom. How are you telling it to return DNG's?
 
Here are a few more factoids that might help.
1. I'm attaching a screenshot of the altered image I sometimes (seemingly at random) get when I try various things with the images. It looks to me like the first or a very early image in the image stack--the focus in the front. This implies that at least one image from the stack is included in the .dng file.
2. One time when I started LrC (9.2.1) I got a message along the lines of "This file appears to be damaged" for the 19C,S2 file. Next time I opened LR and looked at the same file, I didn't get that message. Huh?
3. When I downleaded 19C, S2 a second time and imported into LR, then tested if Lens Corrections changed it, I got a message along the lines of "Do you want to save the altered metadata to the file?" I've never seen this message before in LR. I answered "no", and the message didn't reappear for this or any of the other files as I fiddled with them.

I can only conclude, based on the inconsistencies I'm seeing, is that LR is confused by the contents of these files and can't interpret them in a consistent way.
FirstLayer.JPG
 
Hello,

I just happened to look here for some othe rinfo and spotted this thread, so I thought I best sign up as I'm pretty sure I know what the underlying problem is.

I had a similar problem a few years ago with a Leica M240 whose DNG previews kept occasionally showing an old image instead of the expected one. Eventually I tracked it down to a bug in the M240 DNG implementation.

What happens with DNG previews in Lightroom and Photoshop is that they index their previews in the catalogue by use of a couple EXIF tags. For Lightroom, the tag it uses is the RawDataUniqueID (and if I fremember correctkly Bridge uses the alternative ImageUniqueID). Importantly, it doesn't use normal file attributes such as name or size. The DNG spec requires any application that generates a DNG to insert a random, unique, ID string into the RawDataUniqueID field.

I downloaded your 3 example DNG files, and they have the same RawDataUniqueID value:

Raw Data Unique ID : 000000000000000005EF2024231F8015
Raw Data Unique ID : 000000000000000005EF2024231F8015
Raw Data Unique ID : 000000000000000005EF2024231F8015

This is why I believe you are seeing the same previews - if its any consolation this drove me nuts with the M240!

If you use an Exif editor and make these unique, you should find the problem goes away. Lightroom isn't actually doing anything wrong (although perhaps its reliance on this one field is perhaps a little optimistic). The problem is with whatever appliction you used to create the DNG files.

I hope that might be of some help,

Jonathan

ps this was Lightroom behaviour about 5 years ago, but your symptoms sound so similar that I'm pretty confident its the same root cause
 
Jonathan, I think you might have hit on the issue. I can see where LrC might inspect the Raw Data Unique ID and come to the conclusion that it already has a preview stored.


Sent from my iPad using Tapatalk
 
Yes, for the Leica M240 it tried to generate a random number, but unfortunately reset its "random" number generator every time it was switched on! The number it generated depended upon how long it had been switched on when you took the picture, so every few months you would get an image that would show an old preview. I used exiftool to replace the RawDataUniqueID for those images affected an d that solved the problem.
 
In case it is of help to the OP when reporting the bug to Helicon, here's the definition of the RawDataUniqueID field from the DNG specification v1.4:
This tag contains a 16-byte unique identifier for the raw image data in the DNG file. DNG readers can use this tag to recognize a particular raw image, even if the file's name or the metadata contained in the file has been changed.

If a DNG writer creates such an identifier, it should do so using an algorithm that will ensure that it is very unlikely two different images will end up having the same identifier.

I hope that helps,
Jonathan
 
I downloaded your 3 example DNG files, and they have the same RawDataUniqueID value:
I confirmed that this is indeed the cause of the misbehavior. When I imported the three sample .dngs into my LR 10, they all showed the same preview. After I used Exiftool to delete the RawDataUniqueID fields and then reimported them, they showed different previews:
Code:
exiftool -rawdatauniqueid= *.dng
 
I confirmed that this is indeed the cause of the misbehavior. When I imported the three sample .dngs into my LR 10, they all showed the same preview. After I used Exiftool to delete the RawDataUniqueID fields and then reimported them, they showed different previews:
Code:
exiftool -rawdatauniqueid= *.dng

So This seems to be a Helicon Bug that needs to be reported.


Sent from my iPad using Tapatalk
 
The "bug" was reported to Helicon a week or so ago. They reproduced the problem, then went silent.

I put the entry from here about the .dng standard in the Helicon Support Forum. I'm guessing they'll disagree and swat the ball back across the net to Adobe.
Hopefully, I'm wrong and they'll fix it if it actually IS a problem on their end.

What I don't get is why this has been going on so long and nobody yelled? I started seeing it as soon as I used the D850 to create stacks in RAW. I can't be the only person the uses Helicon Focus this way and has the Lightroom problem with the .dng files. Wouldn't Helicon have hit this when they tested - it's not like it's some subtle or minuscule thing...
 
What I don't get is why this has been going on so long and nobody yelled? I started seeing it as soon as I used the D850 to create stacks in RAW. I can't be the only person the uses Helicon Focus this way and has the Lightroom problem with the .dng files. Wouldn't Helicon have hit this when they tested - it's not like it's some subtle or minuscule thing...
Rereading the entire thread, it looks like I'm the only one (of about 6?) who was able to reproduce the problem with the sample files you posted. Which suggests subtleties in the conditions that trigger the bad behavior, which means that even reasonably careful testing by Helicon could have missed the issue. In general, bugs involving data caching can be devilishly ticklish to diagnose.
 
Status
Not open for further replies.
Back
Top