• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • Dark mode now has a single preference for the whole site! It's a simple toggle switch in the bottom right-hand corner of any page. As it uses a cookie to store your preference, you may need to dismiss the cookie banner before you can see it. Any problems, please let us know!

metadata catalog vs actually in image/XMP

Status
Not open for further replies.

DocDJ

Docinator
Joined
Jul 16, 2015
Messages
40
Lightroom Experience
Intermediate
Lightroom Version
Classic
Lightroom Version Number
LRC
Operating System
  1. Windows 10
I am working with Exiftool & ImageMagick to extract & manipulate specific metadata fields from my JPG files, so I can use them to modify the images (similar to watermarking) outside LR for display with each image. I have discovered that MANY of the fields (eg; sublocation) that are available in the LR metadata panel, do not get written to the JPG files when I use "Save metadata to file". I have used exiftool -a -u -g1 a.jpg to display ALL the metadata it can find and none of the (non-empty) fields I want, show up. Does this mean that LR can ONLY write standardized fields? Or do they get written outside the EXIF/IPTC area of my JPG files? I have thousands of such images. I have attached a sample of the results from Exiftool for 1 image. You will see that the Country field is available, but not the sublocation field.
 

Attachments

  • jpglist.txt
    14.3 KB · Views: 207
It is a well established fact that the "Save Metadata to XMP" only adds some metadata back to the original. What metadata gets added to a derivative when you export and include "All metadata" in the Metadata section of the Export dialog?
 
How does a newbie to LR metadata find out about such "well-known" information? It would be nice if the non-writable fields were highlighted in the LR panels. I have compared the EXPORT ALL METADATA with the metadata in the file (in WIndows Explorer) from "Save Metadata to File" and used exiftool -a -u -g1exiftool -a -u -g1. The EXPORTED file version has the metadata I want (city).
 
Last edited:
How does a newbie to LR metadata find out about such "well-known" information? It would be nice if the non-writable fields were highlighted in the LR panels. I have compared the EXPORT ALL METADATA with the metadata in the file (in WIndows Explorer) from "Save Metadata to File" and used exiftool -a -u -g1exiftool -a -u -g1. The EXPORTED file version has the metadata I want (city).

You acquire these bits of information by reading to learn about Lightroom from sites such as this one. I know of no other way.
Adobe does not provide a list of what metadata is excluded from the “Write Metadata to XMP” and the whole Save metadata to a file has been downgraded since Lightroom was first introduced. In Lightroom classic, the metadata in the catalog is usually out of sync with what is stored in the image file. In the Cloud based version of Lightroom “Write Metadata to XMP” is not even an option. Source image files are really not safe to work with if you want to maintain the integrity of the Lightroom inventory. If you want a file that has been processed by Lightroom, you should export and create a derivative file that contains your edits and metadata additions.


Sent from my iPad using Tapatalk
 
You acquire these bits of information by reading to learn about Lightroom from sites such as this one. I know of no other way.
Adobe does not provide a list of what metadata is excluded from the “Write Metadata to XMP” and the whole Save metadata to a file has been downgraded since Lightroom was first introduced. In Lightroom classic, the metadata in the catalog is usually out of sync with what is stored in the image file. In the Cloud based version of Lightroom “Write Metadata to XMP” is not even an option. Source image files are really not safe to work with if you want to maintain the integrity of the Lightroom inventory. If you want a file that has been processed by Lightroom, you should export and create a derivative file that contains your edits and metadata additions.


Sent from my iPad using Tapatalk
My plan has always been to work with derivative files. I never want to modify what I see in LR, unless I have used LR to pass it to PS. But, as a retired professional programmer and teacher of programming, I expected more coding integrity and documentation clarity in a professional product than I see in this area. Thank you for the clarification.
 
How does a newbie to LR metadata find out about such "well-known" information? It would be nice if the non-writable fields were highlighted in the LR panels. I have compared the EXPORT ALL METADATA with the metadata in the file (in WIndows Explorer) from "Save Metadata to File" and used exiftool -a -u -g1exiftool -a -u -g1. The EXPORTED file version has the metadata I want (city).
I tested this on an original jpeg in LrC, and when I examine using Jeffrey Friedl's web-based image viewer (which uses Exiftool) I see all the location fields. I'm therefore unable to explain why you do not.
 
I tested this on an original jpeg in LrC, and when I examine using Jeffrey Friedl's web-based image viewer (which uses Exiftool) I see all the location fields. I'm therefore unable to explain why you do not.
Is it possible to see what Exiftool parameters he /you used to get ALL the data.
 
Is it possible to see what Exiftool parameters he /you used to get ALL the data.
I don't think so, but you could ask him yourself ([email protected]).

BTW, I'm a little confused by your first post in which you said you could see the country but not the sub-location, and in a alter post you said the data you wanted was the City....yet the City is already showing in the original text file (Nairobi). What am I missing?
 
I'm a user of EXIFTOOL and EXIFTOOLGUI myself and like to give myself headaches delving into the issues of metadata. ;)

I took an empty JPG and ran "exiftool -a -u -g1 " against it to produce FROM_CAMERMA.TXT. I then went into LRc 9.4, imported the JPG, and set the following fields.

1612906098791.png

Performed as 'Save Metadata to File', ran EXIFTOOL again to get SAVED_FROM_LRc.TXT.

When I look at, I can see the values in IPTC I set for the "LRc Image *" values.

1612906781811.png


I then exported the JPG to another JPG with ALL METADATA to EXPORT_FROM_LRC.

So, I'm not sure what the issue is. Are you looking at the IPTC-EXTENSION metadata fields by chance?
 

Attachments

  • 1612906299077.png
    1612906299077.png
    14 KB · Views: 154
  • Export_from_LRc.txt
    9.3 KB · Views: 208
  • Saved_from_LRc.txt
    13.7 KB · Views: 179
  • From_Camera.txt
    11.2 KB · Views: 222
I don't think so, but you could ask him yourself ([email protected]).

BTW, I'm a little confused by your first post in which you said you could see the country but not the sub-location, and in a alter post you said the data you wanted was the City....yet the City is already showing in the original text file (Nairobi). What am I missing?
I HAD originally wanted "location", but when I exported, instead of grabbing the file from Explorer, I saw "City", so decided I could live with that. Sorry for the sloppy wording and weird segue.
 
I'm a user of EXIFTOOL and EXIFTOOLGUI myself and like to give myself headaches delving into the issues of metadata. ;)

I took an empty JPG and ran "exiftool -a -u -g1 " against it to produce FROM_CAMERMA.TXT. I then went into LRc 9.4, imported the JPG, and set the following fields.

View attachment 16044
Performed as 'Save Metadata to File', ran EXIFTOOL again to get SAVED_FROM_LRc.TXT.

When I look at, I can see the values in IPTC I set for the "LRc Image *" values.

View attachment 16049

I then exported the JPG to another JPG with ALL METADATA to EXPORT_FROM_LRC.
I'm a user of EXIFTOOL and EXIFTOOLGUI myself and like to give myself headaches delving into the issues of metadata. ;)

I took an empty JPG and ran "exiftool -a -u -g1 " against it to produce FROM_CAMERMA.TXT. I then went into LRc 9.4, imported the JPG, and set the following fields.

View attachment 16044
Performed as 'Save Metadata to File', ran EXIFTOOL again to get SAVED_FROM_LRc.TXT.

When I look at, I can see the values in IPTC I set for the "LRc Image *" values.

View attachment 16049

I then exported the JPG to another JPG with ALL METADATA to EXPORT_FROM_LRC.

So, I'm not sure what the issue is. Are you looking at the IPTC-EXTENSION metadata fields by chance?


So, I'm not sure what the issue is. Are you looking at the IPTC-EXTENSION metadata fields by chan
Thanks so much for posting your examples. Using them, I discovered that SubLocation is ACTUALLY spelled as 'Sub-Location' (the hyphen is required) and I can now have my cake and eat it too!!!
 
As a general rule, all metadata fields in LR's Metadata panel and all Develop settings are written to industry-standard locations in the photo or the .xmp sidecar (for raws and .heic's). LR does not write metadata for videos. Other catalog metadata, such as stacking or collection membership, generally isn't written into file metadata.

Note that LR often uses names for fields that differ from the industry-standard names. And for some LR Metadata panel fields such as Caption, LR follows the industry standards and writes them into multiple file locations:

Code:
$ exiftool -a -G a.jpg  | grep -i caption
[EXIF]          Image Description               : My caption
[XMP]           Description                     : My caption
[IPTC]          Caption-Abstract                : My caption

The easiest way to determine the correspondence between a LR Metadata panel field and file field is to put a unique value in the Metadata panel field, do Metadata > Save Metadata To File, and then use "exiftool -a -G" to examine the file. Use "-a", since there are some metadata fields that have the same name but appear in multiple metadata sections (e.g. EXIF and XMP). Use "-G" to prefix the fields with the section in which they appear.
 
As a general rule, all metadata fields in LR's Metadata panel and all Develop settings are written to industry-standard locations in the photo or the .xmp sidecar (for raws and .heic's). LR does not write metadata for videos. Other catalog metadata, such as stacking or collection membership, generally isn't written into file metadata.

Note that LR often uses names for fields that differ from the industry-standard names. And for some LR Metadata panel fields such as Caption, LR follows the industry standards and writes them into multiple file locations:

Code:
$ exiftool -a -G a.jpg  | grep -i caption
[EXIF]          Image Description               : My caption
[XMP]           Description                     : My caption
[IPTC]          Caption-Abstract                : My caption

The easiest way to determine the correspondence between a LR Metadata panel field and file field is to put a unique value in the Metadata panel field, do Metadata > Save Metadata To File, and then use "exiftool -a -G" to examine the file. Use "-a", since there are some metadata fields that have the same name but appear in multiple metadata sections (e.g. EXIF and XMP). Use "-G" to prefix the fields with the section in which they appear.
Yes, that is what I've discovered and have used that command many times to find my text. As a retired programmer and retired teacher of Computer Science, I find it appalling that LR has put names in its panels that do not match the actual fieldnames inside the metadata. I would fire programmers that did that. And fields that are ONLY in the catalog, should be in a special sub-panel clearly marked as "LR catalog-only" fields. It would be nice to have a pop-up tip on a field to show it's actual name as written in the metadata. Exiftool doesn't show the actual fieldname either ("Image Height" vs "ImageHeight").
 
I find it appalling that LR has put names in its panels that do not match the actual fieldnames inside the metadata.
It's a messy situation, and overall I think LR has navigated the naming issue reasonably well (though not perfectly).

Photo apps have long used field names different from the industry-standard names, for a number of reasons: The old legacy standards were created for much different users and use cases (e.g. IPTC for the press), there are multiple standards with different names for what's essentially the same field, and product developers often didn't care about standards (e.g. very old Microsoft photo apps).

Consider LR's "Caption": It gets stored in three standard fields, EXIF:ImageDescription, XMP:Description, IPTC:Caption-Abstract. Yet most pro and consumer photographers have been trained by many apps and web services, not just LR, over many years to call it a "caption".

The various date-time fields are more confusing, no fault of LR. The EXIF standard (created by Japanese camera manufacturers) calls "capture date" EXIF:DateTimeOriginal. The EXIF:DateTime field (shown as Date Time in the Metadata panel) doesn't, as most users think, refer to when the shutter was pressed -- rather, it's when a camera or app conforming to the standard last modified the photo file. So I think it was reasonable for LR to call it "capture date" rather than by a name, Date Time Original, a name chosen by engineers for whom English was a second language.

Exiftool doesn't show the actual fieldname either ("Image Height" vs "ImageHeight").

Exiftool is very good about preserving the standards' field names, with some exceptions. By default, it shows the "pretty name" of a field (e.g. "Date Time Original"), but if you use the -s option, it shows the standard's name ("DateTimeOriginal"). There are some exceptions: It replaces "/" in IPTC fields with "-", and it rarely renames a field. For example, it renames the EXIF DateTime field to ModifyDate (perhaps because the author got tired of answering countless questions from confused users). And I recall it renames a very small number of fields that shared the same names but different meanings between standards (though I can't recall those offand).
 
This thread led me back to looking for the "Metadata Working Group" which was to help resolve a lot of the issues handling multiple metadata schemas in an image file.

Someone may have more information but the web site for this group is no longer active www.metadataworkinggroup.org .

I found this paper from 2010 which provides some insights "GUIDELINES FOR HANDLING IMAGE METADATA"
 
It's a messy situation, and overall I think LR has navigated the naming issue reasonably well (though not perfectly).

Photo apps have long used field names different from the industry-standard names, for a number of reasons: The old legacy standards were created for much different users and use cases (e.g. IPTC for the press), there are multiple standards with different names for what's essentially the same field, and product developers often didn't care about standards (e.g. very old Microsoft photo apps).

Consider LR's "Caption": It gets stored in three standard fields, EXIF:ImageDescription, XMP:Description, IPTC:Caption-Abstract. Yet most pro and consumer photographers have been trained by many apps and web services, not just LR, over many years to call it a "caption".

The various date-time fields are more confusing, no fault of LR. The EXIF standard (created by Japanese camera manufacturers) calls "capture date" EXIF:DateTimeOriginal. The EXIF:DateTime field (shown as Date Time in the Metadata panel) doesn't, as most users think, refer to when the shutter was pressed -- rather, it's when a camera or app conforming to the standard last modified the photo file. So I think it was reasonable for LR to call it "capture date" rather than by a name, Date Time Original, a name chosen by engineers for whom English was a second language.



Exiftool is very good about preserving the standards' field names, with some exceptions. By default, it shows the "pretty name" of a field (e.g. "Date Time Original"), but if you use the -s option, it shows the standard's name ("DateTimeOriginal"). There are some exceptions: It replaces "/" in IPTC fields with "-", and it rarely renames a field. For example, it renames the EXIF DateTime field to ModifyDate (perhaps because the author got tired of answering countless questions from confused users). And I recall it renames a very small number of fields that shared the same names but different meanings between standards (though I can't recall those offand).
I've dealt with standards and misnomers a bit. In some of my work, I have put the "common" name juxtaposed to the "standard" name whenever possible. I have been on "standards" committees, where we had to define our terms relative to old common usage. It's hard work and sometimes mind-bending, especially when working with people who refuse to recognize that standards will (eventually) make their job easier. Ah well, I'm retired now so it's up to the new generation to tackle it and for me to learn to live with what I cannot change and rely on others who have discovered the differences and published them as you all here have done. And many thanks to you for doing it. (PS. I'm one of those curmudgeons who hates it when people say "and also" without recognizing the redundancy. Darwin: where are you?)
 
Yes, that is what I've discovered and have used that command many times to find my text. As a retired programmer and retired teacher of Computer Science, I find it appalling that LR has put names in its panels that do not match the actual fieldnames inside the metadata. I would fire programmers that did that. And fields that are ONLY in the catalog, should be in a special sub-panel clearly marked as "LR catalog-only" fields. It would be nice to have a pop-up tip on a field to show it's actual name as written in the metadata. Exiftool doesn't show the actual fieldname either ("Image Height" vs "ImageHeight").
With your background and skill sets, it might not be too difficult to learn the Lua scripting language and write a LR plug-in that does exactly what you want. There are several respected plug-in authors who are members of this forum, and they could even advise you on commercialization.

Phil Burton
 
With your background and skill sets, it might not be too difficult to learn the Lua scripting language and write a LR plug-in that does exactly what you want. There are several respected plug-in authors who are members of this forum, and they could even advise you on commercialization.

Phil Burton
That's a great idea. I've used many different scripting languages, so what's one more? I'll have to dig into it. Thanks for the tip. I currently have it working with BAT files (ugh) and plan to port it to OOREXX, but a plugin would be even better.
 
Someone may have more information but the web site for this group is no longer active www.metadataworkinggroup.org . I found this paper from 2010 which provides some insights "GUIDELINES FOR HANDLING IMAGE METADATA"
The Metadata Working Group consortium let its web site go dark a couple years ago. The Guidelines for Handling Image Metadata is actually written as a specification (despite its title), and Adobe explicitly stated that LR follows that spec (which it does, with a couple imperfections). You can download the document from the Internet Archive:
https://web.archive.org/web/20181006115950/http://www.metadataworkinggroup.org/specs/
 
Consider LR's "Caption": It gets stored in three standard fields, EXIF:ImageDescription, XMP:Description, IPTC:Caption-Abstract. Yet most pro and consumer photographers have been trained by many apps and web services, not just LR, over many years to call it a "caption".
Before I started using LR, I wanted to set the metadata fields in my photos so started to use EXIFTOOL with BAT files. That's when I discovered the duplication across different schemas. My BAT file captures the value once then assigns it across the different schemas. like you were observing JohnRellis
 
The Metadata Working Group consortium let its web site go dark a couple years ago. The Guidelines for Handling Image Metadata is actually written as a specification (despite its title), and Adobe explicitly stated that LR follows that spec (which it does, with a couple imperfections). You can download the document from the Internet Archive:
https://web.archive.org/web/20181006115950/http://www.metadataworkinggroup.org/specs/
Thanks for the links. I downloaded the specs and test files.
 
Before I started using LR, I wanted to set the metadata fields in my photos so started to use EXIFTOOL with BAT files. That's when I discovered the duplication across different schemas. My BAT file captures the value once then assigns it across the different schemas. like you were observing JohnRellis
Exiftool now provides MWG composite tags that read and write the multiple fields according to the Metadata Working Group spec:
MWG Tags

This will approximate how LR reads and writes the multiple fields (LR diverges from the spec in at least a couple corner cases).
 
Status
Not open for further replies.
Back
Top