Lightroom Classic constantly re-saving XMP file information

[MikePhotog]

Lightroom Version Number

15.1.1

Operating System

1. Windows 11

Hello,

My apologies if this has been covered already, I searched through the forum and only found a vague reference.

I have the setting to save to XMP turned on, I am considering turning it off, but that's a different subject.

LR seems to constantly be re-saving XMP information even through apparently nothing changed in the file information. Just this morning I tested this a few times, I opened a catalog, exited LR, received the message that LR was still saving XMP information, I waited to make sure the prompt went away. I closed LR and reopened the catalog again. When I tried to exit, LR tells me it is still saving the information.

Has anyone run into this, and have you found a solution?

I will greatly appreciate your feedback. This is an annoyance for me; I keep two catalogs Master and Working and switch between them quite often.

Mike.,

 

[Califdan]

Has anyone run into this

Yes, in the thousands going back a decade or more

, and have you found a solution?

No other than turning off the automatic write of XMP to disk

Some people (myself included) have this problem and some don't. It's unlclear why some have the problem and some don't. Adobe so far has not been able to get it fixed. Unclear if it is something in some images or something in the catalog or something else entirely. Symptom is that literally seconds after you save XMP data to disk and the "Metadata mismatch' icon go off, it comes right back on.

 

[Zenon]

Out of curiosity. Why do you have XMP enabled?

 

[BobT]

As I understand it, the main purpose of an XMP file is to allow other members of your workgroup to be able to access the image at the same time. If that's not an issue, yes, certainly turn off "Save to XMP." To tidy up, you could select all you images and run "Save Metadata to File". Then backup everything and delete all your XMP files. A word of warning, all my raws are converted to DNG. It worked for me. I can't vouch for it working with proprietary raws.

Even then, you may still repeatedly see a hamburger and a down arrow stating "Metadata file needs to be updated" despite it just having been updated. This is not the same as your issue but similar. Running "Convert photo to DNG", despite it already being a DNG, sometimes fixes the problem. Other options that sometimes work are to move the file to another folder and back, or to export it and reimport it. I guess what you are trying to do is to clean out any corruption that might have sneaked into the metadata.

 

[Califdan]

As I understand it, the main purpose of an XMP file is to allow other members of your workgroup to be able to access the image at the same time. If that's not an issue, yes, certainly turn off "Save to XMP."

There are several reasons to want XMP data in or with (XMP data is store 'with' proprietary RAW files as an seperatre XMP file, the XMP data is stored 'in' the actual image files for all other file types, including DNG's) your image files.

1. As a last resort backup of your metadata changes and image edits. This only applies to people who do not have an adequete backup system for their catalog and it should never be necessary that 'all is lost' except the XMP data in/with files. However this does not prevent it from being a 'last resort' way to not lose everything if the unthinkable happens. I have had clients who through negligence and lack of understanding coupled with just plain terrible luck have had to rely on the XMP data to save decades of work in LrC.
   
   
2. Allows images with your LrC edits to be seen and possibly used by other image processing or viewing programs without having to export them
   
   
3. I guess in some 'team' workflows XMP data in/with images can facilitate passing images around the team.

But, for the most part it is a "peace of mind" thing. Having said that, If my system didn't have the "never ending writing XMP files" problem I'd probably turn it on unless I was was using a lot of non proprietary RAW files (e.g. Tiff, PSD, DNG, etc.). If I were converting everything to, say, DNG I would not turn it on as every change in LrC would update the large file which in turn would trigger a large file to be backed up by each of my backup systems - and that would form quite a bottleneck. If it were just those tiny seperate XMP files the backups would not be an issue.

To tidy up, you could select all you images and run "Save Metadata to File". Then backup everything and delete all your XMP files.

Not sure how this would tidy things up as those XMP files on the backup drive would quickly become out of date

A word of warning, all my raws are converted to DNG. It worked for me. I can't vouch for it working with proprietary raws.

Every change would be written into your DNG's, causing the file to be marked as "modified" for backup purposes. Proprietary RAW's actually work much better as only these small XMP (and now ACR) files get updated by LrC or need to be backup

Even then, you may still repeatedly see a hamburger and a down arrow stating "Metadata file needs to be updated" despite it just having been updated. This is not the same as your issue but similar.

The "hamburger' icon (as you call it) - of which there are a few flavors - is an indication that LrC believes there is a mismatch between the metadta in/with the image file and the metadata saved in the catalog and if on, will mark that image for the Automatic XMP save operation.

Running "Convert photo to DNG", despite it already being a DNG, sometimes fixes the problem.

Hadn't heard that. Worth a look.

Other options that sometimes work are to move the file to another folder and back, or to export it and reimport it. I guess what you are trying to do is to clean out any corruption that might have sneaked into the metadata.

Again, many things have been tried and AFAIK, even though some ideas like those you mention may work for some images or some people, none that I know of have been shown to be effective over a large number of people. This may be indicative of there being many different root causes of the problem which produce the same basic result.

 

[Gnits]

Just fyi.
I use side car xmp files at the import stage. I can populate the side car files with metadata of my choosing (in my case, Titles, Locations, Comments, etc) specific to the card being imported. If the side car file is well formed, Lr will populate the catalog of the related image file. Works a treat for me. I got the idea from the way PhotoMechanic transfer ratings and other metadata to LrC at the point of import to LrC.

I am not a fan of the Saving to Xmp parameter, as I see no personal benefits … but other use cases may have specific benefits.

I never thought of the use of xmp files in a multi user environment, expecting in the early days of LrC a multi user version would be released, but that looks like a dim prospect now.

 

[PhilBurton]

I never thought of the use of xmp files in a multi user environment, expecting in the early days of LrC a multi user version would be released, but that looks like a dim prospect now.

Because the LrC catalog is stored as an sqlite3 database, it would be technically irresponsible to set up LrC as a multi-user environment. Too much risk of catalog corruption. A guru like John B or John R could explain much better than I can.

 

[Gnits]

I could visualise a setup where by individual users with their own LrC workstations (and local catalog) would have a check in / check out system of an image and related metadata. Image stays on a common (perhaps heavy duty company server), xmp travels to one of the client workstations for editing and then gets checked back in again. I can imagine lots of rules and constraints, such as only one person working on a specific image at a time.

I no longer have a need or the energy to build such a system… but a managed xmp based app might work, probably supported by a client server type app and separate database to manage checking out and in images for editing and managing gaps in the metadata not included in the xmp files.

I am travelling in the reverse direction, ie trying to simplify my workflow and sharing my catalog on an external ssd between my desktop and travel laptop has seriously simplified things , with multiple benefits to me.

 

[Zenon]

I asked because I have no use for it. It has always been clutter to me. These days with Denoise AI and a few other AI tools the catalogue can get big and some of that is offset by XMP and .acr folders. I'm not too worried about the catalogue size at this point anyway. If the storage does not go into the locate.data folder it winds up somewhere else.

 

[Maurizio]

Many years ago, when I moved from Photo Supreme to Lightroom , I attempted to continue using both DAM systems simultaneously, relying on XMP to synchronise metadata between the two. My intention was to benefit from the strengths of each environment. In practice, however, this approach proved to be extremely problematic and eventually turned out into a nightmare.
Since that experience, I have disabled the option “Automatically write changes into XMP”. For me, allowing Lightroom to manage metadata internally within its catalogue provides a far more stable and predictable workflow.
That said, embedding or writing XMP to files can certainly be useful in specific situations, particularly when exporting images to external platforms such as publishing services, client delivery systems or multi-user DAM environments. In those cases, having metadata travel with the file ensures that keywords, captions, copyright information and basic adjustments remain accessible outside Lightroom.
It is worth remembering that Adobe introduced XMP in the early 2000s as an open standard to store metadata in a structured, application-independent format. The aim was to improve interoperability between different software applications and operating systems, replacing older and more limited metadata standards. In principle, XMP provides a common language for metadata exchange. In practice, however, each application may interpret or implement parts of the standard slightly differently, particularly regarding proprietary editing data. This is where synchronisation across multiple DAM systems can become unreliable.
For single-application workflows, especially when working exclusively within Lightroom Classic, relying on the internal catalogue is often simpler and more robust. Writing XMP makes most sense when interoperability, archiving independence, or cross-platform collaboration is a genuine requirement.

 

[Gnits]

In the early days with the arrival of xmp style metadata approaches, I was involved in several commercial level collaborations to try and establish industry level standards for data interchange. The focus was business to business interactivity and the ability to allow client server style architecture interaction with the massive mainframe databases backend legacy applications. Everyone involved could see the massive potential… but no schema was agreed which would (then) operate at a vertical industrial sector level. It was probably at this stage Adobe published their XMP standard. I complement Adobe on this… It is amazing that so much time has passed and we are still using very faded standards for vip current workflows… eg ,DCIM structure on SD cards, lack of video metadata standards, etc. I understand backward compatibility is an ever present consideration.

 

[Califdan]

These days with Denoise AI and a few other AI tools the catalogue can get big and some of that is offset by XMP and .acr folders.

Saving XMP data to the image files does not alter what or how much data is stored in the LrC catalog files - it just replicates some of it.

If the storage does not go into the locate.data folder it winds up somewhere else.

I assume you mean the .lrcat-data folders. Again, saving XMP data to disk does not prevent it from going into this folder as well.

That said, embedding or writing XMP to files can certainly be useful in specific situations, particularly when exporting images to external platforms such as publishing services, client delivery systems or multi-user DAM environments. In those cases, having metadata travel with the file ensures that keywords, captions, copyright information and basic adjustments remain accessible outside Lightroom.

AFAIK, XMP data is included inside EXPORTED files (or in XMP and ACR side car files if exporting proprietry RAW files as "original") depending on the settings you make in the Export dialog. Unless you are exporting proprietary RAW files using "original", image edits are baked into the pixels in the exported files and in the case of exporting proprietary RAW files in thier original format the image edits and metadata are saved in XMP and ACR side car files. This is independent of whether or not you have saved XMP data to disk outside of the Export dialog. Saving the XMP data to disk outside of Export allows you to pass the original master image files around (rather than an exported copy of it) and have your edits and metadata go along for the ride.

 

[Zenon]

Saving XMP data to the image files does not alter what or how much data is stored in the LrC catalog files - it just replicates some of it.

I assume you mean the .lrcat-data folders. Again, saving XMP data to disk does not prevent it from going into this folder as well.


AFAIK, XMP data is included inside EXPORTED files (or in XMP and ACR side car files if exporting proprietry RAW files as "original") depending on the settings you make in the Export dialog. Unless you are exporting proprietary RAW files using "original", image edits are baked into the pixels in the exported files and in the case of exporting proprietary RAW files in thier original format the image edits and metadata are saved in XMP and ACR side car files. This is independent of whether or not you have saved XMP data to disk outside of the Export dialog. Saving the XMP data to disk outside of Export allows you to pass the original master image files around (rather than an exported copy of it) and have your edits and metadata go along for the ride.

Yes I mean the .lrcat-data folders.

 

[Maurizio]

AFAIK, XMP data is included inside EXPORTED files (or in XMP and ACR side car files if exporting proprietry RAW files as "original") depending on the settings you make in the Export dialog. Unless you are exporting proprietary RAW files using "original", image edits are baked into the pixels in the exported files and in the case of exporting proprietary RAW files in thier original format the image edits and metadata are saved in XMP and ACR side car files. This is independent of whether or not you have saved XMP data to disk outside of the Export dialog. Saving the XMP data to disk outside of Export allows you to pass the original master image files around (rather than an exported copy of it) and have your edits and metadata go along for the ride.

Yes, when I export JPEGs (or TIFF in case of exporting to an external application) XMP metadata are always embedded, independent of the status of “Automatically write changes into XMP”.

 

[clee01l]

Yes, when I export JPEGs (or TIFF in case of exporting to an external application) XMP metadata are always embedded, independent of the status of “Automatically write changes into XMP”.

There is no reason that XMP data needs to be saved outside of the catalog file. Edits are applied to the pixels in the derivative file and automatically stored in the derivative file’s header.

Edit settings (XMP) cannot be used and applied in any non Adobe image app.

If you look at the newer Lightroom (cloudy) app it is not available as an option. Everything is always stored in the cloud catalog unless you use the Local option which does not use the cloud and then everything is stored in the XMP in lieu of a catalog.


Sent from my iPhone using Tapatalk

 

[Maurizio]

There is no reason that XMP data needs to be saved outside of the catalog file. Edits are applied to the pixels in the derivative file and automatically stored in the derivative file’s header.

Edit settings (XMP) cannot be used and applied in any non Adobe image app.

I wasn’t referring to XMP edit settings, which are proprietary. I was referring to descriptive metadata embedded in the file, such as title, headline, description, copyright, subject, location, and similar fields.

These metadata can be read and used by the destination application (for example, a web gallery), provided that two things are understood:
(1) which embedded metadata fields (whether XMP or legacy) Lightroom writes its catalog metadata into, and
(2) which embedded metadata fields are actually used by the external application.

I had to run a few tests with ExifTool to clarify the first point.

 

[Swingman]

There is no reason that XMP data needs to be saved outside of the catalog file. Edits are applied to the pixels in the derivative file and automatically stored in the derivative file’s header.

Edit settings (XMP) cannot be used and applied in any non Adobe image app.

If you look at the newer Lightroom (cloudy) app it is not available as an option. Everything is always stored in the cloud catalog unless you use the Local option which does not use the cloud and then everything is stored in the XMP in lieu of a catalog.


Sent from my iPhone using Tapatalk

Cletus,
Like you I imported all my files from iPad to LrC. I hover notice all (most) photos have xmp files and a lua file in each date folder in the SSD. So Why the edit info is stored in the catalog?
Is it because all AI edit info is still stored in XMP files?. As you know all my imports were done on 15.1/15.2.
Is there a Preser Preference or Preset to force to the edit data to catalog?

 

[clee01l]

I was referring to descriptive metadata embedded in the file, such as title, headline, description, copyright, subject, location, and similar fields.

These are not XMP fields. They are EXIF and IPTC fields also stored in the header. They are only stored in the Catalog file or the Original file if you Save metadata back to the original. In a non Proprietarry file like TIFF, JPEG or DNG, there are three sections of the header block that can contain text data. These are EXIF, IPTC and optionally XMP. There are other blocks like thumbnail images etc .The title, headline, description, copyright, subject, location etc are contained in the EXIF or IPTC data blocks.

 

[clee01l]

Like you I imported all my files from iPad to LrC. I hover notice all (most) photos have xmp files and a lua file in each date folder in the SSD. So Why the edit info is stored in the catalog?

Lightroom Mobile has two different options In mobile these are "Device". and "Lightroom" The Lightroom option stores images in the cloud. And you want to use this option when importing if your goal is to sync to your master catalog in Lightroom Classic. The Device option stores the images locally and does not interact with the Adobe Cloud. Because there is no Cloud to store the edit adjustments, Lightroom mobile creates XMP files

 

[Swingman]

Lightroom Mobile has two different options In mobile these are "Device". and "Lightroom" The Lightroom option stores images in the cloud. And you want to use this option when importing if your goal is to sync to your master catalog in Lightroom Classic. The Device option stores the images locally and does not interact with the Adobe Cloud. Because there is no Cloud to store the edit adjustments, Lightroom mobile creates XMP files

You managed to read through my typo. I am on of the worst typist as I miss lot of letters.
Does this mean, all my future edits in LrC also stored in XMP files by adding to it or that saved with catalogs?

 

[clee01l]

You managed to read through my typo. I am on of the worst typist as I miss lot of letters.
Does this mean, all my future edits in LrC also stored in XMP files by adding to it or that saved with catalogs?

No the XMP is local to Lightroom Mobile ONLY for the files stored and edited locally. When Lightroom updates the Adobe Cloud it is the XMP data stored locally that is sent to the Adobe Cloud to be applied everywhere (Mobile devices, LrC Catalog) Once a local image file has been sent to the cloud, the Local XMP is no longer needed. I do not know if Lightroom then deletes the XMP files.

 

[Jim Wilde]

No the XMP is local to Lightroom Mobile ONLY for the files stored and edited locally. When Lightroom updates the Adobe Cloud it is the XMP data stored locally that is sent to the Adobe Cloud to be applied everywhere (Mobile devices, LrC Catalog) Once a local image file has been sent to the cloud, the Local XMP is no longer needed. I do not know if Lightroom then deletes the XMP files.

I don't think that LrMobile creates XMP files for the images stored in the cloud or locally. Edits and other added metadata are stored in the local catalog and synced to the cloud, and it remains in the local catalog after the image(s) have uploaded to the cloud and the local originals removed (thus allowing for offline use). Locally stored images accessed via the Device tab are automatically moved into the Lightroom tab when any adjustments are made (thus added to the catalog), and from there synced to the cloud.

 

[clee01l]

I don't think that LrMobile creates XMP files for the images stored in the cloud or locally. Edits and other added metadata are stored in the local catalog and synced to the cloud, and it remains in the local catalog after the image(s) have uploaded to the cloud and the local originals removed (thus allowing for offline use). Locally stored images accessed via the Device tab are automatically moved into the Lightroom tab when any adjustments are made (thus added to the catalog), and from there synced to the cloud.

When The Local Option was first introduced, XMP files started appearing locally. Swingman noticed the XMP files too. https://www.lightroomqueen.com/comm...aving-xmp-file-information.54396/post-1365485
Hence my response. If using the Lightroom Local option why is there a need to store the edit details in the local library? Why is there even a local Library for local files if not using the Adobe Cloud? I know that if you sync locally edited files to the Cloud those same edits must follow but the local option is available for those users that do not want to sync to the Adobe Cloud.

 

[Jim Wilde]

When The Local Option was first introduced, XMP files started appearing locally. Swingman noticed the XMP files too. https://www.lightroomqueen.com/comm...aving-xmp-file-information.54396/post-1365485
Hence my response. If using the Lightroom Local option why is there a need to store the edit details in the local library? Why is there even a local Library for local files if not using the Adobe Cloud? I know that if you sync locally edited files to the Cloud those same edits must follow but the local option is available for those users that do not want to sync to the Adobe Cloud.

The local option in Lightroom Desktop does indeed create XMP files, but the local (Device) option in Lightroom Mobile is not the same. Using the Device option in Lightroom Mobile allows the user to browse the local files in Photos (Apple) or Gallery (Android), but any attempt to edit or add metadata to those local images silently imports them into the Lightroom app and from there they will be synced to the Adobe Cloud. XMP files are not a factor in that process.

 

[Swingman]

When The Local Option was first introduced, XMP files started appearing locally. Swingman noticed the XMP files too. https://www.lightroomqueen.com/comm...aving-xmp-file-information.54396/post-1365485
Hence my response. If using the Lightroom Local option why is there a need to store the edit details in the local library? Why is there even a local Library for local files if not using the Adobe Cloud? I know that if you sync locally edited files to the Cloud those same edits must follow but the local option is available for those users that do not want to sync to the Adobe Cloud.

Just to confirm, I don’t see any XMP files in LrM. They are only in LrC as all my photos in LrC were synced down from cloud.
I don’t actively use device option in LrC except have seen all my files are there (mirror image) of what in the Lightroom.
The photos in the device folder each photos have Lr and a Downarrow enclosed in filled circle at the top left.
I don’t see any XMP files in the iPad.

Few months ago I used Lightroom Downloader to extract photos and each photos have its own XMP files as well. So they exist in the cloud as separate files. I assume these XMP files were synced down to LrC with their respective photos.

The questions are:
If you directly download photos into LrC ( I haven’t tried it) do you get any sidecar after edits?
If the answer is no, is any of the info in my XMP files get absorbed into the catalog?
If it does, Can I safely delete them?
By the way, in the `preference options, I have writting to XMP turned off.