Not writing to XMP despite setting

[David Gordon]

Lightroom Version Number

15.4.1

Operating System

1. macOS 15 Sequoia

My catalogue setting for 'automatically write changes into sidecar files' is ticked. Metadata is not being written to my XMP files.

I'm annoyed because I followed my usual workflow which involves fine tuning captions back in Photo Mechanic after I've processed files. I then have LrC read metadata from file which means I'm exporting the updated captions etc. So now I've just reset all my develop setting...

Any clues as to why has started happening?

 

[Gnits]

I used PM , incl metadata... which was subsequently updated within LrC as images were imported.

However, I always regarded this route as one way. I never considered using PM to maintain metadata going forward. I also never considered writing data back to xmp files, so PM would use this updated info.

Perhaps I do not fully understand your workflow.

 

[David Gordon]

Perhaps I do not fully understand your workflow.

Perhaps I'm not good at explaining!

[the following uses "edit" in its original and correct form... ]

Files are ingested via Photo Mechanic which includes a basic caption and all the other ownership info. This creates the XMP.
Make a selection in PM using the stars to edit a set.
Import the edit into LrC and process. LrC should now write the processing data to the XMP.
(Sometimes) Return to PM to fine tune captions, add further, specific, information about each image. PM updates the XMP.
Back in LrC, update the metadata by "read metadata from files".
Export from LrC complete with fully updated caption and keyword etc data.

I've always maintained metadata in XMP as well as within LrC. For me the XMP file is the definitive metadata so I'll use whichever tool, Photo Mechanic or Lightroom or whatever to update it. Writing back to XMP is my escape route should I need out of Adobe. It worked when Apple dropped Aperture!

 

[clee01l]

XMP files are only created for proprietary RAW file types. For known public file types, XMP is written into the header (DNG, JPEG, TIFF etc. So what filetypes are you expecting to see separate XMP files?

 

[Johan Elzenga]

There is a known issue if Photo Mechanic saves XMP files with the extension in capitals. Lightroom uses lower caps and because Lightroom is case sensitive, it chokes on this. I believe you can change this in PM.

 

[David Gordon]

So what filetypes are you expecting to see separate XMP files?

Yes, they are raw files.

 

[David Gordon]

There is a known issue if Photo Mechanic saves XMP files with the extension in capitals. Lightroom uses lower caps and because Lightroom is case sensitive, it chokes on this. I believe you can change this in PM.

Thanks, I'll try that.

Do you know if this is an issue that will get fixed, its a reported bug? I wonder if this is why I'm now seeing the 'metadata needs updating' icon on a lot of images as I browse through in grid view. These are generally archive images which haven't been touched by any app recently. But they will have XMP sidecars, not xmp sidecars. So although setting PM to create XMP files with a lower case suffix will help today, this may still bite me.

 

[David Gordon]

For uppercase suffixes I ran (using Terminal on macOS)

for f in *.XMP; do mv "$f" "${f%.XMP}.xmp"; done

in case anyone else needs a fix!

 

[Ron-Alexander]

For uppercase suffixes I ran (using Terminal on macOS)

for f in *.XMP; do mv "$f" "${f%.XMP}.xmp"; done

in case anyone else needs a fix!

If it's just one folder at a time, you can do it in finder. For a bunch of folders wrapping a tree walker around that code snippett will do an entire photo collection.

 

[David Gordon]

If it's just one folder at a time, you can do it in finder. For a bunch of folders wrapping a tree walker around that code snippett will do an entire photo collection.

Tree walking is a new experience for me! Apparently this should work, but I’ve yet to try.

cd "/top/level/folder"

find . -type f -name '*.XMP' -print0 | while IFS= read -r -d '' f; do
dir=$(dirname "$f")
base=$(basename "$f" .XMP)
mv "$f" "$dir/$base.xmp"
done

 

[Ron-Alexander]

Tree walking is a new experience for me! Apparently this should work, but I’ve yet to try.

cd "/top/level/folder"

find . -type f -name '*.XMP' -print0 | while IFS= read -r -d '' f; do
dir=$(dirname "$f")
base=$(basename "$f" .XMP)
mv "$f" "$dir/$base.xmp"
done

I am not strong with scripting, but it looks close. To test, change the mv to echo so nothing permanent happens. Allow it to run for a few pages at the terminal then when you are SURE it is working, Control-C to stop the script, BACK UP the folder tree, and change the echo back to mv and let it rip.
I would even copy a small part of the affected tree to a small portable drive or USB stick to try it on first before running it against the main drive.

 

[Ron-Alexander]

Tree walking is a new experience for me! Apparently this should work, but I’ve yet to try.

cd "/top/level/folder"

find . -type f -name '*.XMP' -print0 | while IFS= read -r -d '' f; do
dir=$(dirname "$f")
base=$(basename "$f" .XMP)
mv "$f" "$dir/$base.xmp"
done

That way will be very slow, here is a faster version. The comments should help. I would still test on a USB stick or simiar.
#!/bin/bash
set -euo pipefail

TOP_DIR="/top/level/folder"
DRY_RUN=1 # 1 = preview only (no files touched), 0 = actually rename

# --- Rename all .XMP files to lowercase .xmp -------------------------------
# Uses a two-step rename (via .tmp_rename) because macOS's default
# filesystem (APFS) is case-insensitive but case-preserving — a direct
# rename from .XMP to .xmp can silently fail/no-op since the OS sees
# them as "the same" filename. Routing through a temp name forces it
# through unambiguously.
find "$TOP_DIR" -type f -name '*.XMP' -print0 | while IFS= read -r -d '' f; do
if [ "$DRY_RUN" -eq 1 ]; then
echo "Would rename: $f -> ${f%.XMP}.xmp"
else
mv -- "$f" "${f}.tmp_rename"
mv -- "${f}.tmp_rename" "${f%.XMP}.xmp"
fi
done

 

[Ron-Alexander]

Tree walking is a new experience for me! Apparently this should work, but I’ve yet to try.

cd "/top/level/folder"

find . -type f -name '*.XMP' -print0 | while IFS= read -r -d '' f; do
dir=$(dirname "$f")
base=$(basename "$f" .XMP)
mv "$f" "$dir/$base.xmp"
done

I just noticed your OS is older, so there could be subtle differences in bash. The set -euo pipefail will stop the script as soon as any error is detected so make sure you have clean access to the tree. STRONGLY suggest make a backup, test on a test drive like a USB drive. DRY_RUN first

 

[Ron-Alexander]

Just one more thing, because your OS is older, I don't know if it will complain re security. If it does just handle it and either the script will continue or you will need to just run it again. NO harm with that, so don't be thrown if that should happen.

 

[David Gordon]

It now seems to me this is not an xmp or XMP lower or uppercase problem.

I've set Photo Mechanic to create xmp files with a lowercase suffix and its doing so. However LrC is still not automatically writing metadata to file despite the preferences being set to do so. I've just processed a set of images and had to manually save the metadata to file. Once I do so and make another Develop change I need to save that file again. XMP vs xmp seems to be a red herring.

Perhaps I could reset the LrC preferences, delete the prefs file and start again, see if that fixes it. But I'd rather not have to reset LrC and recreate my working space if I can avoid that.

 

[Gnits]

Just to clarify.

Is LrC not writing the metadata to the XMP file ... or is PM not reading the edited data from the XMP file....

 

[David Gordon]

Just to clarify.

Is LrC not writing the metadata to the XMP file ... or is PM not reading the edited data from the XMP file....

Not writing.

I ingested via PM (as workflow derailed above) and processed as usual. When done I went to the Grid view and all the files reported metadata needed saving. The metadata written by PM was correctly read. iI haven’t yet done my second round of caption tweaking as a caution. The XMP files written by PM are .xmp suffix. Tweaking Develop results in another alert metadata needs saving. Exactly as you’d expect if the catalogue setting for “save metadata” was unticked.

 

[Gnits]

If I get Pm working.. I will do a test later this evening.

 

[johnbeardy]

Does your script also rename the originals? I'm thinking that xyz.raf would expect its sidecar to be xyz.xmp. And have you tried tickling Lightroom into saving the xmp, eg by temporarily adding a dummy keyword?

xmp writing isn't always immediate. As it's intended only as a support/communication method, it happens in the background.

Also look in the Metadata panel's EXIF field grouping, review the sidecars field:

 

[David Gordon]

Does your script also rename the originals? I'm thinking that xyz.raf would expect its sidecar to be xyz.xmp. And have you tried tickling Lightroom into saving the xmp, eg by temporarily adding a dummy keyword?

Photo Mechanic's Ingest simply copies the (in my case) RAF files to a location of choosing. It also creates an XMP file with all the metadata I want to add, ownership details, location, caption etc. At this point, no, I'm just using the camera filename. So I end up with file1.raf and file1.xmp. (I also have file1.JPG as I've been shooting some jobs raw+JPEG. But that can't have any relevance, can it...).

I added a test keyword to one file as I started this post, LrC is still asking if the metadata should be saved, its not happening automatically either 'instantly' or after a few minutes.

The files do show "xmp" as sidecars in the EXIF view. Those with masks also have "acr", both as expected.


[sorry, I can't see to delete this now empty quote!]

 

[David Gordon]

What's the correct Preference file for LrC? I'm currently looking in
/Users/davidgordon/Library/Preferences/com.adobe.LightroomClassicCC7.plist
where I see

which suggests "automatically write changes to sidecar files (XMP/ACR)" is not enabled "false" when it is ticked in the application.

Should I edit the plist? Set it to "true"? What could possibly go wrong!

 

[johnbeardy]

That is the correct preference file for Mac. It's odd that the highlighted setting doesn't match Catalog Settings. Does its timestamp match the last time you exited Lightroom?

Providing Lightroom is shut when you do it, and that you have some kind of backup, on Windows I wouldn't hesitate to edit it. I'm less familiar with how Mac handles preference files, but it's worth trying. If it screws up, you can always reset the preferences

 

[David Gordon]

That is the correct preference file for Mac. It's odd that the highlighted setting doesn't match Catalog Settings. Does its timestamp match the last time you exited Lightroom?

Yes, the timestamp was recent.

Anyway, I made a copy, renamed the original to keep it safe and edited the new plist. No change.

I think I'll just reset the LrC prefs (somehow). Probably easy to rebuild my personal preferences than continue down rabbit holes. If that works we'll never know.

And the answer is: we'll never know. Hold down Option-Shift on a LrC launch and you get a dialog offering to reset the prefs. That appears to have solved my problem. For the moment!

 

[johnbeardy]

I don't consider it's so true on Windows, but on Mac clearing the preferences is a cure-all for the weirdest problems.

 

[Ron-Alexander]

Yes, the timestamp was recent.

Anyway, I made a copy, renamed the original to keep it safe and edited the new plist. No change.

I think I'll just reset the LrC prefs (somehow). Probably easy to rebuild my personal preferences than continue down rabbit holes. If that works we'll never know.

And the answer is: we'll never know. Hold down Option-Shift on a LrC launch and you get a dialog offering to reset the prefs. That appears to have solved my problem. For the moment!

I think you need to re-set the store with catalogue option to retain a bunch of settings.