“Automatically write changes into XMP”

[argonauta]

Lightroom Version Number

Classic 7.3

Operating System

1. macOS 10.13 High Sierra

I love this option, “Automatically write changes into XMP”. But we know that performance worsens in this way.
I have a suggestion, there might be an option to write changes into XMP when we close the program. What about?

 

[Zenon]

Does this option leave sidecars after?

 

[johnbeardy]

Well, we don't all know that performance worsens in this way. It's a background task.

The option creates sidecars for proprietary raw file formats, not for other file types. It can also trigger image backup routines.

 

[Zenon]

Thanks

 

[PhilBurton]

Well, we don't all know that performance worsens in this way. It's a background task.

The option creates sidecars for proprietary raw file formats, not for other file types. It can also trigger image backup routines.

Specifically, if you use DNG RAW files, changing just a few bytes of metadata will trigger a backup of the entire multi-MB DNG file.

 

[Laura Smith]

Specifically, if you use DNG RAW files, changing just a few bytes of metadata will trigger a backup of the entire multi-MB DNG file.

Any idea if CrashPlan is smart enough to only upload the changes rather than the whole DNG?

 

[palengh]

Will We be able to (force) write XMP sidecars for every file?
I know LR does it for RAW, and other proprietary formats, but how do we does LR to make XMP’s for all files?


Regards

Pål Engh
Fredrikstad, NORWAY

 

[Jim Wilde]

Will We be able to (force) write XMP’s for every file?
I know LR does it for RAW, and other proprietary formats, but how do we does LR to make XMP’s for all files?

You can automatically write changes into XMP for every supported file-type, but XMP sidecars are only created for proprietary Raw files. With all other file-types (e.g. DNG, Jpeg, Tiff), Lightroom writes the updated XMP data directly into the XMP section of the file header.

 

[palengh]

Will We be able to (force) write XMP’s for every file?
I know LR does it for RAW, and other proprietary formats, but how do we does LR to make XMP’s for all files?

You can automatically write changes into XMP for every supported file-type, but XMP sidecars are only created for proprietary Raw files. With all other file-types (e.g. DNG, Jpeg, Tiff), Lightroom writes the updated XMP data directly into the XMP section of the file header.

I meant sidecars.
We use Fotoware at work. ( http://www.fotoware.com). My idea would be to use Lightroom as a client for meta tagging. I have not yet tested if Fotoware picks up the XMP’s embedded into the file. I assume it will.
Sadly the file change date is updated after an XMP change, fuzzing the FotoWeb sort order.

Any ideas would be appreciated.


Regards

Pål Engh
Fredrikstad, NORWAY

 

[Linwood Ferguson]

Any idea if CrashPlan is smart enough to only upload the changes rather than the whole DNG?

Highly unlikely.

This is an area where I've periodically turned it on the turned it off, not exactly for DNG (which I do not use) but old JPG's. I will run off and start doing face recognition, or maybe correcting typos in keywords. I might change one keyword, and affect 1000 images including a lot of old JPG's, and suddenly LR is rewriting all those files. Even if they are small, I hate having all that churn (every time you rewrite a file you risk it being corrupted -- tiny risk, but do a tiny risk enough times and.... ).

So mostly I keep it off, and periodically go back and do a mass update of the XMP's. Not as good as a backup, but it does cut down on the churn.

Keywords, frankly is the worst of it. I (almost) never apply any kind of develop setting to more than a few images (other than ones just ingested), but just some minor keyword changes and you can touch thousands.

 

[Hoggy]

I have a suggestion, there might be an option to write changes into XMP when we close the program. What about?

If you create that suggestion on the Adobe site, please post the link here. I'll certainly vote for it.
It's really what I always do anyways, but manually via a smart collection method.

And yep, I know about those minor keyword changes causing a rewrite of possibly thousands. I still do it though.

So if such an option would be added to preferences, then maybe have a checkmark box within the backup dialog, stating "Write changes into XMP (x images need updating)".

 

[LouieSherwin]

I am pretty sure that writing to XMP is a background task so that will already minimize any performance contention. But as Linwood says adding multiple keywords can trigger multiple writes to the same files and when working with anything other than XMP sidecar files for proprietary raw files you can trigger a lot of disk activity both local and for any backups.

You could mimic a save metadata at close by using the "Metadata Status" in the filter bar. Select all in that need to be updated and use the Metadata ->Save Metadata to file (Cmd-S) to write all the changes at one time.

The save XMP feature was really intended to provide interoperability with Bridge and Photoshop and other external editors in the early days. I don't believe that it was never intended as any kind of backup for metadata though it is often mentioned in that context.

I turned off "Automatically write changes to XMP" a long time ago having come to the conclusion that there is nothing to be gained over having a consistent and reliable backup procedure for your catalog and your images. Backing up the catalog frequently especially when doing a lot of important work will be the best protection against problems.

-louie

 

[Laura Smith]

I turned off "Automatically write changes to XMP" a long time ago having come to the conclusion that there is nothing to be gained over having a consistent and reliable backup procedure for your catalog and your images. Backing up the catalog frequently especially when doing a lot of important work will be the best protection against problems.

Yeah, I might do the same. I turned it on with the mindset of insurance against Lr being mysteriously discontinued in the middle of the night one day. But in hindsight if that happens I think I'll have bigger things to worry about ...

 

[Gnits]

Remember...this is a per catalog setting, not a system wide setting.

 

[tspear]

Depending on how much you are doing and how fast you work, automatic writing meta-data can slow your system.
Easiest example, make a keyword change and then run an import of a 1000 images with facial recognition turned on.
However, for my normal usage, I never notice that write meta-data is enabled.

Now, a trick I have not tried since early 6.X. I convert to DNG (skipping whole reason why), but did not want the updates to meta-data in the DNG at the time. So I enabled the read-only flag on all my DNG files; Lr then proceeded to create an XMP sidecar file for each DNG. I did not do this very long, but it was possible. I no longer have a reason to test this functionality (and not sure I would recommend it anyway) but it did work.

 

[Hoggy]

Now, a trick I have not tried since early 6.X. I convert to DNG (skipping whole reason why), but did not want the updates to meta-data in the DNG at the time. So I enabled the read-only flag on all my DNG files; Lr then proceeded to create an XMP sidecar file for each DNG. I did not do this very long, but it was possible. I no longer have a reason to test this functionality (and not sure I would recommend it anyway) but it did work.

That's a good trick to try and keep in mind. Right now, I have an extension that will write xmp sidecars for all file types, including virtual copies - should I ever feel the need to. But it may stop working some day, and I wouldn't know enough about Lua to fix it or write my own. And something makes me feel that Adobe would not add such functionality natively, although it would be nice.

 

[PhilBurton]

That's a good trick to try and keep in mind. Right now, I have an extension that will write xmp sidecars for all file types, including virtual copies - should I ever feel the need to.

Hoggy,

Extension? Do you mean plug-in? Link?

Phil

 

[Hoggy]

Extension? Do you mean plug-in? Link?

Tomato, tomaato.
pm sent.

 

[tspear]

@Hoggy

Send me a PM too! I wanna know.

Tim

 

[palengh]

... I have an extension that will write xmp sidecars for all file types, including virtual copies - should I ever feel the need to.

@Hoggy

Send me a PM too! I wanna know.


Regards

Pål Engh
Fredrikstad, NORWAY

 

[johnbeardy]

It's certainly possible for a plugin to create xmp files for files that don't usually have them.

But don't overestimate the value of those sidecars. If you try to read the metadata into LR and other apps, it's not always clear whether they read the metadata from the image file or from the sidecar. [Without checking] I am pretty sure that because Lr doesn't expect to find an xmp sidecar for a DNG, it will ignore it.

 

[Johan Elzenga]

It's certainly possible for a plugin to create xmp files for files that don't usually have them.

But don't overestimate the value of those sidecars. If you try to read the metadata into LR and other apps, it's not always clear whether they read the metadata from the image file or from the sidecar. [Without checking] I am pretty sure that because Lr doesn't expect to find an xmp sidecar for a DNG, it will ignore it.

Correct. Apple Aperture can write sidecar files for any type of image, so also for jpeg and tiff, for example. But when you import such a jpeg or tiff into Lightroom, metadata like keywords will not come across, because Lightroom ignores the sidecar.

 

[Hoggy]

Yeah, I'm not sure how LR would natively handle external sidecars for filetypes that support embedding. One would definitely need to check that. But that plugin creates menu items for reading from the external sidecars - and also recreating missing virtual copies from xmp sidecars from virtual copies that were saved by it.

BUT - and this is important: that plugin stopped being developed, so it may not work fully anymore - or some aspects to it may stop working at any point in future versions of LR. So make sure you test things, if you're planning to rely on it.

Also, if interested, just pm me directly - without writing a message here. I honestly wasn't intending quite the stir-up, and don't want to burden this thread.

 

[Hoggy]

I honestly wasn't intending quite the stir-up, and don't want to burden this thread.

Yet, OTOH, the devil in me says that maybe people should write here.. :devil::laugh: If anyone from Adobe is reading, then they might realize that people really do want such functionality.
Come to think of it, even though I haven't used that plugin in quite a while - I probably should write xmp sidecars for just all my virtual copies. The fact that there's no secondary/tertiary external backup for those has always made me feel a bit uneasy about using them, which is why I always make sure to snapshot them and 'save metadata' the main image. The problem comes in, though, when LR won't show the 'has changed' badge on the main image (nor the VC) for snapshots created that way... And I may easily forget to force a save - or it doesn't quite register, and I don't notice that it hasn't.

And then there's the aspect that should an unlikely total meltdown occur, VC's won't automatically be created by LR natively. At least with that plugin, there's a chance that that part will still work down the road. Although the problem would still remain, of the VC's or main image never showing a 'has changed' badge.

 

[johnbeardy]

Yet, OTOH, the devil in me says that maybe people should write here.. :devil::laugh:
If anyone from Adobe is reading, then they might realize that people really do want such functionality.

No, it does make sense to mention the existence of this plugin, but it's important to mention the dependence on the plugin for reading these sidecars and the fact that it's no longer being developed. From what I know of the area, I would expect that it still works, but I would not wish to fix it if there is a problem.

I'd be very surprised if Adobe opened this can of worms, and the reading issue extends to third party apps, as Johan's example shows. Sidecars are a workaround for proprietary file formats that aren't publicly documented, not for standard file formats that don't need them.

John