Library module Can no longer save metadata

[dotcalm50]

Lightroom Version Number

14.0.1

Operating System

1. macOS 14 Sonoma

Mac Sonoma 14.7.1 Lightroom Classic 14.0.1
I'm unsure what happened beyond my immediately reversed decision to turn on Automatically Write Changes into XMP. For years, I've routinely saved metadata with the simple Ctrl-S. My catalog has over 500K images.
When I try to save metadata in any of the different manners, the saving metadata progress bar remains empty, and the metadata never gets saved, even after 10's of minutes. I created a new catalog from a single folder (147 images), and AM able to save metadata changes in that folder. Currently I'm exporting files in my main catalog to a new one to see if that solves the issue. I know similar (or identical) problems have been described and resolved here, (I think @John Ellis helped), but frankly, I couldn't quite follow the process for resolution.

 

[clee01l]

When I try to save metadata in any of the different manners, the saving metadata progress bar remains empty, and the metadata never gets saved, even after 10's of minutes. I created a new catalog from a single folder (147 images), and AM able to save metadata changes in that folder.

In System preferences, does Lightroom have "Full Disk Access"? With the 147 image new catalog are the images stored in a different folder/volume from your master catalog? Is the volume where you can not write full and perhaps out of disk space?

 

[dotcalm50]

Thank you Cletus,
Yes, LR has Full Disk Access, and the 147 images are in the same folder.

 

[clee01l]

Have you reset Preferences and restarted LrC? This some times doew magic and fixes strange weirdness.

What purpose are you wanting to achieve by saving XMP? Are you trying to sync to the Adobe Cloud also? There is some incompatibility between syncing and writing to XMP.

 

[dotcalm50]

Thanks again Cletus.
I was able to resolve the issue by simply exporting the entire catalog to a new one....at least for now. My intuition suggests that I corrupted the catalog when I turned on-then off, after a few minutes, the Automatic saving of metadata in the Catalog Prefs.
To answer your question, I catalog Nikon, Panasonic, and Olympus raw files, which require (unless I'm missing something) settings to be stored in XMP sidecar files. That said, I also have thousands of DNG and JPG files in my catalog in which I was also unable to save the metadata. I hope that clears things up. If you know something I don't, I'm all "ears".

 

[Johan Elzenga]

To answer your question, I catalog Nikon, Panasonic, and Olympus raw files, which require (unless I'm missing something) settings to be stored in XMP sidecar files.

You are missing something. Lightroom Classic saves all settings in the catalog. Writing to XMP is never required. It is an option, that you do not need to use.

 

[dotcalm50]

Thank you Johan. In my experience the LRC catalog is too vulnerable to store develop/metadata settings only within the catalog. Too many times I have lost hours of work, so I now have a near-real time backup (as well as a daily backup of the catalog) of each image. It’s a comfortable, reliable workflow for me. There’s no real-time perfect solution that I can see.

 

[clee01l]

Thank you Johan. In my experience the LRC catalog is too vulnerable to store develop/metadata settings only within the catalog. Too many times I have lost hours of work, so I now have a near-real time backup (as well as a daily backup of the catalog) of each image. It’s a comfortable, reliable workflow for me. There’s no real-time perfect solution that I can see.

Almost all of the Lightroom community finds the catalog file the most secure and least vulnerable place to store ALL of your metadata. You may be the lone exception. I do not recall ever having a corrupt catalog file in the 15 or so years that I have been using Lightroom.

I used to have a Time Machine backup every hour. Now I only have timeMachine backup once a day. It alternates between two backup drives Backblaze runs once a day also. I used to use Acronis as a 4th backup but did not renew my subscription this year.

I run LrC 7X24 and need to remember to exit to get LrC to make a copy of the catalog file which I have accummulating on an old EHD kept for that single use purpose.

The only down time I have had was when my Primary Disk failed in my iMac and I had to get Apple to install a new one. I've never creatd separate XMP and never wished I had. I have had to go back 6 months to find a backup catalog to recover keywords from ~2000 images that I deleted in a Stupid User Mistake. IMO writing to both the master catalog file and an XMP file is a performance hit. If an image file format allows for the inclusion of XMP, then the System Backup overhead increases since the whole image file needs to be backed up again and again.

 

[dotcalm50]

Almost all of the Lightroom community finds the catalog file the most secure and least vulnerable place to store ALL of your metadata. You may be the lone exception. I do not recall ever having a corrupt catalog file in the 15 or so years that I have been using Lightroom.

I used to have a Time Machine backup every hour. Now I only have timeMachine backup once a day. It alternates between two backup drives Backblaze runs once a day also. I used to use Acronis as a 4th backup but did not renew my subscription this year.

I run LrC 7X24 and need to remember to exit to get LrC to make a copy of the catalog file which I have accummulating on an old EHD kept for that single use purpose.

The only down time I have had was when my Primary Disk failed in my iMac and I had to get Apple to install a new one. I've never creatd separate XMP and never wished I had. I have had to go back 6 months to find a backup catalog to recover keywords from ~2000 images that I deleted in a Stupid User Mistake. IMO writing to both the master catalog file and an XMP file is a performance hit. If an image file format allows for the inclusion of XMP, then the System Backup overhead increases since the whole image file needs to be backed up again and again.

Cletus, Appreciate the informed response. I have had a catalog corrupted at least three times in the (many) years I have used LRC. I've certainly never lost more than a few hours of work, but it nevertheless was annoying enough to incentivize my work flow. I'm always open to other ideas, and I suppose "almost all of the Lightroom community" carries some weight here.

 

[Johan Elzenga]

There is nothing wrong with a workflow that uses both catalog backups as well as XMP. I do not use XMP for two reasons: Reason 1: writing to XMP means that if you have DNG files and/or RGB files (TIFF,PSD, JPG) among your images, then the writing to XMP changes the original file itself, not just a small XMP sidecar file. And that means that your images backup will be triggered to write those files to the backup again. I don’t like that, because any write action can create corruption, and consequently that corruption would propagate into my backups.
Reason 2: Writing to XMP means a lot of extra disk activity, which can only slow down Lightroom.
For me, those two reasons outweigh the possible loss of some edits I made in the last hour, but of course that is entirely personal.

 

[John R Ellis]

When I try to save metadata in any of the different manners, the saving metadata progress bar remains empty, and the metadata never gets saved, even after 10's of minutes. I created a new catalog from a single folder (147 images), and AM able to save metadata changes in that folder. ... I was able to resolve the issue by simply exporting the entire catalog to a new one

These symptoms suggest that the original catalog's Helper.lrdata folder got corrupted. It caches metadata for speeding display in LR Library, and LR occasionally soils it. It may be that Save Metadata To File is trying to update Helper.lrdata and failing and hanging.

---
For future reference, it is safe to delete Helper.lrdata:

a) Do Catalog Settings > General > Show to open Finder / File Explorer on the current catalog folder.
b) Exit LR.
c) In that folder, delete the folder "<catalog> Helper.lrdata".
d) Restart LR, and it will rebuild the folder.

The Helper.lrdata folder caches information about metadata, keywords, folders, collections, and other things to speed the performance of LR and it can sometimes get corrupted. In recent versions, LR has used it more heavily to improve the speed of displaying metadata, and there have been a fair number of bugs with it.

 

[ml_lrcl]

There is nothing wrong with a workflow that uses both catalog backups as well as XMP. I do not use XMP for two reasons: Reason 1: writing to XMP means that if you have DNG files and/or RGB files (TIFF,PSD, JPG) among your images, then the writing to XMP changes the original file itself, not just a small XMP sidecar file. And that means that your images backup will be triggered to write those files to the backup again. I don’t like that, because any write action can create corruption, and consequently that corruption would propagate into my backups.

I've always written the XMP header to DNG/RGB files as backup and portability (within Adobe), as you say.. I knew it increases file size and, in my own case, Time Machine disk usage. I hadn't thought about it as a potential source of corruption, though. That's useful to consider. I'm curious in terms of workflow: Does this just mean you ignore any "metadata status has changed" notices, and never "save the changes to disk?"

 

[Johan Elzenga]

Does this just mean you ignore any "metadata status has changed" notices, and never "save the changes to disk?"

It does indeed. You can turn off that warning in the View Options, so I do not even see those warnings.

 

[dotcalm50]

These symptoms suggest that the original catalog's Helper.lrdata folder got corrupted. It caches metadata for speeding display in LR Library, and LR occasionally soils it. It may be that Save Metadata To File is trying to update Helper.lrdata and failing and hanging.

---
For future reference, it is safe to delete Helper.lrdata:

a) Do Catalog Settings > General > Show to open Finder / File Explorer on the current catalog folder.
b) Exit LR.
c) In that folder, delete the folder "<catalog> Helper.lrdata".
d) Restart LR, and it will rebuild the folder.

The Helper.lrdata folder caches information about metadata, keywords, folders, collections, and other things to speed the performance of LR and it can sometimes get corrupted. In recent versions, LR has used it more heavily to improve the speed of displaying metadata, and there have been a fair number of bugs with it.

These symptoms suggest that the original catalog's Helper.lrdata folder got corrupted. It caches metadata for speeding display in LR Library, and LR occasionally soils it. It may be that Save Metadata To File is trying to update Helper.lrdata and failing and hanging.

---
For future reference, it is safe to delete Helper.lrdata:

a) Do Catalog Settings > General > Show to open Finder / File Explorer on the current catalog folder.
b) Exit LR.
c) In that folder, delete the folder "<catalog> Helper.lrdata".
d) Restart LR, and it will rebuild the folder.

The Helper.lrdata folder caches information about metadata, keywords, folders, collections, and other things to speed the performance of LR and it can sometimes get corrupted. In recent versions, LR has used it more heavily to improve the speed of displaying metadata, and there have been a fair number of bugs with it.

Thank you John. Very useful information indeed.