Is there a way to tell when LR is done writing XMP files?

Joined
Mar 12, 2017
Messages
101
Lightroom Experience
Beginner
Lightroom Version
Classic 7
#1
Lightroom Version
Lightroom Classic version: 7.5 [ 1186149 ]
Operating System
macOS 10.13 High Sierra
Is there a way to tell when LR is done writing XMP files?

I have Lightroom set up to do this because I lost work once with a corrupted Catalog. When I'm doing Catalog maintenance, it would help me to know when Lightroom is done writing the XMP files.

Thx.
 

Laura Shoe

Lightroom Guru
Staff member
Lightroom Guru
Joined
Apr 2, 2009
Messages
98
Location
Seattle
Lightroom Experience
Advanced
#2
If you select lots of photos to write metadata to, you'll see progress in the status bar. Otherwise I don't think there's a way to see it. If you close Lightroom before it is finished, it will warn you and give you a chance to cancel the quit.

Note that frequent catalog backups are a better approach to protecting against corruption, since writing to XMP leaves out pick/reject flags, virtual copies, presence in collections, and step-by-step edit history (but not the edits).
 
Joined
Mar 12, 2017
Messages
101
Lightroom Experience
Beginner
Lightroom Version
Classic 7
#3
Note that frequent catalog backups are a better approach to protecting against corruption, since writing to XMP leaves out pick/reject flags, virtual copies, presence in collections, and step-by-step edit history (but not the edits).
Thank you. Can you expand on this? I have Lightroom set to test Catalog integrity, optimize Catalog, and back-up on exit. This takes a few minutes — which I don't have time for when working. What is the recommended workflow for making frequent Catalog back-ups over the course of a day?
 
Joined
Nov 16, 2015
Messages
1,422
Location
California, USA
Lightroom Experience
Intermediate
Lightroom Version
Classic 7
#4
Is there a way to tell when LR is done writing XMP files?

I have Lightroom set up to do this because I lost work once with a corrupted Catalog. When I'm doing Catalog maintenance, it would help me to know when Lightroom is done writing the XMP files.

Thx.
Does your system have a drive activity light? When writing XMP files, it should be on almost 100%. Once XMP files are all written, disk activity will probably be intermittent, e.g. OS background activity.
 

Jim Wilde

Lightroom Guru
Staff member
Moderator
Lightroom Guru
Joined
Feb 1, 2010
Messages
10,910
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
Classic 7
#5
Thank you. Can you expand on this? I have Lightroom set to test Catalog integrity, optimize Catalog, and back-up on exit. This takes a few minutes — which I don't have time for when working. What is the recommended workflow for making frequent Catalog back-ups over the course of a day?
I'd probably uncheck the Optimise and Test Integrity steps on all but perhaps the last of the day, which would certainly speed up the intermediate backups.
 
Joined
Mar 12, 2017
Messages
101
Lightroom Experience
Beginner
Lightroom Version
Classic 7
#6
Does your system have a drive activity light? When writing XMP files, it should be on almost 100%. Once XMP files are all written, disk activity will probably be intermittent, e.g. OS background activity.
Thank you. I'll look into the usability of this. It's a 30 TB RAID with activity lights (currently out of sight).
I use iStat menus, so I can take a quick look at drive activity via its Menu Bar icon.

Still, I would find it useful to have this specific information (telling me the status of Lightroom's XMP updating) easily shown by Lightroom within Lightroom. Although if I change my workflow and turn off "Write to XMP" it wouldn't be useful.
 
Joined
Mar 12, 2017
Messages
101
Lightroom Experience
Beginner
Lightroom Version
Classic 7
#7
I'd probably uncheck the Optimise and Test Integrity steps on all but perhaps the last of the day, which would certainly speed up the intermediate backups.
I thought this would be a quick bridge over the bog, but it isn't.

Even with those two important processes skipped during the while-working back-ups forced by closing Lightroom, it's taking between 3m30s and 21m00s (three efforts, two HD's) for Lightroom to close (just copy & compress Catalog). Catalog is 7 GB. Back-ups are to ED's. Some of those drives may be being used by other programs.

I have no idea why it takes so long, but since the system works well I will hypothesize that the slowness comes from Lightroom and not from the system.

Even 3½ minutes is too long to stop working while cycling through a Catalog save.
 

Laura Shoe

Lightroom Guru
Staff member
Lightroom Guru
Joined
Apr 2, 2009
Messages
98
Location
Seattle
Lightroom Experience
Advanced
#8
Hi Kirby, regarding catalog backups: all of your work is contained in the catalog. Making backups over time allows you to recover from a catalog corruption by allowing you to recover from one that was made before the corruption. I would go into Edit (PC) or Lightroom (Mac) > Catalog Settings, and change the backup frequency to Every Time Lightroom Exits (or Once a Day). Then close LR regularly and back up. As Jim mentions, you can uncheck Optimize This Catalog - do that maybe once a month or if you notice Lightroom slowing down. See if that speeds up the backups enough - if not, you can uncheck Test Integrity, but only if you are sure that you'll do a backup once a day with it checked - otherwise leave it checked.

These catalogs will accumulate and eventually take up a lot of hard drive space. Periodically go into your Backups folder and delete all but a handful of the most recent ones. (I have a reminder once a month on my calendar to do this.) The backup location is specified in the backup prompt that comes up when you close LR. Speaking of that, if you haven't done any significant work in your current session, then you can hit Skip.
 

Laura Shoe

Lightroom Guru
Staff member
Lightroom Guru
Joined
Apr 2, 2009
Messages
98
Location
Seattle
Lightroom Experience
Advanced
#9
We were writing at the same time. Try storing your backups on your internal drive - see if that speeds up the process. As long as you back up your internal drive, that would be fine.
 

Laura Shoe

Lightroom Guru
Staff member
Lightroom Guru
Joined
Apr 2, 2009
Messages
98
Location
Seattle
Lightroom Experience
Advanced
#10
3.5 minutes may seem long, but if you use collections and the other items I mentioned that aren't saved to XMP, I'd venture to say you'd burn through 3.5 minutes many times over trying to recover from a catalog corruption without a backup. Do you get up from your desk periodically for meals, nature calls, etc? That would be a good time to do the backups.
 
Joined
Mar 12, 2017
Messages
101
Lightroom Experience
Beginner
Lightroom Version
Classic 7
#12
Making backups over time allows you to recover from a catalog corruption by allowing you to recover from one that was made before the corruption. I would go into Edit (PC) or Lightroom (Mac) > Catalog Settings, and change the backup frequency to Every Time Lightroom Exits (or Once a Day). Then close LR regularly and back up.
Just to be sure: I have every desire to be able to recover all data and work at any time.

What I've been doing (since a Lightroom crash some time ago) is to
- set Lightroom to write changes to XMP
- back up Catalog on exit
- back up the back-ups to a second spindle.

My personal pomodoro is about 2½ hours. ¯\_(ツ)_/¯ . I can accept losing a whole plump tomato worth of work if "tighter" data/work security cuts my tomato into pieces.

You suggested above relying solely on regular back-ups (and not using, and therefore not creating, up-to-date XMP files).
frequent catalog backups are a better approach to protecting against corruption, since writing to XMP leaves out pick/reject flags, virtual copies, presence in collections, and step-by-step edit history (but not the edits)
As long as I can accept:
- doing a full close of Lightroom with attendant back up, at the end of every work session, and
- risking the loss of any work done during the most recent work session
do you recommend the workflow of:
+ Set Catalogs to NOT write changes to XMP
+ Open Catalog
+ Work on Photos
+ Close Catalog and back up
+ On last back-up of day, have Lightroom check Catalog integrity and optimize Catalog?

Is there any reason to have Lightroom "automatically write changes into XMP" (as it is worded at "Catalog Settings ▹ Editing")?

I should note that the XMP issue I brought up in this thread may be resolved when I move this very large Catalog+Previews to a dedicated drive.
 

johnbeardy

Senior Member
Staff member
Moderator
Lightroom Guru
Joined
Oct 8, 2007
Messages
4,706
Location
London
Lightroom Experience
Power User
Lightroom Version
#13
My preference is to rely on daily backup of the catalogue by my third party backup software, and I only occasionally run Lightroom's backup upon exit, and then mainly for the integrity check and optimisation.

I don't run the automatic saving to XMP because it's primarily designed for data exchange with other apps, not for backup, and it misses loads of the work I'd do in LR (eg flags, stacking, collections). Another aspect which may be relevant is that XMP doesn't just mean the small sidecars - with PSDs, TIFs, JPEGs, DNGs, it is written into the files and can therefore trigger unnecessary backup of large files.
 
Top