• Welcome to the Lightroom Queen Forums! We're a friendly bunch, so please feel free to register and join in the conversation. If you're not familiar with forums, you'll find step by step instructions on how to post your first thread under Help at the bottom of the page. You're also welcome to download our free Lightroom Quick Start eBooks and explore our other FAQ resources.
  • Stop struggling with Lightroom! There's no need to spend hours hunting for the answers to your Lightroom Classic questions. All the information you need is in Adobe Lightroom Classic - The Missing FAQ!

    To help you get started, there's a series of easy tutorials to guide you through a simple workflow. As you grow in confidence, the book switches to a conversational FAQ format, so you can quickly find answers to advanced questions. And better still, the eBooks are updated for every release, so it's always up to date.
  • It's Lightroom update time again! As well as the usual new cameras, lens profiles and bug fixes, Lightroom Classic has a new local Hue tool, improved ISO-specific defaults, performance improvements and more. The Lightroom cloud ecosystem also gets local Hue, as well as Versions, Activity notifications, text watermarking and the ability to upload your own Discovery tutorials. Here are the full updates about Lightroom Classic and the Lightroom cloud ecosystem.

more sidecar and catalog questions

Joined
Aug 31, 2019
Messages
62
Lightroom Version
9.2.1
Operating System
macOS 10.15 Catalina
So there is a lot out there, and here, on the above. Ive been reading, re-reading and re-re-reading to the point where im possibly more confused than I was before I started. It began from another thread when I mentioned .dng and .cr2 files, convert or not to - another ball of wax itself.... So, if I may, lay out what I understand to be correct for confirmation and some questions. I save all original files externally from local hard drive outside of LR. I currently shoot raw .cr2 files and had been importing them to LR as such and have the "Automatically write changes into XMP" checked in the catalog Metadata settings. QUESTION - Doing such with that setting checked gives me .xmp sidecar files, right?

I have also, after reading, started to import .cr2 files as .dng files, saving the original .cr2 externally and have the same "Automatically write changes into XMP" checked in the catalog Metadata settings. QUESTION - Because i now have that setting checked w/ a .dng file, LR writes to the XMP section of the file and not a xmp sidecar file?

Ive read that metadata refers to develop settings, tags, labels, crops, etc. that are primarily stored in the catalog file. QUESTION - This is the default whether the above setting is checked or not checked, right? Where can I see that / those files?

To make sure I fully understand this and possibly redundant - jpeg, png, tiff and psd and dng files - allow for writing to the xmp section of the file by having the above option checked and proprietary .cr2 and HEIC will get xmp sidecar files when checked?

Some examples of other proprietary raw files besides .cr2, HEIC ?

Confirm: Ive read that all changes to pictures are always recorded to the catalog - yes or no? If yes, when recorded to the catalog, does this imply to a xmp sidecar file or just the catalog?

If saving to a sidecar file, will that picture be able to be opened in PS as is, w/o further changes?

Confused - Under the Catalog Settings window, Metadata, "Automatically write changes into XMP", is this what writes the sidecar file or is this referring to the "into xmp" section of the file?

Are .xmp sidecar files hidden by default? Im on a Mac.

So now that ive probably asked the same question 18 different ways - all this reading as mentioned has led to confusion for me. Additionally some places are written by those much more experienced and I think sometimes some things are taken for granted and maybe not explained very well and im just trying to make sure I fully understand this before getting too deep into it.

Additionally, from the other thread I briefly made reference to dng vs cr2 --- this led to convert or not to - a whole other argument - file check vs no file check - in hind site, at this point, probably a non-issue for me. Still trying to learn that just because i read it doesn't mean I have to try it. Maybe just concentrate on basic workflow.

Finally, this is what id like to accomplish, and my understand, please correct if not - as mentioned, on a Mac and I use time capsule and store originals on external drive. Its my understand(please confirm), that if I write to the xmp portion of a file, those changes are picked up as file changes by time capsule and I get a new back up copy with every change made, right? This seems like a waste of space compared to if I were just writing to a sidecar file and that was the file being backed up and not the whole image file.

What settings should I set to use sidecar files and not back up the picture with every edit made?

Finally, If worried about space, and if I were to keep the settings as is and seeing as I save originals external, Im safe to exclude this LR (picture only directory) from Apple backups and still be safe? I am backing up my catalog in LR AND time capsule.

Thanks all for any help here in getting me straight!

Attached is what I have now.
 

Attachments

Roelof Moorlag

Senior Member
Joined
Jan 27, 2013
Messages
1,309
Location
Netherlands
Lightroom Experience
Power User
Lightroom Version
I currently shoot raw .cr2 files and had been importing them to LR as such and have the "Automatically write changes into XMP" checked in the catalog Metadata settings. QUESTION - Doing such with that setting checked gives me .xmp sidecar files, right?
Yes

QUESTION - Because i now have that setting checked w/ a .dng file, LR writes to the XMP section of the file and not a xmp sidecar file?
Yes

Ive read that metadata refers to develop settings, tags, labels, crops, etc. that are primarily stored in the catalog file. QUESTION - This is the default whether the above setting is checked or not checked, right? Where can I see that / those files?
This data is saved into the Lightroom database (SQLite)
 
Last edited:

Roelof Moorlag

Senior Member
Joined
Jan 27, 2013
Messages
1,309
Location
Netherlands
Lightroom Experience
Power User
Lightroom Version
Confused - Under the Catalog Settings window, Metadata, "Automatically write changes into XMP", is this what writes the sidecar file or is this referring to the "into xmp" section of the file?
Both! It writes XMP files for photo's wich i can not write into the XMP section of the file itself

Additionally, from the other thread I briefly made reference to dng vs cr2 --- this led to convert or not to - a whole other argument - file check vs no file check - in hind site, at this point, probably a non-issue for me. Still trying to learn that just because i read it doesn't mean I have to try it. Maybe just concentrate on basic workflow.
Preserving the integrity of your files is probalby 'basic workflow' i think? In that case, the file validation option of DNG can be very valuable.

What settings should I set to use sidecar files and not back up the picture with every edit made?
If you use sidecar files they are the only files that are changed. Only changed files are in the backup.
When you have backups of your files and from you catalog (.cat file) then there is no need for XMP however.

inally, If worried about space,
Why? Storage is very cheap these days
 
Last edited:
Joined
Nov 5, 2017
Messages
752
Location
UK
Lightroom Experience
Power User
Lightroom Version
Classic
When you have backups of your files and from you catalog (.cat file) then there is no need for XMP however.
I agree with all your comments, including this one - there is no need. However, as an additional step (in event of catalog failure and inability to repair) or for reading by some other programs outside of Lr, they can have their place.
 
Joined
Aug 31, 2019
Messages
62
Both! It writes XMP files for photo's wich i can not write into the XMP section of the file itself


Preserving the integrity of your files is probalby 'basic workflow' i think? In that case, the file validation option of DNG can be very valuable.


If you use sidecar files they are the only files that are changed. Only changed files are in the backup.
When you have backups of your files and from you catalog (.cat file) then there is no need for XMP however.
To clarify - so if I have the box check w/ .dng file, it writes to the XMP section of the file and if I have it checked w/ .cr2 files I get a .xmp sidecar file?

yes

when you refer to "XMP", do you mean XMP Metadata section of file or .xmp sidecar? Nomenclature ? Because I have .cr2 files mixed with .dng files in the same catalog I do not want to be backing up each .dng when I make a small change. Thats why I wanted to know if I could unselect that directory from the time machine back up.
 
Last edited:
Joined
Jul 2, 2015
Messages
9,999
Location
Netherlands
Lightroom Experience
Power User
Lightroom Version
I agree with all your comments, including this one - there is no need. However, as an additional step (in event of catalog failure and inability to repair) or for reading by some other programs outside of Lr, they can have their place.
Please consider two things. Catalog failure is best dealt with by making regular catalog backups, not by writing changes to XMP. Too many catalog stuff is not written to XMP. And writing changes to XMP will trigger your image backup application if you use DNG. The entire DNG file will be copied again.
 
Joined
Aug 31, 2019
Messages
62
Sorry for the confusion - I didnt mean to say, if I did, that anything was not necessary. I'm just trying to figure out the above and what happens when, so that I have a full understanding of all the settings. As far as I can see now, as a new user, seems redundant if I have an original unmolested file in offsite storage to keep making backups when I might just be making simple edit (by writing into the XMP section of a file). So I'm seeing what I need to have set to only have the sidecar file, that the changes be made there and that would be the file that would be included in the backup by Time Machine and how to accomplish that and any additional provisions, if necessary(another question), that would need to be done to have the file to be be able to be used by PS or another edit program while using sidecar file.....
 

Roelof Moorlag

Senior Member
Joined
Jan 27, 2013
Messages
1,309
Location
Netherlands
Lightroom Experience
Power User
Lightroom Version
As far as I can see now, as a new user, seems redundant if I have an original unmolested file in offsite storage to keep making backups when I might just be making simple edit (by writing into the XMP section of a file)
1. Writing to XMP is not necasary for a dam backup
2. If you decide to write into XMP then only the XMP sidecarfiles are changed so in the backup. When the file has a XMP section, there wil not be written a sidecar file. So, in these situations the whole file wil be in the backup
3. There is no option in Lightroom to force XMP sidecar files on files with a XMP section (as DNG)
 
Joined
Aug 31, 2019
Messages
62
1. Writing to XMP is not necasary for a dam backup
Ok, you are a "Power User" with thousands of posts and I am new trying to understand this - there's no need to curse. I never said writing to XMP is necessary for a backup.
2. If you decide to write into XMP then only the XMP sidecarfiles are changed so in the backup. When the file has a XMP section, there wil not be written a sidecar file. So, in these situations the whole file wil be in the backup
Tks
3. There is no option in Lightroom to force XMP sidecar files on files with a XMP section (as DNG)
Tks
 
Joined
Aug 31, 2019
Messages
62
If saving to a sidecar file, will that picture be able to be opened in PS as is - showing edits, w/o further changes?

Or do the changes have to written to the file and then opened in PS?

What settings should I set to use sidecar files and not back up the picture with every edit made?
 
Joined
Nov 5, 2017
Messages
752
Location
UK
Lightroom Experience
Power User
Lightroom Version
Classic
If saving to a sidecar file, will that picture be able to be opened in PS as is - showing edits, w/o further changes?
Saving to XMP is irrelevant if you open Ps directly from Lr - see page 336 of your Classic book for the options.
 
Joined
Feb 1, 2010
Messages
12,345
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
Saving to XMP is irrelevant if you open Ps directly from Lr - see page 336 of your Classic book for the options.
And while reading that page, you could skip forward a few pages to page 345 to the section called "Saving Metadata to the Files"....there you'll find the answers to most of the questions you've been asking here. If there's anything that you don't understand then tell us what that is.

What settings should I set to use sidecar files and not back up the picture with every edit made?
It's not a question of settings, it's a question of which file-type you are using....sidecar files will only be created when you save XMP to proprietary raw files (CR2/3, NEF, ARW, etc.). For ALL other file-types in your catalog saving XMP will update the actual files, and so would cause your backup software to back up those updated files. All this is covered in your book as specified above.
 
Joined
Aug 31, 2019
Messages
62
Sorry! English is not my native language and i meant to write 'sound'...
Got it - tks for the help - sometimes that spell check can get you - dam is close to sound......

Saving to XMP is irrelevant if you open Ps directly from Lr - see page 336 of your Classic book for the options.
ok. tks.

And while reading that page, you could skip forward a few pages to page 345 to the section called "Saving Metadata to the Files"....there you'll find the answers to most of the questions you've been asking here. If there's anything that you don't understand then tell us what that is.
So that's why I put all this up--- my very 1st sentence of the post says it all, "So there is a lot out there, and here, on the above. Ive been reading, re-reading and re-re-reading to the point where im possibly more confused than I was before I started." Ive read a lot in Victoria's book and Martin Short's, specifically pages 626-634 and more.... I was simply confused and trying to understand the details. For instance, Should I write to XMP? section on 346... Is very helpful and everyone's situation is different, but its not clear at all which are Pros and which are Cons... Also, on that page at the end, another confusion to me and another question I asked but no answer I think - Which of Lightroom's data isn't stored in XMP - nomenclature - when its said, "stored in XMP", this will always refer to the XMP section of the file like the next section How do I write to XMP, also referencing the XMP section of the file? Compared to when "xmp sidecar" is used will always refer to the sidecar file?

So, to confirm...... Catalog Settings, Metadata, ☑ Include Develop metadata settings inside JPEG, TIFF, and PSD files when checked will always write to file XMP and when not checked to XMP sidecar. Same with the next setting, Automatically write changes into XMP - when checked the XMP section of the file will get updated. If unchecked, one gets an XMP sidecar file?
 
Joined
Feb 1, 2010
Messages
12,345
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
So, to confirm...... Catalog Settings, Metadata, ☑ Include Develop metadata settings inside JPEG, TIFF, and PSD files when checked will always write to file XMP and when not checked to XMP sidecar. Same with the next setting, Automatically write changes into XMP - when checked the XMP section of the file will get updated. If unchecked, one gets an XMP sidecar file?
No, you still don't seem to get this. The only option it to write to XMP or not to write to XMP (the default is NOT to write to XMP as all that data, plus more, is always stored in the catalog database). If, however, you do choose to write to XMP then how that is done is determined (as I said earlier) by the type of files that you are dealing with.....for proprietary Raw files, the XMP data is stored in XMP sidecar files, but for all other file-types the XMP data is stored in the XMP section of the file header. That is automatically determined by Lightroom, the user cannot control how the XMP data is stored in the files.
The other option (Include Develop Settings in metadata....) only determines if the develop settings are additionally included with all the other metadata when writing to XMP.....if you don't write to XMP, having this option checked is irrelevant,
 
Joined
Nov 5, 2017
Messages
752
Location
UK
Lightroom Experience
Power User
Lightroom Version
Classic
For instance, Should I write to XMP? section on 346... Is very helpful and everyone's situation is different, but its not clear at all which are Pros and which are Cons...
They are listed as Pros and Cons - the first three can be considered potential pros, the five that follow (that start 'on the downside) are cons.

Which of Lightroom's data isn't stored in XMP - nomenclature - when its said, "stored in XMP", this will always refer to the XMP section of the file like the next section How do I write to XMP, also referencing the XMP section of the file? Compared to when "xmp sidecar" is used will always refer to the sidecar file?
Page 346, the paragraph immediately before 'Should I write to XMP?' makes it clear from most file types this will be in the file header, but for proprietary files it is written to a separate sidecar file.

So, to confirm...... Catalog Settings, Metadata, ☑ Include Develop metadata settings inside JPEG, TIFF, and PSD files when checked will always write to file XMP and when not checked to XMP sidecar. Same with the next setting, Automatically write changes into XMP - when checked the XMP section of the file will get updated. If unchecked, one gets an XMP sidecar file?
No, nothing is written to any type of file unless you tell it to - by Ctrl/Cmd+S or ticking the Automatically write changes into XMP
 
Joined
Aug 31, 2019
Messages
62
No, you still don't seem to get this. The only option it to write to XMP or not to write to XMP (the default is NOT to write to XMP as all that data, plus more, is always stored in the catalog database). If, however, you do choose to write to XMP then how that is done is determined (as I said earlier) by the type of files that you are dealing with.....for proprietary Raw files, the XMP data is stored in XMP sidecar files, but for all other file-types the XMP data is stored in the XMP section of the file header. That is automatically determined by Lightroom, the user cannot control how the XMP data is stored in the files.
The other option (Include Develop Settings in metadata....) only determines if the develop settings are additionally included with all the other metadata when writing to XMP.....if you don't write to XMP, having this option checked is irrelevant,
Thanks Jim - I think im getting it. Sorry to be so dense here, but it may actually be a medicine thing too..... So.... To beat a dead horse more, write to XMP refers more to a format/platform(as stated in the book) than location (unless a sidecar file)and format? As I quoted earlier, all info is written to the catalog and you just confirmed - tks.

Which are prioprietary formats and which are not?
 
Joined
Feb 1, 2010
Messages
12,345
Location
West Sussex, UK
Lightroom Experience
Advanced
Lightroom Version
Yes, they are proprietary (i.e. raw files which are specific to each different camera manufacturer). Non-proprietary files are "open-standard" formats such as Jpeg, Tiff, PSD, DNG.
 
Joined
Jun 20, 2009
Messages
16,057
Location
Houston, TX USA
Lightroom Experience
Power User
Lightroom Version
Classic
Im sorry - those are proprietary or are not?
ANY RAW file that is not in the open DNG format is considered proprietary to the camera manufacturer. Writing into an XMP block in those RAW file leaders would destroy the format integrity of the RAW file format and therefore Lightroom will not write to those source files. DNGs, JPEGs, TIFFs etc are documented public formats and have rules to writing into the XMP section for the file header blocks.
 
Last edited:
Top