Slow LR 6.14 Catalog Backup Problem

[marinephoto]

Lightroom Version

6.14

Operating System

1. macOS 10.13 High Sierra

Awhile ago I began to experience painfully slow Catalog backups so I posted here in search of advice. By painfully slow I mean that when exiting Lightroom after clicking the checkboxes for Optimize Catalog and Backup Catalog it would take several hours for the process to complete. My Library contains around 75,000 images and the Catalog at the time was 13.7 GB in size. People responding said that was a very large catalog size for 75,000 images so I asked if an extensive Develop History could account for the large size and slow backup speed, Folks said it could so eventually I made a duplicate catalog for experimenting with and deleted the Develop History from all 75,000 photos, This reduced the catalog size from 13.7 GB to 2.34 GB. I decided that I wanted to retain the Develop History for my 4 ✩ or better images, so I did a sort for all images less than 4 ✩ and deleted the Develop History from all but 2,500 of the 4 ✩ or better images. This resulted in a catalog size of 5.79 GB. Hoping that this would result in a much faster catalog backup time I Optimized and Backed Up the modified, now much smaller, 5.79 GB catalog, but to my dismay it still takes more than an hour to complete. I don't know precisely how long because I've never had the time to sit there and wait. Neither do I know just how long it should take to complete a catalog backup.

Some had mentioned that this slow backup could indicate a problem with catalog corruption but I have never gotten any error code, nor have I seen any other problem that might indicate catalog corruption. It did occur to me today, however that the ESET Cyber Security software I am running might be playing a role in this??? Does anyone know if this might be the case? If not, any idea what the problem might be causing such slow catalog backups? I am using a 2017 21.5" Retina Screen iMac 3 Ghz Intel Core i5 Processor with 16 GB of memory, running High Sierra 10.13.6 My equipped with a 1 TB Fusion Drive with 375 GB available and I am saving the catalog to that internal drive

Thanks!

 

[clee01l]

I forgot to mention I am having a problem with slow catalog backup. I deleted Adobe and reinstalled but it is still very slow.

Often a slow backup or other slowness with Lightroom Classic can be attributes to a lack of free space on the primary disk Lightroom creates a lot of temporary files and these reside in working storage in the free space on the primary disk (C:\ for Windows). I recommend keeping at 20- 25% of the primary disk one at least 100GB free for working storage to hold temporary files from Lightroom Classic and other apps.


Sent from my iPad using Tapatalk

 

[debi.lightcraft]

Often a slow backup or other slowness with Lightroom Classic can be attributes to a lack of free space on the primary disk Lightroom creates a lot of temporary files and these reside in working storage in the free space on the primary disk (C:\ for Windows). I recommend keeping at 20- 25% of the primary disk one at least 100GB free for working storage to hold temporary files from Lightroom Classic and other apps.


Sent from my iPad using Tapatalk

My catalog is on the hard drive of my Imac. It has 718.98 GB available of 1TB. (My photos are all stored on external hard drives. ) So I don't think the problem is space. But I read that large develop histories can cause slow back up and I am wondering how to delete histories without affecting the developed photo and if that is a good idea?

 

[Jim Wilde]

Hello I am a new member. Would you be able to explain how to trim develop histories without the photo being affected? Thank you.

Select the photos that you want to remove the Develop history from, switch to the Develop Module, then from the Develop menu click on "Clear History". Before you do that, however, take a catalog backup just in case you change your mind and want to revert.

Removing the history does not affect the images at all, but not every user likes clearing it (personally, I do it routinely and it doesn't bother me at all).

 

[Jim Wilde]

But I read that large develop histories can cause slow back up and I am wondering how to delete histories without affecting the developed photo and if that is a good idea?

Yes, large develop histories can indeed be the cause of a "slow catalog backup", though there are other reasons as well of course. How big is your catalog, both in the number of images and the size on disk (just the .lrcat file, excluding all the other catalog-related stuff like previews)? As I said above, it won't affect your photos....but only you can say if it's a good idea, i.e. if you often work within the history panel, such as doing "before and after" comparisons, or using the history to step back in order to "undo" edits, you might think about only deleting the history for "finished" older photos.

But unless you do an awful lot of spot removals or frequently make extensive use of the brushes and other selective tools, clearing the history might not achieve very much in terms of the catalog backup. The sizings might give us a better idea.

 

[debi.lightcraft]

Thank you Jim for your reply. I will look these things up and get back soon.

 

[LRList001]

Some data.

I am running LR 6.14 under Windows 10 latest, 64 bit.

I have some 115k images in the LR catalogue.
The catalogue is 1.6GB
The backup (which as a compressed file crudely indicates how much 'unique' data is actually in there) is 170MB.
Exiting LR, I always prompt for a full optimisation and backup, usually I select 'yes', this takes about a minute.
The catalogue is on an SSD, separate from the main C drive, which is also an SSD. The image files are on local, moderately fast HDDs.

All images will have history, though not much on many of them. I couldn't say how many have had lots of developing, it looks like a few thousand. I believe it is local adjustments that are heavy on catalogue size, I generally avoid having lots of local adjustments, as I find local adjustments are not one of LR's strong points.

HTHs

 

[debi.lightcraft]

Yes, large develop histories can indeed be the cause of a "slow catalog backup", though there are other reasons as well of course. How big is your catalog, both in the number of images and the size on disk (just the .lrcat file, excluding all the other catalog-related stuff like previews)? As I said above, it won't affect your photos....but only you can say if it's a good idea, i.e. if you often work within the history panel, such as doing "before and after" comparisons, or using the history to step back in order to "undo" edits, you might think about only deleting the history for "finished" older photos.

But unless you do an awful lot of spot removals or frequently make extensive use of the brushes and other selective tools, clearing the history might not achieve very much in terms of the catalog backup. The sizings might give us a better idea.

Some data.

I am running LR 6.14 under Windows 10 latest, 64 bit.

I have some 115k images in the LR catalogue.
The catalogue is 1.6GB
The backup (which as a compressed file crudely indicates how much 'unique' data is actually in there) is 170MB.
Exiting LR, I always prompt for a full optimisation and backup, usually I select 'yes', this takes about a minute.
The catalogue is on an SSD, separate from the main C drive, which is also an SSD. The image files are on local, moderately fast HDDs.

All images will have history, though not much on many of them. I couldn't say how many have had lots of developing, it looks like a few thousand. I believe it is local adjustments that are heavy on catalogue size, I generally avoid having lots of local adjustments, as I find local adjustments are not one of LR's strong points.

HTHs

Yes, large develop histories can indeed be the cause of a "slow catalog backup", though there are other reasons as well of course. How big is your catalog, both in the number of images and the size on disk (just the .lrcat file, excluding all the other catalog-related stuff like previews)? As I said above, it won't affect your photos....but only you can say if it's a good idea, i.e. if you often work within the history panel, such as doing "before and after" comparisons, or using the history to step back in order to "undo" edits, you might think about only deleting the history for "finished" older photos.

But unless you do an awful lot of spot removals or frequently make extensive use of the brushes and other selective tools, clearing the history might not achieve very much in terms of the catalog backup. The sizings might give us a better idea.

My LR catalog has 30,773 images. The .lrcat has 1.72 GB.

 

[debi.lightcraft]

My LR catalog has 30,773 images. The .lrcat has 1.72 GB.

My LR catalog has 30,773 images. The .lrcat has 1.72 GB.

Yes, large develop histories can indeed be the cause of a "slow catalog backup", though there are other reasons as well of course. How big is your catalog, both in the number of images and the size on disk (just the .lrcat file, excluding all the other catalog-related stuff like previews)? As I said above, it won't affect your photos....but only you can say if it's a good idea, i.e. if you often work within the history panel, such as doing "before and after" comparisons, or using the history to step back in order to "undo" edits, you might think about only deleting the history for "finished" older photos.

But unless you do an awful lot of spot removals or frequently make extensive use of the brushes and other selective tools, clearing the history might not achieve very much in terms of the catalog backup. The sizings might give us a better idea.

Yes, large develop histories can indeed be the cause of a "slow catalog backup", though there are other reasons as well of course. How big is your catalog, both in the number of images and the size on disk (just the .lrcat file, excluding all the other catalog-related stuff like previews)? As I said above, it won't affect your photos....but only you can say if it's a good idea, i.e. if you often work within the history panel, such as doing "before and after" comparisons, or using the history to step back in order to "undo" edits, you might think about only deleting the history for "finished" older photos.

But unless you do an awful lot of spot removals or frequently make extensive use of the brushes and other selective tools, clearing the history might not achieve very much in terms of the catalog backup. The sizings might give us a better idea.

Jim Wilde--sorry for all the repeats above. I'm just getting used to this forum. To answer your question, MY LR catalog has 30,773 images. The .lrcat has 1.72 GB. Does this suggest anything about the slow performance of the backup?

 

[Jim Wilde]

Jim Wilde--sorry for all the repeats above. I'm just getting used to this forum. To answer your question, MY LR catalog has 30,773 images. The .lrcat has 1.72 GB. Does this suggest anything about the slow performance of the backup?

It's difficult to say for certain. Yes, the size of the catalog is a little larger than might be expected, but not exceptionally so. For comparison, my catalog of 21k images occupies around 670MB, but that contains no develop history at all.

I can't see anywhere that you say how long the catalog backup it taking, you only say it's slow. So how long is it typically taking? And what drive type are you backing up to?

You could delete the history to see if it makes a difference, but take a backup of the catalog first so that you can restore it should clearing the history make no difference.

 

[debi.lightcraft]

So I am backing up to the hard drive on my 2017 retina 4k with 1 terabyte storage 8 GB Ram imac. I keep my photos on separate external hard drives. It takes a couple of hours to do the full backup.
Should I be unplugging the external hard drives when I do the back up? Does it matter one way or the other?

 

[debi.lightcraft]

So I am backing up to the hard drive on my 2017 retina 4k with 1 terabyte storage 8 GB Ram imac. I keep my photos on separate external hard drives. It takes a couple of hours to do the full backup.
Should I be unplugging the external hard drives when I do the back up? Does it matter one way or the other?

Also I should add I know 8 gigs of ram is not optimal, but I am of the impression it should not affect performance this much.

 

[Jim Wilde]

So I am backing up to the hard drive on my 2017 retina 4k with 1 terabyte storage 8 GB Ram imac. I keep my photos on separate external hard drives. It takes a couple of hours to do the full backup.
Should I be unplugging the external hard drives when I do the back up? Does it matter one way or the other?

"A couple of hours" is simply not right for a backup of a 1.7GB catalog. Can you tell from the backup process if any of the stages seems to take up most of that time?

Where does the catalog reside? If it's on the same internal drive that you are backing it up to, then that's something you should consider changing (a system drive crash means you've lost the catalog and the catalog backups).

Unplugging external hard drives shouldn't have any effect on the speed of the catalog backup.

 

[LRList001]

"A couple of hours" is simply not right for a backup of a 1.7GB catalog. Can you tell from the backup process if any of the stages seems to take up most of that time?

Where does the catalog reside? If it's on the same internal drive that you are backing it up to, then that's something you should consider changing (a system drive crash means you've lost the catalog and the catalog backups).

Unplugging external hard drives shouldn't have any effect on the speed of the catalog backup.

When I exit LR (6.14) I check and optimize the catalogue and back it up. The total time is a minute or two. Of that time, the initial checking takes a good proportion of the time and the progress bar barely moves, LR then goes to optimise various other aspects and right at the end of the various processes, backups up the catalogue. If the progress bar is to be believed, the actual backup of the catalogue takes a very few seconds. It is all the different optimisations that take the time.

 

[clee01l]

If the progress bar is to be believed, the actual backup of the catalogue takes a very few seconds. It is all the different optimisations that take the time.

Often orphans database records got left behind. I don't know if the number and quality of the plugins play a part in this or not. Optimization and Consolidation might take some time to work thru these but they do not go away.
My solution is to open a new empty catalog and user the "Import from another catalog" option to import the master catalog into it there by creating a new master catalog . My reasoning is that the Import won't copy the orphans database recored because that are not included in the referential integrity of the imported images.

When you delete images from Lightroom there are empty spaces in the database records. Optimization rewrites the data base and collapses the empty space resulting in a smaller file.