Catalogs Backing up the Catalog now takes forever.

[clee01l]

Lightroom Version Number

14.0.1

Operating System

1. macOS 15 Sequoia

I've noticed that in addition to the general slowness of LrC refreshing changes. The Back up catalog process also takes "forever" And in addition to the, Backing up the Catalog and optimizing the catalog there is an additional step of Optimizing the Previews Catalog ( presumably the files "previews.db" and "root-pixels.db" inside the "...Previews.lrdata" folder. ). Do others mirror my experience? While these two files are small, it does add to the Backup process.

 

[Paul McFarlane]

I would assume the Previews part is due to the new Preview Management in Classic (Cache size). However, it's not something I'm experiencing...

(LrC 14.0.1, macOS 15.1.1, Silicon)

 

[clee01l]

I would assume the Previews part is due to the new Preview Management in Classic (Cache size). However, it's not something I'm experiencing...

(LrC 14.0.1, macOS 15.1.1, Silicon)

That would make sense. If LrC removed previews due to cache constraints, then Optimizing the database is probably needed (In SQL that process is called VACUUM ).

 

[Califdan]

I may be foolish, but I only do the integrity check and optimize once every 3 or 4 months, Been doing it that way for maybe 10 years with no problems.

 

[Gnits]

Me too, but always before an upgrade. (ie also 3/4 months ...10 years )

 

[BobT]

I may be foolish, but I only do the integrity check and optimize once every 3 or 4 months, Been doing it that way for maybe 10 years with no problems.

Back in the day, we would compact (as we called it) databases often or things got slow quickly. It's hard to change old habits.

 

[Califdan]

Back in the day, we would compact (as we called it) databases often or things got slow quickly. It's hard to change old habits.

Same here. Ahhhh- the good old days when a good day was any day without a CICS crash or an operator tripping over a cable and pulling the plug out of a disk drive.

But even so, with a Catalog over over 110,000 images where I probably touch less than 0.1% of them on any day, the amount of clutter and debris that could possibly accumulate in the DB each day is infinitesmal (even in those old days) so cleaning house every few months seems adequate - probably evebn that is overkill - and even so rarely results in a detectable change in catalog size or performance.

 

[Gnits]

Side story. I ran IT for a large bank in Saudi Arabia at one stage. We had Stratos fault tolerant machines running a very large ATM network and Tandem fault tolerant machines for a Real Time Gross Settlement ( Saudi Inter Bank) system. I would rarely be in the machine room, but was walking thru one day with the Operations Director. Suddenly…. alarm bells started ringing from multiple places….

A Stratos Engineer was doing routine mtce… and was very nervous doing so. He was checking one of the drives… All drives had live realtime backups. Before he returned the drive under mtce to a live status… he decided to double check everything was in order…. but the idiot opened up the now only live disk system (to compare settings) …… and the whole world instantly crashed.

I have not heard the word CICS for so .so. so long.

 

[clee01l]

…
I have not heard the word CICS for so .so. so long.

Me either. CICS is where I first learned SQL.


Sent from my iPhone using Tapatalk

 

[BobT]

Oh, you young whipper snappers. I'm talking of a time when there was one IBM PC in the building. MS DOS had to be loaded daily from a 5 1/4" floppy and a hard disk was a large box that sat at the end of the desk. You didn't dare bump the desk while the HD was running lest it crashed... and you used archaic language like "lest"

 

[Gnits]

I wrote apps in assembler, manually converted them to machine code , then punched the code into memory and save the resultant program to paper tape. These were very early data entry machines… which saved the data (say timesheets) to cassettes. Hard disks did not exist at that stage… but the tech developed very quickly after that. Interesting times.

 

[Califdan]

The IBM PC (or any PC for that matter) arrived over 10 years into my IT career - my first IT job was in 1973. So in relation to BobT I'm an elder. Not including shcool, my first IT job (programmer - at that time a "developver" was someone building tract homes in the desert) our only input mechanism to the computer was punch cards and the only output was green bar paper or perhaps a 12" reel of magnetic tape. No workstations, no CRT's, no monitor, no 'connected' devices, no internet. You coded on paper forms with a pencil and sent it to the "Keypunch" team, 24 to 48 hours later you got a deck of punch cards tied with a rubberband. You then visually read every cards looking for typo erros and if you found one, there was 1 Keypunch machin in the programmers area for 40 programmers and you could re-punch the card. I once fixed a keypunch error in the machine room using scotch tape and an exacto knife at 3:00 am. You'd then send the card deck along with input data (also on cards) to operatons and the next day would get a print our of your compiled program and couple of pounds of greenbar paper all filled with hexadecimal characters in 8 character blocks - the dreaded DUMP. Oh what fun!

 

[clee01l]

Oh, you young whipper snappers. I'm talking of a time when there was one IBM PC in the building. MS DOS had to be loaded daily from a 5 1/4" floppy and a hard disk was a large box that sat at the end of the desk. You didn't dare bump the desk while the HD was running lest it crashed... and you used archaic language like "lest"

PCs were not in Existence whe I first Started working out of college. The Company that I worked for had 3 IBM mainframes that were in a room with AirConditioning to keep the machines from over heating.

 

[PhilBurton]

Me either. CICS is where I first learned SQL.

The IBM PC (or any PC for that matter) arrived over 10 years into my IT career - my first IT job was in 1973. So in relation to BobT I'm an elder. Not including shcool, my first IT job (programmer - at that time a "developver" was someone building tract homes in the desert) our only input mechanism to the computer was punch cards and the only output was green bar paper or perhaps a 12" reel of magnetic tape. No workstations, no CRT's, no monitor, no 'connected' devices, no internet. You coded on paper forms with a pencil and sent it to the "Keypunch" team, 24 to 48 hours later you got a deck of punch cards tied with a rubberband. You then visually read every cards looking for typo erros and if you found one, there was 1 Keypunch machin in the programmers area for 40 programmers and you could re-punch the card. I once fixed a keypunch error in the machine room using scotch tape and an exacto knife at 3:00 am. You'd then send the card deck along with input data (also on cards) to operatons and the next day would get a print our of your compiled program and couple of pounds of greenbar paper all filled with hexadecimal characters in 8 character blocks - the dreaded DUMP. Oh what fun!

CICS? Wowzers. The fact that actually know what CICS actually did makes me feel like I should be in one of the exhibit cases here. https://computerhistory.org/

 

[PhilBurton]

My first job in tech was for long-gone Shugart Associates. After that I worked for UNIX workstation startup Fortune Systems, also long gone (and best forgotten, run by a bunch of stock market cheats). My resume reads like a whose who of mostly Silicon Valley that have disappeared for one reason or another.

 

[Califdan]

Talk about hijacking a thread......but it seems to be the core group, so why not

Like Phil, I'm in Palo Alto too. and John Ellis is just up the hill in LaHonda. I guess we have our own little LRQ cluster here

When I visited the Computer History Museum in nearby in Mt. View the follks I was with got tired of hearing me say "Yeah, I worked on one of those" each time we stopped to look at some ancient computing tech.

 

[Jim Wilde]

I started my IT career in 1968, operating several IBM 7010s and a new 360/40.

Back on topic, what exactly does "forever" mean, Cletus? I'm not seeing much of any difference when I back up my 20k image catalog. Both optimize and check integrity are always enabled (why not?) and the whole process always takes less than a minute (about 40secs). How big is your catalog now, and how long does the full backup take?

 

[clee01l]

Talk about hijacking a thread......but it seems to be the core group, so why not

It was my thread. Geezers always have "war stories". I'm no exception. I think the initial post was answered quickly. Should I move this over to the "Lounge"?

 

[Thomas J]

I've also noticed recently that exiting LrC with backup and optimization is taking a lot longer than 4~5 months ago.
With 126,000 photos my catalog .lrcat file is nearly 1.5GB and it's now taking 90secs - feels longer !
It used to take about 20secs - barely time to tidy my desk and stand up .......
Windows 10 - both C: drive and D: with current year photos, LrC, Cache and catalog are NVME M2 SSD - the backup is an internal WD HDD.

 

[BobT]

My 1.6GB catalogue took about 1min 14sec to backup and optimise. That seems not a lot different than before.
Windows 11/64, i7 32GM RAM, both C: and D: on 1TB NVMe each.

 

[Thomas J]

Yes I did some more checking - for some reason my internal HDD where I've been keeping my backups has really slowed down.
I need to run more diagnostics to see if I can find a cause.
Saving to an internal NVME M2 my catalog backup with optimization takes 1 minute which isn't so bad but still slower than it was taking 6 months ago ......

 

[clee01l]

I just timed the backup process on my master catalog. The 1.75GB catalog file took 9:27 mins. My Catalog file is on an EHD that has 4TB free and my Working storage has 200GB free.