Reducing Catalog Size

[dkperez]

I've got a couple LR catalogs that seem to be incredibly large for what's being stored... The personal one has about 3'K images and last time I checked, the catalog was well over 24GB. The client one has around 2'K images and it seems appallingly large too.....

Is there a recommended set of preferences and/or whatever else can be set in LR to provide the smallest possible catalog while still retaining functionality? By this I mean - if I have to wait 1' seconds for a preview to show up and be usable, that's not a usable functionality... So, how get best performance with smallest size?

And HOPEFULLY speed up the whole cataloguing process too, 'cause THAT seems appallingly slow too....

Oh, and the box is an e67'' with 4GB of matched memory, RAID SATA drives in pairs to separate the images and catalogs onto separate spindles, separate from the OS, pagefile, and cache. OS can be either XP64 or XP32.

 

[Victoria Bampton]

Is it the catalog file that's 24gb, or the previews? Likely the previews.

Best preview size will depend on your screen resolution, and how you view the files.

For example, on one of my catalogs, I only ever look in Grid mode if I'm in the Library module, or I'm switching to Develop module (which reads the full res file anyway), so on that catalog I have the smallest lowest quality previews - I don't need anything bigger.

On another catalog, I often view the images in Loupe mode on a 23" high res screen, so those I have set to a large preview and a medium quality.

When you say you're having to wait 1' seconds for a preview to show up, I'm guessing you're in Develop module? The thing you need to understand about that is that it first shows the preview file, then does a quick read of the raw file, frees up the sliders, and then finishes reading the raw file properly. Your preview setting will only affect the very first view you see, and after that it's down to disc read speed, whereas in Library module you are viewing the previews you've already created. Best thing you can do in Library module is to render previews before you start looking at the files, either in the Import dialog or by selecting all and using Library menu > Previews > Render Standard Previews.

 

[ernie]

When you say you're having to wait 1' seconds for a preview to show up, I'm guessing you're in Develop module? The thing you need to understand about that is that it first shows the preview file, then does a quick read of the raw file, frees up the sliders, and then finishes reading the raw file properly. Your preview setting will only affect the very first view you see, and after that it's down to disc read speed, whereas in Library module you are viewing the previews you've already created. Best thing you can do in Library module is to render previews before you start looking at the files, either in the Import dialog or by selecting all and using Library menu > Previews > Render Standard Previews.

Victoria, thank you for that explanation. I have wondered about what was happening during that process in develop. I too have to wait for each pic to quit loading. I don't use the dam function of LR and it is empty other than what I'm working on at the moment, so I thought the wait was just because my pc was a little old and not much memory by todays standards.
But if that wait is only a matter of disc read speed then a new pc wouldn't help me unless I get one with 1',''' rpm (mine is 72'' now). Is that right? If it is, yippee, I don't have to buy a new pc--but boo hoo, I'm stuck with having to wait for every pic to load slowly.
Or do I have it all bass-ackwards and it's something else entirely?

 

[Victoria Bampton]

Well that depends Ernie.... overall system spec does make a difference - I have less than a second to wait for a picture to load in Develop.

Initial thoughts for speeding up the spec you've got though:
1. Put the catalog/previews on a different hard drive from the images
2. Make sure both are on internal, not external drives, ideally fairly fast ones
3. Make sure there's plenty of RAM available - keep an eye on task manager to see if you're maxing out - if so, adding RAM may help
4. Restart the computer now and again. LR doesn't like memory fragmentation.

 

[ernie]

I don't use LR like most folks. I use it mainly just to develop raws, although I'm playing with printing a little more lately. I use another program for my dam tool. I only have one or two folders at a time in LR just while I'm working on them. Once I've developed everything I delete the folder from LR only (not from disc). It still takes 8-1' sec. to load an image, and even if I load one, go to another briefly, then come back to that one it still has to re-load.
I have 1 gig of ram, but that is all my machine will allow. I do system clean-up and defrag about twice a month. My hd is 4' gig and I keep it to only 2' to 3' gig of used space. (but that doesn't make any difference in loading times of images.) Once I edit a folder I get it sent to other bigger drives for storage.
What should my task manager settings be? I have cpu usage and page file usage. I don't know what that is. If all this is getting out of the realm of LR I understand. I just don't want to buy a new pc and be in the same boat.
Yours loads in under a second-heck, I'd be tickled with under 3 seconds. Other than loading everything else works fine. Thanks for your time, Ernie

 

[ernie]

I developed a small folder of raws and watched the task mgr as you suggested. Every time I moved to another pic cpu usage jumped to 1''% while loading. Oh, and no other programs were open. And then when I exported them all you could see the graph jump to 1''% and flat line accross the top while each pic was being processed.
Am I probably right in thinking that LR is just a big user of ram and I just don't have enough?

 

[Braders]

Ernie

2 things made LR almost unworkable to real fast for me.

I had 1gig ram and bumped it to 2 after a couple of months with LR. World of difference.

Then it got bogged down a little as my cat got bigger, so i put all folders and Ircat on an external HD with firewire4'' and she rocks now.

 

[ernie]

Thanks for the reply Brad. I'm afraid I'm just stuck with buying a new pc later this year. My 1 gig of ram is all my machine will take. And since catalog size is not an issue for me, I will just have to juggle my want list around, again...

 

[dkperez]

Thanks for the replies......

My catalog/previews are on a completely separate spindle from the images, O/S, and LR software.

I'm running 4GB of 1'33 matched RAM, on a dual-processor e67'' with SATA II drives in RAID mirror configuration.

It sounds suspiciously like if I want to have a reasonbly sized catalog with a reasonable number of images, AND decent performance, LR isn't going to do the job. As it is, I can't even take the catalog on the road with me because the catalog is so huge it takes a ludicrous amount of time to copy it to the laptop drive.

Am I reading correctly that to reduce the size of the catalog I have to make the previews so small it cripples performance for anything other than viewing thumbnails in the Library? And to have larger previews I get a gigantic, bloated catalog? Is there some setting that'll provide a reasonable medium? What SHOULD my preferences be set to? I'm using a pair of 24" monitors so having a miniscule preview is worthless... What do I set standard preview size, preview quality, and discard times to?

I must confess that I was hoping someone here was going to tell me 1.4.1 is a MASSIVE performance improvement or at worst, 2.' is a HUGE performance improvement... Doesn't sound like this is the case.

 

[Victoria Bampton]

2.' may be a huge performance increase yet David - optimization is the last thing to be done, so we just don't know yet.

If want to take the catalog on the road, consider an external hard drive (it'll be quicker to plug that into your desktop to transfer the catalog) or more likely, just take the part of the catalog that you need, using the Export as Catalog command. You can merge it back in when you get back, without needing to take the whole catalog with you.

Preview quality - what resolution are your monitors set to? And do you need to zoom in Library module all the time, or are you happy for it to re-render the preview when you do need to? Easiest option is to set it to the smallest size, and then flick through the photographs as you usually would in Library module and see how often it tells you it's rendering a larger preview. If it's doing it often, set the previews to the next size up and try again.

 

[johnbeardy]

You can't make an omelette without breaking some eggs, so preview size/speed /quality is an inevitable trade off with catalogue size. It's your choice, and LR does make sure you have options. If the catalogue and preview size worry you, reduce the preview quality, size, and time they're kept. Just in case there's some junk in there, maybe start with a clean slate by deleting the preview folders in Explorer - LR will rebuild them when needed and in accordance with the new settings.

John

 

[kheinemann]

A strange thing I just noticed was that all my missing harddrive space had been sucked into my library Adobe RAW Processing cache. Basically, it looks like LR creates duplicates in that cache - full size 8MB files, adding up quickly to some 4GB of space, while my actual LR catalogue looked nice and slim and manageable. Why is that? My preview size is set to 144' pixels/medium. Even stranger, I backed-up the whole cache folder onto a drive and then got rid of it on my computer, and as I am monitoring it as I am processing images, some cache data is rendered as 5''kb, and other images create 8MB files. Super confusing to me.