problem with Macbook Pro only importing/adding so many images on a Catalogue

[geoffreybillett]

Lightroom Version Number

Classic 10.3

Operating System

1. macOS 11 Big Sur

hello everyone; I come begging at the font of Lightroom knowledge.

I am trying to build a new catalogue in Lightroom, adding approx 2tb of images ( approx 45,000 images in total ) from my NAS, leaving the originals on the NAS. On every import I try, my Macbook Pro ( 250 ssd/8gb ram ) successfully imports around 25,000 images and then stops importing. The previews are minimal, there is plenty of space for the Catalogue to grow on my ssd, and those which do import render well. Does anyone have any thoughts what might be wrong. The same images import without a problem on my Ryzen 9/32gb ram pc. Thanks.

 

[clee01l]

hello everyone; I come begging at the font of Lightroom knowledge.

I am trying to build a new catalogue in Lightroom, adding approx 2tb of images ( approx 45,000 images in total ) from my NAS, leaving the originals on the NAS. On every import I try, my Macbook Pro ( 250 ssd/8gb ram ) successfully imports around 25,000 images and then stops importing. The previews are minimal, there is plenty of space for the Catalogue to grow on my ssd, and those which do import render well. Does anyone have any thoughts what might be wrong. The same images import without a problem on my Ryzen 9/32gb ram pc. Thanks.

First, a point of clarification. Images are never stored in the catalog. The catalog contains a reference to the path to the originals. The originals can be copied or moved to an new location or left in place. I assume that you are adding the images to the catalog and leaving the originals in place on the NAS.
When Lightroom imports images, it creates some temporary files in working storage, and previews stored alongside the catalog. You need to keep about 100GB of your Primary disk free for working storage. And with a 256GB primary disk, it probably take about 25,000 images imported before working storage is full and Lightroom Classic stops working. A MBP with only 256GB of disk space and 8 GB ofRAM is IMO inadequate for importing a large number of images into the Lightroom catalog.


Sent from my iPad using Tapatalk

 

[geoffreybillett]

First, a point of clarification. Images are never stored in the catalog. The catalog contains a reference to the path to the originals. The originals can be copied or moved to an new location or left in place. I assume that you are adding the images to the catalog and leaving the originals in place on the NAS.
When Lightroom imports images, it creates some temporary files in working storage, and previews stored alongside the catalog. You need to keep about 100GB of your Primary disk free for working storage. And with a 256GB primary disk, it probably take about 25,000 images imported before working storage is full and Lightroom Classic stops working. A MBP with only 256GB of disk space and 8 GB ofRAM is IMO inadequate for importing a large number of images into the Lightroom catalog.


Sent from my iPad using Tapatalk

Hi Cletus
Many thanks for your prompt reply. Yes your assumption is correct, but this is strongly alluded to in my original text.

However your explanation makes sense and was something along the lines of what I was also thinking. I checked whether there was a buffer or something I could extend but there appears none. Its more than just slightly disappointing; the lower spec Macbook essentially becomes ineffective for managing moderate to large catalogues, even when the images are stored off-Mac. I guess this memory requirement and restriction would extend across more than 1 catalogue, so there would be no point in trying to import the same number of images across several catalogues. And for people thinking they could use external storage to manage large image collections, this MBP would not be appropriate. It looks like the current penchant for SSDs brings its own limitations. Thanks.

 

[Jim Wilde]

If you break down the import into smaller chunks, you should be able to get all the images cataloged successfully. However, you'll probably have an ongoing issue in trying to manage within that small SSD space. I've just switched from a late-2013 MBP which had a 512GB SSD, and although it would generally operate OK with the LrC catalog on that drive, I'd have to manage the free space carefully....and my catalog was only 20k images. Eventually I got a bit fed up with that situation and moved my catalog to an external USB3-connected SSD.

 

[geoffreybillett]

If you break down the import into smaller chunks, you should be able to get all the images cataloged successfully. However, you'll probably have an ongoing issue in trying to manage within that small SSD space. I've just switched from a late-2013 MBP which had a 512GB SSD, and although it would generally operate OK with the LrC catalog on that drive, I'd have to manage the free space carefully....and my catalog was only 20k images. Eventually I got a bit fed up with that situation and moved my catalog to an external USB3-connected SSD.

Hi Jim
Thanks. So for clarity, it I bought an external SSD it would enable me to catalogue those images without issue? I want a portable collection of the previews for these images, not necessarily to work on, but for quick demonstration purposes. So i guess you would put the catalogue on the external ssd and there would be sufficient space for Lightroom to buffer.

As an aside, I see you have the M1 mini which is something im considering. Do you have issues with catalogue management with that?

Thanks again.
Geoff

 

[clee01l]

Hi Jim
Thanks. So for clarity, it I bought an external SSD it would enable me to catalogue those images without issue? I want a portable collection of the previews for these images, not necessarily to work on, but for quick demonstration purposes. So i guess you would put the catalogue on the external ssd and there would be sufficient space for Lightroom to buffer.

As an aside, I see you have the M1 mini which is something im considering. Do you have issues with catalogue management with that?

Thanks again.
Geoff

I just got an M1 iMac. If you are not using a 2nd monitor, the iMac with the P3 display makes the perfect choice as the existing monitor can be put to used as a second monitor with the iMac.

The key point IMO to preventing issues with catalog management are as follows:
1. You need an adequate working storage area at all times. 100GB is optimum But 15-25% is required for volumes less than 1TB. Apple chose unwisely when that released smallish computers with the 356GB SSD. These machines are designed for people the use computers to browse the web, for email and little else. They are IMO inadequate for computation intensive applications like Lightroom Classic.
2. A machine with 16GB of RAM is necessary for heavy duty Lightroom use.
3. Lightroom will use up to 6-8 cores of CPU and performance is enhanced with more cores are available.

If you need a mobile computer and an iPadPro or iPhone won’t meet your needs, then consider the M1 MBP spec’d as I outlined above. Most people do not need a mobile computer except as a second computer. So the iMac and the Mac Mini make the optimum choices. I think the M1 chip will prove to be a game changer.


Sent from my iPad using Tapatalk

 

[Jim Wilde]

Hi Jim
Thanks. So for clarity, it I bought an external SSD it would enable me to catalogue those images without issue? I want a portable collection of the previews for these images, not necessarily to work on, but for quick demonstration purposes. So i guess you would put the catalogue on the external ssd and there would be sufficient space for Lightroom to buffer.

Yep, that should work just fine. I re-used an internal SSD that I freed up from a Windows desktop and put it in a USB3 housing, and that worked very well. It was easily mobile if I needed it to be (though that was a rarity), and performance-wise it was excellent.

As an aside, I see you have the M1 mini which is something im considering. Do you have issues with catalogue management with that?

Absolutely no issue at all with catalog management. I went with the 1TB SSD model, which is plenty big enough for all my catalogs and previews, and LrC simply flies compared to my old MBP. All the processes which are easily timed for comparison purposes (preview generation, exports etc.) are all at least twice as fast on the M1 Mini....I couldn't be more pleased with the purchase. I had originally intended to get the 2TB SSD model, even though I knew it would be overkill, but balked at the Apple premium (£400 for an extra 1TB).....so with the £400 that I saved, I bought 2 x 2TB Samsung T7 SSDs, one of which now houses my image library.

The only thing I was losing was mobility, but I already can do most things remotely using just my iPhone (all my images are synced with the cloud), in fact there is very little that I can't do away from home even without a laptop. The phone might not be the most suitable device for an extended trip, but I'm about to buy an M1 iPad which will I think let me do 90% of my workflow on the road, the remaining 10% I can defer until I get home.

 

[geoffreybillett]

Yep, that should work just fine. I re-used an internal SSD that I freed up from a Windows desktop and put it in a USB3 housing, and that worked very well. It was easily mobile if I needed it to be (though that was a rarity), and performance-wise it was excellent.



Absolutely no issue at all with catalog management. I went with the 1TB SSD model, which is plenty big enough for all my catalogs and previews, and LrC simply flies compared to my old MBP. All the processes which are easily timed for comparison purposes (preview generation, exports etc.) are all at least twice as fast on the M1 Mini....I couldn't be more pleased with the purchase. I had originally intended to get the 2TB SSD model, even though I knew it would be overkill, but balked at the Apple premium (£400 for an extra 1TB).....so with the £400 that I saved, I bought 2 x 2TB Samsung T7 SSDs, one of which now houses my image library.

The only thing I was losing was mobility, but I already can do most things remotely using just my iPhone (all my images are synced with the cloud), in fact there is very little that I can't do away from home even without a laptop. The phone might not be the most suitable device for an extended trip, but I'm about to buy an M1 iPad which will I think let me do 90% of my workflow on the road, the remaining 10% I can defer until I get home.

Thanks Jim

That is reassuring with the mac mini. The M1 macs seem to certainly perform well. Re SSDs, the M.2s are beginning to get more affordable too. We are lucky to have so many solutions. I'll try to add the images to the catalog on the SSD and should be good to go. My next M1 purchase will definitely have a larger SSD!!

 

[geoffreybillett]

I just got an M1 iMac. If you are not using a 2nd monitor, the iMac with the P3 display makes the perfect choice as the existing monitor can be put to used as a second monitor with the iMac.

The key point IMO to preventing issues with catalog management are as follows:
1. You need an adequate working storage area at all times. 100GB is optimum But 15-25% is required for volumes less than 1TB. Apple chose unwisely when that released smallish computers with the 356GB SSD. These machines are designed for people the use computers to browse the web, for email and little else. They are IMO inadequate for computation intensive applications like Lightroom Classic.
2. A machine with 16GB of RAM is necessary for heavy duty Lightroom use.
3. Lightroom will use up to 6-8 cores of CPU and performance is enhanced with more cores are available.

If you need a mobile computer and an iPadPro or iPhone won’t meet your needs, then consider the M1 MBP spec’d as I outlined above. Most people do not need a mobile computer except as a second computer. So the iMac and the Mac Mini make the optimum choices. I think the M1 chip will prove to be a game changer.


Sent from my iPad using Tapatalk

The iMac colours are nice I use a NEC monitor which I am pleased with, and link my PC and MBP to that monitor. A mac mini would make an ideal companion for the monitor. My main pc is an Ryzen 9 3900x with 32 gb ram - so very capable. Sometimes its easier to casually pp images on the couch whilst watching a film, hence the MBP, and of course for travel. For heavy work the PC is a winner, though Ive just purchased a Fujifilm GFX50r - and the slow down is noticeable with the larger files.

I agree with your comments re smaller SSDs; something I have learnt. I'll just have to be happy with light imaging whilst I travel and process when I return home - at least for now. The Ryzen provides the grunt and multiple cores to make the best use of Lightroom when home. Just a thought, perhaps replacing the MBP with a more powerful Windows laptop. Where does it end

Thanks for your considered thoughts.

 

[clee01l]

Thanks for your considered thoughts.

One more thought. I used to have an MBP. I retired it for a similarly sized iPadPro. I just replaced the iPadPro with an M1 iPadPro and 1TB internal storage. I use it for the front end to may master catalog importing images into the Adobe cloud where they sync with my master Lightroom Classic catalog. I can. Start my initial culling and post processing on the iPadPro when I am too lazy to go upstairs to my office where my iMac Resides.


Sent from my iPad using Tapatalk