Import LrC unresponsive for a long time after import

[atolkachev]

Lightroom Version Number

13.5.1

Operating System

1. macOS 15 Sequoia

My two biggest banes in LrC have been sync and import. With sync under control -- thanks in a large part to this forums -- I want to concentrate on import. Is it me, is it them, or is it just what it is.

My average import is 1000 images. I don't think it's considered extravagant. It would have been 2-3 times that if I did not pre-cull images in Photomechanic. I used to import everything I shot into LrC and then cull. Now my first culling pass is outside of LrC, which reduces the number of images being synced to the cloud by a factor of 2 or 3, and that'a good thing. But there may have been unintended consequences.

First, I will describe the behavior and I want to compare notes whether all, most or no one is going through the same. I want to hear from users who consistently import 500+ images at a time.

I have the following settings:

1. Prior to, or immediately after, importing a medium (500) to large (1000+) size batch, I pause cloud syncing. This is to both rule out sync as the cause or contributing factor to the slowness described below and to avoid unnecessary syncing before the initial batch edits are applied and re-trigger sync.
2. All imported images are added to a new collection. The collection is enabled for syncing, but sync is paused as explained above.
3. Build previews: Embedded & Sidecar. This was suggested in an article or video a couple of years ago as a game-changer to the culling process. At the time, switching from photo to photo in a newly imported batch took forever (in Library as well Develop), and culling was a drag. Today, given my use of Photomechanic with its reported creation of XMPs, and presumed improvements in LrC, I am wondering if this setting is a good idea anymore.
4. Build smart Previews: Off. I don't need the initial import to be any more draggy than it is.
5. In Preferences, Use Graphic Processor is set to Custom with all three boxes checked: Use GPU for display, for image processing, and for Export. Other options would be Auto and Off, and I am willing to experiment with those. I don't understand the recommendations along the lines of "if things are slow, turn off hardware acceleration", but I will take experts' word for it.

Once I hit Import, photos show up in the Current Import temp collection lightning fast. Whether JPEG or RAW. Leaps and bounds over what it was 2-3 years ago. But then the misery starts, which was not the case before. I am trying to figure when it started. I would say 4-6 months ago. I have been using Photomechanic for three years. I believe the misery may have started with LrC 13.3, but I am not sure.

Symptoms of the misery.

1. The Fetching Initial Previews progress bar appears frozen at 2%. It is not frozen, and will advance to 5 or even 10% after 15-30 minutes, but appears frozen. I started writing this post after doing an import of 1400 images about an hour ago, and the Fetching Initial Previews has only reached about 10%. Then, at some point, it will magically jump to about 60% and after that will finish within reasonable time. Presumably, it is the Embedded & Sidecar previews it is fetching. Is it doing anything else?

2. Right-clicking any of the imported images and selecting Go to Collection> shows "Not in any collections", even though the collection has been created and can be navigated to.

3. Right-clicking an image and selecting Show in Finder will take 1-2 minutes before Finder opens.

4. Right-clicking an older image and selecting Develop Settings> Copy Settings... will take 30-60 seconds before the dialog opens. And pasting setting onto new images -- forget it, I don't even try it anymore until this phase is over.

5. Dragging an image from the current import to an existing collection does not increment count of the images in the target collection for a while. Something like 1-2 minutes.

6. Hitting rating number key on a newly imported image reports "Set Rating to 3" in an overlay, but the stars do not appear in the Grid Library view, or Loupe view or Develop module for a while. Quite a while. It has been 3 minutes and counting since I hit "3" on an image.

Update: the Fetching Initial Previews progress bar is to about 15%. The import was done 1h 15 minutes ago. This is the slowest I have ever seen it.

Update: the 3 stars appeared on the image in Grid view and long-ago clicked "Copy develop settings" dialog popped up at the same time after about 5 minutes. It is like pipes get unclogged in fits and starts. Fetching Initial Previews progress is to about 22%. The process speeds up toward the end, but it has been 1.5 hour, and it is only a 1400 image import.

7. If I were foolish enough to start editing or (God forbid) pasting settings at this point, the things I would notice would be: edits not taking, edits disappearing, edits flickering in and out repeatedly, empty History, History repopulating after 30-60 seconds, Undo not undoing the last operation, but a random operation from the past, etc., etc. I am not doing any edits until star ratings and pick flags start taking right away, and it seems I am a long way from that moment.

I will definitely try my next import with Minimal instead of Embedded & Sidecar selected for the Build Previews option ("None" is not an option). I may also experiment with turning on Build Smart Previews import option. I used to have it on and ended up blaming it for slow initial import -- this is going back three years, when the slowness happened during the import as opposed to just after as it does now.

Basically, I was willing to hypothetically blame the after-import LrC unresponsiveness (ostensibly due to the slow Fetching Initial Previews) on my use of Photomechanic, which reportedly creates XMPs, plus my choice of using Embedded & Sidecar previews, plus some recent change in LrC.

But I checked my folders and I see that Photomechanic is NOT in fact creating XMPs for JPGs. For RAW, yes, but I am dealing with 1400 JPGs in this import, and this is the longest I have ever seen Fetching Initial Previews take. It is only about 30% done after more than 1.5 hour.

I am happy to blame Adobe for this screwup, but will be happier if something about my setup and my choices explains this behavior and can fix it. I did run First Aid on my SSD drive, and it did not find any issues.

Sorry about another long post. My LrC have been out of commission for close to two hours, so I had nothing better to do.

 

[atolkachev]

50% completion of "Fetching Initial Previews" 2 hours after import of 1400 images. I have to wonder if slow "Fetching Initial Previews" is the cause or the victim (symptom) of whatever else is happening in LrC following an import.

Almost exactly 2 minutes (I broke out the stop watch) from hitting "3" to 3 stars appearing on an image in Grid.

 

[atolkachev]

4 min 15 sec from hitting "3" on another image and the stars appearing in the Grid. Fascinating stuff.

 

[atolkachev]

"Fetching Initial Previews" operation is ostensibly cancelable (little x to the right of the progress bard). I am tempted -- just to see if the the rest of LrC becomes responsive. I have had an experience when trying to cancel an operation actually broke (completely froze) LrC. So I am going to resist the temptation, even though it could be a good test:

- if canceling "Fetching Initial Previews" makes LrC responsive, then "Fetching Initial Previews" is the culprit and should be troubleshooted/experimented with further
- if canceling "Fetching Initial Previews" does nothing to LrC responsiveness, then the slow "Fetching Initial Previews" is just a symptom/victim, same as hitting the "3" button taking 4 minutes to register

 

[atolkachev]

"Fetching Initial Previews" about 75% done. Things are speeding up 2.5 hours into the import. If I were under a deadline to deliver these images, I would be up a proverbial creek.

From hitting "3" to 3 stars appearing on image in Grid view -- 3 min 45 sec.

 

[atolkachev]

From hitting "0" to stars clearing on five selected images -- 3 min 40 sec.

"Fetching Initial Previews" about 80% done, seems frozen again, but is not.

Gathering these stats for myself if nothing else.

 

[atolkachev]

Perhaps the last 2-3 updates were a waste of your and my time, but here is an interesting data point.

"Fetching Initial Previews" completed about 3 hours after the import.

After the completion of "Fetching Initial Previews", hitting 3 on an image in Grid view still takes 3 min 15 sec to register as stars appearing on a thumbnail in the Grid. Makes me thing slow "Fetching Initial Previews" is a victim and not the cause of the slowness that starts at the moment of import and lasts for an indeterminate amount of time.

I am thinking that in the past restart of LrC cleared the slowness. I also remember an Adobe engineer, who was troubleshooting my sync issue with me, complaining that users restarting LrC was creating/compounding sync issues. I somehow did not believe that, but the notion of a restart not being a benign operation stuck with me, so I use it less frequently.

Supposing a restart is needed to clear the sluggishness, what is the appropriate time to execute the restart? At the end of "Fetching Initial Previews"? Right after import? At a random point in time?

Funny (funny?) thing: after I chose to shut down LrC and opted out of the bi-weekly backup, it took 3 min and 10 sec with the spinning color wheel for LrC to shut down. This is consistent with the reaction time to my pressing the rating number keys. It is not like the shutdown took 3+ minutes. It's that LrC did not react to my choosing to shut down for 3+ minutes.

I say import triggers something really nasty inside LrC, and I would like to know what it is and how it can be fixed on our end or whether Adobe is working on fixing it on their end.

I meant to but forgot to check the resources LrC was using before the shutdown. I will it next time I am in this import morass.

After restart

A funny (funny?) thing, again. I was all set to report that "as expected, after restart LrC is responsive", but the reality is that hitting "0" on an image in Grid view took 3 min 40 sec to clear the stars from the thumbnail. So whatever "it" is that starts inside LrC following an import has to be given time to finish. Can't flush the system by restarting.

LrC started at 2.6 Gb of "Real Mem" used, which slowly climbed to 2.73 Gb before jumping to 3.96 Gb even as I type. Sync is still disabled, can't blame sync. Threads are between 113-117. %CPU is between 102% and 113% (interesting math there, would love an explanation of these numbers).

The next question I am going to ask is what is the path off LrC and to LR Cloud or another product for a mid-volume shooter such as myself. I turn over about 100K images a year. I do not need all of them in the cloud. I have 1 Tb of cloud storage with my Adobe plan, which I have been careful not to use up too quickly. It seemed that LrC + LR was working fine, but I just don't understand where LrC is headed with this steady deterioration of usability.

 

[atolkachev]

125% CPU, nice. Quickly back down to 111%. 115 threads (the high was 121). And Real Memory is up to 4.24 Gb after briefly coming down to 3.86 Gb. Needless to say -- all this is without my doing a thing other than watching and occasionally hitting a rating key.

I think the real indicator of usability/responsiveness of LrC is the CPU number. It has to settle (when not touched) to something in the teens or 20s before one attempts to use LrC again. A nice red herring "Fetching Initial Previews" turned out to be. "Fetching Initial Previews" took a long time because LrC is busy, not other way around. I don't think there is any reason to give up on using Embedded & Sidecar previews.

CPU% has been bouncing between 60 and 100 as I write this. Real Mem slightly down to 4.14 Gb. So tired of this. You may be annoyed by this thread, but this has been the past 3.5 hours of my life -- and thank goodness I don't have a deadline.

 

[atolkachev]

132% CPU is the new record.

What. The Actual. Fuck?

 

[clee01l]

I note that you are using Sequoia for your OS. It would not be unexpected for there to be performance issues in the MacOS that might shortly be corrected or improved with a decimal release. I haven'y upgraded to MacOS15 and don't really expect to until I see a LrC v14 too. But then LrC v14 will need some time to find and fix the bugs that usually show up in a dot "0" release.

By referencing GPU, I surmise that you are still running an Intel Mac. While Apple is still supporting Intel Macs, I would think that they are not likely to tune the newest OS for Intel performance if for nothing more than to encourage you to move on.

FWIW, I have never found it necessary to build Embedded & Sidecar prewiews. I have for some time only built "mimimal previews on import. This is always sufficient for initial culling. And as soon as the first imported images appear, I start culling not waiting for the import to finish. Import is a multi threaded process and Culling is easily accommodated as an additional task inside LrC.

 

[atolkachev]

Very sorry to muddy the waters. I picked the latest from the list assuming (with auto update on) I am on the latest. I am in fact on 14.5 Sonoma.

I am on the M1 MacBook Pro.

I used to start culling as soon as first images appeared. That was pre-13.3 and pre- many things. In my today's situation, with star ratings and pick flags not sticking, there is no point in culling until the dust from the import settles. I just want to understand the nature of the dust and to speed up its settling.

 

[atolkachev]

Again, to reiterate a point (perhaps more suscinctly). It used to be that an import of something like 1000 images would take a while, and the little-by-little imported images were immediately workable or at least cullable (ratable, pickable, rejectable). Now the import appears lightning-fast (as in 1.5 minutes for 1400 images), but the imported images cannot be touched -- at least not productively -- for hours. It can't be just my experience.

Can it?

 

[atolkachev]

I took a nice long walk and left LrC alone to exercise its demons. I am back an hour later. CPU is at 104%, Threads went up to 125, and clicking "0" on an image in the Grid to clear the star rating is still not taking an immediate effect.

What can it possibly be doing? Is there a way to figure out? Is this a reportable behavior, or is this the new norm?

Sync is still paused. Don't blame sync. Sync has been #1 problem, but for now sync is working, and I want to keep it out of the equation for now.

 

[clee01l]

That changes things a lot.
I had an M1 iMac. It had only 16GB of Unified memory and was a Slug when I tried to run Denoise.

IMO 16 GB of unified memory is not enough. The other think that can slow everything to a crawl is not having any freespace on the primary drive for temporary files. You need at least 100GB free for Lightroom Classic and other processes, especially if you are importing a large number of images. How much unified memory do you have?

 

[atolkachev]

32 Gb "Memory". Not sure if it is "unified".

150 Gb free on the SSD.

 

[clee01l]

That changes things a lot.
I had an M1 iMac. It had only 16GB of Unified memory and was a Slug when I tried to run Denoise.

IMO 16 GB of unified memory is not enough. The other think that can slow everything to a crawl is not having any freespace on the primary drive for temporary files. You need at least 100GB free for Lightroom Classic and other processes, especially if you are importing a large number of images. How much unified memory do you have?

 

[clee01l]

32 Gb "Memory". Not sure if it is "unified".

150 Gb free on the SSD.

I can't see any thing else that might cause a problem. I can suggest that you might want to go to the Adobe Creative Clous App manager and uninstall Lightroom Classic, then reboot and reinstall.

Therer is also the CC Cleaner app https://helpx.adobe.com/creative-cloud/kb/cc-cleaner-tool-installation-problems.html.

I know it is no comfort to know that your issue is probably related to your computer and what is installed on it. Are you running any antiMalware apps that might interfere with LrC getting access to the filse that it needs?

 

[atolkachev]

I don't think so. This is not where I expected the discussion to go. I had mentioned sluggish post-import behavior in my other threads on my sync issues -- as a side issue. And now I want to talk about it as my primary issue. I gathered an impression that performance issues were widespread in recent versions of LrC. I would like to hear from more people who import ~1000 files and see (or don't see) LrC CPU% go up over 100% for several hours and find that they can't do anything in LrC without waiting for several minutes for the effects of their actions. From assigning stars to pasting edits.

 

[atolkachev]

CPU at 138.5%. Still 125 threads. Memory usage is off the peak and is 3.30 Gb.

 

[atolkachev]

CPU% hit 162.3%. Granted, these are spikes, not steady loads. It goes back down to "only" 100%, then 113%, then jumps around.

What if it's doing face recognition on 200K+ photos? Address lookup? Something else? How would I know? Is there a log of LrC's activity I can review or share with the experts?

Can I ask the question this way: if I am not touching LrC, and sync is paused, and CPU% is reported at 100%+ for several hours, is there a way of knowing what LrC is doing?

 

[clee01l]

Not a widespread issue at all. I for instance have never had any performance problems. And I’m not alone. The people that do have problems are the ones you hear about. Mostly it’s the ones who had sync issues before Adobe fixed the syncing with v13.3.

The rest with little or no problems have no reason to post.

Since you are not syncing, not automatically writing to XMP and I believe not using the face recognition, I think you are having a problem with the installation of LrC my recommendation to remove and reinstall LrC still stands.


Sent from my iPhone using Tapatalk

 

[Thomas J]

My "standard" imports are smaller than yours, around 500x 20mb RAW photos and maybe 2 or 3 times a month 1000+. Even with 1000 photos I'd barely have time to go and make a coffee - time to check email and maybe sip a coffee looking at the cricket scores but minutes rather than 10s of minutes for importing.

Don't have any measured import speed/lag times to quote but my gut feeling for the last few months has been that my LR Classic (LRC) has been slower than it was last year - no changes to my PC hardware - an oldish Intel i7 7700K 64GB NVME M2 and Nvidea GTX 1070 8GB.
Have been updating LRC within 10 days of their initial announcement after checking for the first rash of user reports.

Usually do some random selections immediately after imports - probably a few 5x HDR merges and a few individual shots "enhanced" and then resized down for email posts.
Have never noticed any lag in rating or colour labels or keywording appearing (and sticking OK) and my random selections would likely be spread throughout the import batch.
Until this week I have set standard size previews and auto write to xmp enabled.
My GPU is set to Custom to do anything it is asked to do (LRC does not seem to make much use of GPU but my Capture One whizzes it).

So summarising so far, I certainly don't experience the severe import slowness or subsequent lag(s) that you are having.

My recent big problem has been LRC using and holding onto HUGE amounts of memory when I do some editing - HRD - panoramas - b&w - presets - etc - whilst simultaneously keywording and editing hierarchies in the keyword list.
Memory usage can hit 20+ GB and eventually after say an hour my LRC becomes unresponsive and or completely stops.
Unlike Capture One which releases memory almost immediately after executing a process - my LRC rarely releases peak level memory usage or if if does it does it VERY slowly - like over 15-20 minutes and when left idle.

This week I have disabled auto xmp writing and done a thorough deep clean of my Windows 10 and HD junk files.
Also not anymore attempting keyword and hierarchy editing at the same time as photo editing.
Been too busy to do more systematic benchmarking but my impression is that tidying up has produced a small improvement.

Online browsing shows that there are a small but statistically significant number of Adobe users who do have problems with performance and memory issues but in the worldwide pool of satisfied users I think we get labeled as "whinging nerds".
Of course some problems are related to unsuitable or outdated computer specs - or user errors in setup and use - and the Adobe Forums with other blogs can sometimes provide useful advice or at least a sympathetic ear.

I wonder how the proportion of "problem beset" Adobe LR users compares with say the proportion of dissatisfied Tesla owners?
60 years ago I remember going to the chemist to get my 6x4 black and white photographs and then back to the uni computer lab to sort my Algol punched cards for a batch job run next Wednesday on a valve computer the size of a detached house !

So perhaps things do get better - it just takes time !
I shall watch your thread for updates
Good luck !

 

[atolkachev]

Thanks!

My LrC memory usage climbed uncontrolably -- and yes, probably toward 20 Gb -- when I was experiencing the neverending sync issue. Once I got past that, the memory usage is reasonable (4.5 Gb at the last check), but it is the CPU% usage that is killing me. Not only 138% CPU sounds wrong, it also manifest in a very unresponsive LrC. I gave up waiting for it to settle and did a Publish of 150 photos (still unedited, but pre-rated in Photomechanic as 4 stars during the first pass culling) to SmugMug. This brought CPU% to a cool 200, but the publishing operation finished eventually, and I was able to sheepishly share unedited photos warning the target audience that they are unedited.

Closed the laptop, took a ride home, ate dinner, walked the dogs. Opened the laptop, an LrC is sitting at 8% CPU (jumped to 13% because I said anything), and actions like hitting a rating key result in immediate result as one would hope.

Time to batch edit 1400 photos and then fine tune the best 200 or so almost 10 hours after the import.

I am happy for those for whom this is not the experience. Both because I am a nice guy and want nice things for you, and because it gives me hope.

I will keep asking the question until I hear someone address it or at least validate it: if I am not touching LrC, and it is running at 100%+ CPU in the background for hours, is there a way to find out what it is doing?

 

[atolkachev]

Not a widespread issue at all. I for instance have never had any performance problems. And I’m not alone. The people that do have problems are the ones you hear about. Mostly it’s the ones who had sync issues before Adobe fixed the syncing with v13.3.

The rest with little or no problems have no reason to post.

Since you are not syncing, not automatically writing to XMP and I believe not using the face recognition, I think you are having a problem with the installation of LrC my recommendation to remove and reinstall LrC still stands.


Sent from my iPhone using Tapatalk

Thanks, I think I do prefer hearing that this is an unusual issue. I guess I will try reinstalling LrC.

Should I try disabling use of GPU? I don't know why people being it up a solution to performance issues, but they do.