Why local space disk gets occupied by LR while importing to external drive?

[Snoop]

Lightroom Version Number

6.12

Operating System

1. macOS 10.14 Mojave

Hello,

Want to see if someone can help a bit understanding an issue I'm facing with LR 6.12 on Mac (latest OS).

Basically, I did set the main catalog, cache and other elements on an external disk and beside the application and some files attached to it, nothing is stored to the local disk. When I make an import, all files go to external drive and I did check it several times. Same for .lrcat and .lrdata and backup files, etc., they are all on the external drive too connected with an USB3 cable.

My problem is as follow: when I make a big import, I see the space on local drive decreasing fast... as I don't have that many space left on my local drive (50 to 80 GB), it gets saturated to the point were LR becomes unresponsive and OS informs me that there is almost no space left on the local drive and it is true... only 200 or 300 MB are free. However, when I take a look to the pics that got imported, they are all located on the external drive.

In order to recover LR or Mac, I need to force Quit LR and thrash stuff on the local drive to recover space. However, even when I empty the thrash, it doesn't recover the space lost while making the import... and local files related to LR tend to stay same volume (around 2.2 GB in total).

I don't get why my local drive is getting "charged" while importing on the external drive and it is driving me nuts.

Do you have any idea or recommandation to understand/fix this issue?

Thanks in advance for your feedback!

 

[Linwood Ferguson]

You might want to see if Mac has a program that will show you actual file activity. You could find out the name and locations of the files that are taking up space, and maybe get closer.

I do know that I have monitored the file movement for importing from cards, and it was... well, all I can say is silly. LR created a ton of separate folders under a parent, copied the files to those folders, then copied the files from those folders to their final destination, with a bunch of intervening activity I forget now. It was horribly inefficient. In my case the temporary copies and final copy were on the same folder so the extra copy was really redundant (vs a move), but maybe they simply always do it and you are seeing those temporary copies.

Maybe.

I also know it is different (at least on windows) whether you import from the cards as devices or as folders (don't ask me why, or even what that means -- as a programmer I would presume they were the same). I've even seen Adobe documentation recommending one over the other (I forget which; I don't let LR import from the cards personally).

Bottom line, at least as of a few versions ago, there's a lot of weird stuff LR does during import. It would not surprise me if it gets confused where to put some of these temporary copies, but to really find out I think you need to find the files involved, and post or ask Adobe.

Oh... and I do Windows, I can't spell Mack, so it may be an Apple thing somehow also.

Linwood

 

[clee01l]

All operating systems and most app make use of Working storage. This is in the free space usually located on the boot drive. (In your case MacIntosh HD). Freespace needs to be reserved for these "temporary" files and for the Swapfile that the OS creates and maintains to free up RAM when running apps require more RAM than is available or is being efficiently used.
The general rule of thumb used to be ~15% but as Disk drives became larger And more modern apps were making extensive use of temporary files, a new recommendation for maintaining free space was developed that suggested 100GB free. Lightroom is one of the apps that makes extensive use of free space, especially on import. The larger the import, the more free space required. Lightroom copies the image file to working storage, then creates intermediate files there while doing the initial processing before moving the original file from the temporary location to its final destination.

If you have a fast SSD with 500GB or less, there is barely enough room for the OS, apps and Working storage for data intensive apps like Lightroom. I have a rMBP with 8GB of RAM and (currently) 123GB of a 500GB SSD available. It often runs into problems importing a large number of 36mp RAW image files from my camera card

 

[Snoop]

Thanks for the feedback.

The main problem I see is that even if while importing, LR make some local cache stuff, it should delete it at the end or when I quit the app. But that is not the case and that says a lot about the qualities of the software. It is a vicious circle, more I import pics, less space I have on my local disk while everything is stored on an external drive! It is absurd.

Anyway... I'll continue investigation.

 

[clee01l]

The main problem I see is that even if while importing, LR make some local cache stuff, it should delete it at the end or when I quit the app.

Working storage falls under the domain of the OS. It is refreshed only when you reboot. Programs including LR do delete files created there when these files are no longer needed by the app. The problem with things like large imports is that LR may need more space to hold all of the files it creates during the import than is available in the free space available to working storage. Also the OS limits apps to a fixed number of file handles which is an OS issue and not a free space issue. When an app can't manage its files and functions due to space constraints, then it loses it ability to handle the housekeeping tasks such as deleting files it no longer needs.
Some programs (and LR is one of them) have memory leaks that cause the available RAM to get consumed and not released. Bug fixes often are attempts by the developer to deal with these memory leaks. Adobe does a pretty good job of finding and fixing these though usually not in the ".0" release.

 

[Linwood Ferguson]

I wonder, and I realize you said no, but are you sure you do not have some portion of Lightroom's file system on that drive, like preview cache, ACR cache (though I would think that would stabilize over time), etc?

While I agree completely with Cletus, I also do not think that is very likely to happen every time you do an import (if indeed that is what you observe), but rather be the exception when it gets errors that cause it to lose track.

I think the key to solving this is figure out what the files are. I do not speak Mac, but some possibilities:

- Previously mentioned: a program that monitors for file activity, if such exist.
- Run a complete directory to a disk file of every file on that disk with sizes, and do so before and after an import, then do a differences (does Mac have a diff program? Can you do sdiff or just diff from the command line?)
- If that's too voluminous, can you run a disk usage report by folder, down a couple levels and see which changes? In linux it might be something like

du -h --max-depth=2 / | sort -h

Once you know which folder(s) are changing you could then do it again and go deeper and deeper until you find where they really are.

I think if you can find the files in question it will quickly become obvious what is causing it, if you are leaking archived files/data into that drive unexpectedly, or if it is garbage left behind. It might also be something like log files or debug information, did you hever have debug turned on for Lightroom trouble shooting, where it might be building a lot of logs up?

But find the files, and you will solve the mystery.

Linwood