My Dropbox saga

[pedz]

Lightroom Version Number

Lightroom Classic version: 11.3.1 [ 202204181225-f90ebff5 ]

Operating System

1. macOS 12 Monterey

I'd like to update the folks here on my Dropbox experience on a Mac.

I had all of my raw images in Dropbox with LR pointing to them. I also had a external disk that the images originally came from as well as a backup of that disk. I changed my process so that I would push a few key files such as the previews to Dropbox and then sync (using rsync) from the Dropbox area to my backup disk. This was working fine for a few months. Then I got an email from Dropbox warning that applications might not work correctly after Apple pushed out an update which occurred at update 12.3.

After 12.3 came out, things appeared to be fine. LR seemed happy. I then ran my script to push things to Dropbox -- that went fine. And then sync the Dropbox copy to my external hard drive. That's when things went awry. After the 12.3 update, the files in the Dropbox area appeared to have zero length. This was true using `ls` from the command line as well as looking at files via the Finder. rsync got confused and zapped all the files on the external hard drive to be zero length. I managed to stop it before it completely finished but there was already 14,000 files that had been zeroed out. Yes... they were still in the Dropbox cloud (as well as my second backup drive) but it upset me pretty bad.

I had already begun the process of moving to BackBlaze B2 because I can not figure out Dropbox's pricing schedule. They have 2TB and they sorta kinda have 3TB but then things get weird and you have to sign up with a different type of account to get above the 3TB limit. At least, that is my interpretation. There are many threads about the topic on the Dropbox community forum. BackBlaze has a pricing structure that is transparent. Also in the works already was me purchasing a large NAS. I have about 100 old disk drives from 6G up to 4TB of backups and random junk that I've collected over my 60+ years. I wanted to dump all of those on one big NAS and the go through and find the useful stuff and delete the other stuff.

The NAS arrived. I asked it to pull down all of my Dropbox data -- which it was happy to. I copied it to another directory on the NAS and ask the NAS to pump it back up to BackBlaze -- which it is still in the process of doing. Meanwhile, I mounted the new area via SMB, pointed LR to the new area, and then asked LR if there were any missing photos. It reported back that there are no missing photos (see attached).

As a test, I imported about 163 images from a Canon R5. Each one about 40MB. The import took about 15 minutes via WiFi. I'm hoping I can figure out some way to have a local cache so I can import at native disk speeds and then bleed the images over to the NAS or to BackBlaze as bandwidth permits. This will be especially important when I start traveling but I know those problems can be solved one way or another. There are things to explore like rclone, FUSE, and UnionFS which I've not done yet.

I've not entirely stopped drinking the Dropbox Kool-Aid but I hope to be completely off of it soon.

To be clear and fair, Dropbox didn't lose any data but it's current implementation on the Mac scares me. That plus the non-obvious pricing structure is why I'm leaving it.

 

[PhilBurton]

@pedz

A few years ago I was working on a personal interest project with a few other photographers. We were supposed to use Dropbox as a central repository for all our photos.

I like to think of myself as (more or less) technically adapt with Windows and cloud-based applications. Most of my career was in software product management and I've also needed to work with MacOS, UNIX (long time ago) and now Linux.

I found the Dropbox user interface confusing and not intuitive. I had to struggle with uploads and downloads all the time. And I was the "smart one" compared to the other people in this project, who were "average skills" Windows and MacOS users, so I became the "support guy" for this project, a most scary thought.

I couldn't get away from Dropbox fast enough. There are other online storage services, although DropBox offers much mroe free storage than competing services.

 

[Gnits]

I only use Dropbox when collaborating with others. I never keep original data in Dropbox folders. If I wish to share anything via Dropbox, I copy the necessary files and folders from my original location to the dropbox folder location. If people are sharing data with me via dropbox, I copy the data from Dropbox to a location of my choice, which works within my drive / folder / file structure and backup strategy.

In other words, at any time, I can afford to lose all my dropbox folders and their contents and not have to worry about it. I regularly audit my dropbox folders to remove folders or files which no longer need to be shared. My dropbox folders should only contain data which is actively been shared.

I adopted this approach after spending months cleaning up a dropbox mess, deciding that Dropbox was a useful tool for sharing data, but I only regard such data as a temporary convenience. This was triggered by the need for me to return a laptop to a supplier and I needed to make sure I had left none of my personal data on the laptop.

 

[Johan Elzenga]

I had all of my raw images in Dropbox with LR pointing to them. I also had a external disk that the images originally came from as well as a backup of that disk. I changed my process so that I would push a few key files such as the previews to Dropbox and then sync (using rsync) from the Dropbox area to my backup disk. This was working fine for a few months. Then I got an email from Dropbox warning that applications might not work correctly after Apple pushed out an update which occurred at update 12.3.

You did not read that message very carefully. What Dropbox explained was that the ‘online only’ option (where the local copy is not a full file but a kind of alias) will have the problem that other applications cannot automatically download the file when needed. The Finder can however. If you do not use the ‘online only’ option, then there is no incompatibility between MacOS and Dropbox.

 

[pedz]

You did not read that message very carefully. What Dropbox explained was that the ‘online only’ option (where the local copy is not a full file but a kind of alias) will have the problem that other applications cannot automatically download the file when needed. The Finder can however. If you do not use the ‘online only’ option, then there is no incompatibility between MacOS and Dropbox.

Yes. If the file is local and consuming space, then Dropbox can make a backup copy but that isn't what I want / need. My laptop can't hold all the files all at the same time.

 

[Woodbutcher]

FYI, watch for some oddball behavior if you use "Edit In" option to send files to photoshop or other external editors when the original is on the NAS. I gave up on the NAS as primary storage because the return from the external editor was a adding a new, non-aliased folder name (newer mac os masks folders to sandbox data from the OS). It would put the new file in the folder with the original on the disk, but LR wouldn't see it there unless you did a Sync folder, and then you had to clean up the bad folder path too.

I can't remember if that was only Monterrey or the later revs of the previous version. Maybe it is fixed now, but it was a bad interaction with the OS and LR. I got tired of dealing with it and got a 2TB ssd that I velcro to the laptop and it holds my images from previous months. Current month is internal to laptop. I rsync to the NAS as one backup now. The other backups go offsite.

 

[pedz]

FYI, watch for some oddball behavior if you use "Edit In" option to send files to photoshop or other external editors when the original is on the NAS. I gave up on the NAS as primary storage because the return from the external editor was a adding a new, non-aliased folder name (newer mac os masks folders to sandbox data from the OS). It would put the new file in the folder with the original on the disk, but LR wouldn't see it there unless you did a Sync folder, and then you had to clean up the bad folder path too.

I can't remember if that was only Monterrey or the later revs of the previous version. Maybe it is fixed now, but it was a bad interaction with the OS and LR. I got tired of dealing with it and got a 2TB ssd that I velcro to the laptop and it holds my images from previous months. Current month is internal to laptop. I rsync to the NAS as one backup now. The other backups go offsite.

I hope I can come up with a system where the last month or two are local to the laptop and the old files get pushed off to some backup somewhere — NAS, BackBlaze, etc. Right now I’m focused on getting my NAS cleaned up and then I’ll start the search for a nice clean way to have effectively a network file system with a large tunable local cache.

Worse case I think I can use FUSE and write user level apps to pump files back and forth like I want but surely someone else has already done that work. … Yea… I know. “Stop calling me Shirley!!!“

 

[Gnits]

I keep the current years images on a very fast M2 Pcie drive, all previous years on a Thunderbolt 3 enclosure and all backups on a Nas drive. Every Jan, I move the previous years images to the Thunderbolt enclosure and create a new folder for my New Years images on the M2 drive. I do not even need to change my backup settings. I am exploring Backblaze or Amazon Glacier as a cloud repository as another layer of protection, but would be scared to use Dropbox as a component of my backup architecture.
I just reviewed today the amount of images from previous years which needed to be backed up and they were a tiny fraction of my overall image storage. I also checked what images in my backups were genuine orphans and could be removed. Again a tiny fraction, almost not worth the effort.

I adopted this approach when I knew I would be away for a month or so and needed to work from a laptop, but was pleasantly surprised how useful this approach has turned out to be. I recognise individuals will evolve a workflow which works best for their need.