Need to delete JPGs if there is a corresponding Raw file

[gwwinaz]

Lightroom Version Number

14.5.1

Operating System

1. Windows 11

My ultimate goal is to delete JPGs that have a corresponding raw file, generate JPGs from the raw files (updated when the raw file is editied, and have all my JPGs in one folder structure.
The JPGs are used in making movies using non-Lightroom software and viewing directly on my TV by accessing them on my NAS configured on my LAN.
My cameras have evolved over the years and I still use two that can not create raw files, only JPGs. The raw files are from three or four different cameras. My raw files started in 2015 shooting RAW+JPG.

I have three high level folder structures all imported into Lightroom:
LR\Pictures\Year\subject and some LR\Pictures\subject
LR\Pictures ARW\Year\subject and corresponding LR\Pictures ARW\subject
LR\Pictures ORF\Year\subject and correspoinding LR\Pictures ORF\subject.

In Pictures there are about 6,000 scanned slides (JPGs) and about 14,000 JPGs that have matching 3D MPO files in LR\Pictures\3D\year\subject. There are a total of 62,000 ARW files and 2,000 ORF files. Total JPGs is about 130,000.

I have Folder Publisher and Collection Publisher in addition to other plugins.

The first part is removing the matching JPGs, a one time process. The second part is having the published or exported JPGs and the JPG only files in one folder for each year\subject, hopefully without duplicating JPGs. When raw pictures are edited then the published or exported JPG will need to be updated, ideally automagically.

I don't know how to accomplish my goal. I could repeat a process by year but not by subject because there are just too many. Suggestions much appreciated!

 

[Califdan]

For Part 1, See: How to remove redundant JPG's from RAW/JPG pairs (Revised 2019). Written for case of RAW+JPG pairs imported as one image but many sections may be applicable to situation where the JPG and RAW were imported as seperate images.

If RAW's and JPG's were imported as seperate images, can try this

1. Select all photos (expand all stacks or otherwise assure that all images are shown)
2. Filter for ONLY JPG's
3. Mark all with a color label (say "Red)
4. Turn off Filter
5. Sort grid by either file name or capture time
6. you should see a pattern of "red" images (which are the Jpg's) intermingled with the RAW images. Delete the JPG's that have a matching RAW

 

[John R Ellis]

To find JPEGs that have a corresponding raw in the same folder, you can use the Any Filter plugin's Search command with this query:

The plugin has a 30-day free trial, so a one-time deletion would be free.

 

[gwwinaz]

Thanks for the link to the very interesting article. If I understand the steps correctly I would have to do this for 222 folders. That isn't practical for me.

I do have the Any Filter plugin. However, currently the raw files are not in the same folders as the JPGs and there are 222 folders with raw files.

Good ideas that may lead to a solution.

 

[clee01l]

I do have the Any Filter plugin. However, currently the raw files are not in the same folders as the JPGs and there are 222 folders with raw files.

John's Any Filter works on the master catalog, The location of your images in the file system should be irrelevant.

 

[John R Ellis]

Unfortunately, the query I posted only identifies JPEG/raw pairs residing in the same folder (that's how the Associated File Types criterion is defined). But gwwinaz's JPEGs reside in separate folders from their associated raws.

 

[Califdan]

Any reson the OP can't drag the JPG's to same folder where the RAW's are (using the LrC Folders Panel of course)?

 

[clee01l]

Any reson the OP can't drag the JPG's to same folder where the RAW's are (using the LrC Folders Panel of course)?

Since the purpose is to delete the Jpeg and remove them from ALL folders. I think this is an excellent suggestion.

 

[gwwinaz]

If I were to merge the raw and jpegs folders, wouldn't I need to do that individually for all 222 folders that have raw files? I also still need generated jpegs from the raw files for videos and the tv. I'll have to think about any complications for this approach. When accessing a folder from the tv that has raw files, they display as a black screen when doing a slideshow of a folder.

My solution may be a ChatGPT created program to walk the arw hierarchy matching to the jpeg hierarchy and moving any matched files to another directory as a temporary backup. I'm testing the program today. Then I just have to tell Lightroom to forget those missing files.

 

[Gnits]

Just a thought… and I am conscious that everyone has a unique view on the best way to structure their image library.


For at least the last 10 years I have structured my Image folders as follows (ie by Year and sequential projects within year).
2025_Photos
2026_Photos
.. 1234_DawnShootDublinBay
.. 1235_TripToWestMayo
.. 1236_ClassReunion

When I started using LrC ( first beta).. my structure was a real mess. In the early days I used to import Raw and Jpg. I evolved to using jpgs when travelling and raw got imported when I got back to base. For at least the last 10 years I ignore camera jpgs and they do not get imported to LrC or my disk drive.

Any raw files edited via Photoshop round trip are saved in the same project folder as the original raw. The same applies to Pano Merges, etc.

All derived images (eg jpgs for email / web, 16 bit tiffs for printing, etc), are stored in suitable named subfolders of the original raw / psd. This is so simple to do using Export presets. If images are created for a person or service then the appropriate named is included in the subfolder name

So… a project folder structure might look like…

1235_TripToWestMayo
.. WebEmailJpgs
.. 16BitTiffsforXyzPrintingServices
.. SmallJpgsForJoeBloggs.

The reason I outlined this approach is that you might consider this as a basis for your work going forward. Use the main project folder to store your raw / psd images, keep your jpgs in a subfolder of this (either camera or exported jpgs) and keep other sub folders as required.

The main reason I do this is that by reviewing the folder hierarchy for a project folder… I have an instant summary of the key outputs from that project folder. Also.. each project is in chronological sequence per year…. so it is a trivial exercise to find specific projects and images within those projects.

 

[gwwinaz]

Thanks for the continuing ideas and suggestions. They are very helpful.
Update: I now have a Python program that will walk all my raw folders and move any matching jpeg files to a different matching folder structure. I can then use Find All Missing Photos to remove those jpegs from the library.

I haven't found any way to easily transfer the raw files into the Pictures file structure with the remaining jpegs except by individually selecting and dragging the photos for each of the 222 lowest level folders. I have other ways of doing that but LC wouldn't know about the moved files. I haven't decided if this should be done subfolder by subfolder.

I may just leave the raw files in their own structure but I still would like to have a folder structure with all jpegs. Originals from cameras without raw capability and jpgs generated from the raw images so they will include any editing of raw files. Unless I can find a way to have these published jpegs in the same folder as the original jpegs, I'll have a lot of wasted storage. My experiments with the Folder Publisher plugin shows I could publish them into the same folder as the original jpegs. LC wouldn't know about them but that would be ok for my video program and tv. I think it would make collections very confusing.

A thought: If I ran the Python program, then used folder publisher to publish new jpegs, they would be in the LC library.

What a tangled web .....

 

[clee01l]

I haven't found any way to easily transfer the raw files into the Pictures file structure with the remaining jpegs

In Lightroom You can filter on RAW filetypes and Select and drag these into the appropriate folder in the folder panel.
It helps if your folder structure is data based. Then you can filter the RAW filetypes by date as well

 

[gwwinaz]

Progress update: It has been a bit of a struggle but I have removed all the jpegs that have a corresponding raw file and migrated all my raw files into my Pictures folder with other jpegs. The biggest problems were cause by inconsistent naming between my Pictures folder and my Pictures ARW folders. While I tried to keep the structures consistent, I missed a few. Also, way back in 2019 I tried to rename images from my A77 because the sequence number rolled over after 9999 even though the counter showed 5 digits. I made mistakes then that caused some duplicate images with different sequence numbers. I may have lost some of my keywords because I didn't realize there were the folder naming issues until long after I had run the Sycomatic plugin although I could still recover them by putting the old jpegs into a different folder structure, importing, and running Sycomatic again.

I used a custom program generated by ChatGPT to remove the jpegs if there was a matching raw file and my backup program, Syncback to move the raw files into the pictures hierarchy.

I think all of the naming issues and duplicates have been resolved!

The step that is left is to generate jpegs from the raw files to use in my video program and for TV viewing. I think I'm going to use Jeffery Friedl's Collection Publisher.

 

[gwwinaz]

Final Update: I have used Collection Publisher for each year that has raw images and published them into a FldrPub folder. For other year folders and subject folders without any raw images, I used my Syncback SE software to mirror the jpgs to the FldrPub folder. I then copied them onto a portable SSD drive for TV viewing.
I also had ChatGPT right another program that extracted the date taken from all of my jpgs taken with various phones over the years. They were in folders only by year. The program compared the date taken with folders in my yearly folders for a subject with the same date (I use yyymmdd subject for folder names). If found then it moved the jpg to the subject folder. if the jpeg was already in the subject folder it moved the jpg to a duplicates folder. I then removed missing pictures from LR and then imported for all of my yearly folders.
This has been a long process but the organization of my pictures is much better. I don't have to create mirror folders for JPG and raw, and if I edit an image the edits will be included in the jpgs I view on TV or use in movies.
A final comment is that I kept a baseline archive before any changes and kept a log of each step and backed up LR frequently in case I messed anything up.

 

[Califdan]

Long journey. But now that you have just the (edited) RAW's in LrC we can look at the second part of your original post.

If I understand, your final goal is to a) have and edit your original images (mostly RAW's) in LrC. b) Have folders on disk containg JPG versions of some of those edited RAW files that you can show on your TV (presumably your better images).

If that is correct, consider using a "Hard Drive" type PS (Publish Service):

1. Shot only and import only RAW's
2. edit RAW's at will
3. Create a "Hard Drive" Publish Service" for the JPG's you want to show on your TV
4. Create collectioons in the PS by as desired
5. Add images to Pub Service Collecitons (drag and drop) or have those collections be smart collecitons (e.g. images in a particular Folder and have more than 3 stars)
6. click "publish" (button at bottom of left column, not shown in my screenshot)

Example:

Every so often, click on the folders in the Publish Service and see if any have been updated in LrC. If so, hit the Publish button at the bottom of the left column (not shown in screen shot). This will update the folder on disk with the new version. Likewise remove a photo from the collection in the publish service and hit "publish" and it will be removed frolm the folder on disk.