Programming extensions to Lightroom

[clee01l]

I'm not sure if this is the right section to put this topic. I'm confident the moderators will relocate it if necessary.

This is a question to the Plug-in developers. I need to get a better handle on Exports, image versioning and repeatability. It would seem the SQL Server database and/or metadata added to the image file would be places to extend the functionality of LR.

Specifically, this is what I would like to do: [list type=decimal]
[li]I need to track export versions[/li]
[li]I need a snapshot of the edit history at the time of export to repeat the export at some future date.[/li]
[li]I'd like to use named Develop Presets that would permit repeatability for making new exports of earlier exports. [/li]
[li]I'd like to embed these named presets in the keyword list [/li]
[li]I would like to manage this without having to leave the Develop module or the Export Module[/li]
[/list] It would seem that a Plug-in could access the database and perhaps the image metadata to 'automagically' track this information and up the database or my tables in the database as well as update the Original image file metadata.

Before I start digging deeper into the SDK and learniong the Lua Scripting language, I would like to know how much of this is possible and if there are other resources that can help me become a skilled Lightroom developer.

 

[johnbeardy]

I've already explained that the way to do most of this is by using a virtual copy - it has all your settings, which you can copy to other images, so there is no need to have Develop presets. Every VC has a Copy Name field, so name it appropriately and there's no need to pollute the keyword list with non-descriptive metadata. But if you really think it's worth inflicting significant pain upon yourself....

Regarding your 5 questions, I mentioned in the other post how to track export activity via an export plug-in. However the SDK has no access to anything in Develop, so no history steps (hence rely on the VC) or presets. The SDK only allows read access to keywords. The database is SQLite - not SQL Server - and the SDK does not have direct access to it. While Lua itself could access the database via ODBC, that will probably result in corrupting the database because it is single user. SDK events can update certain values that are stored in the database. So much of what you want to do is better done by existing non-SDK features, and can't be done via the SDK anyway. Still crazy? Well....

The Lua SDK is all you need, or rather it is all there is. It is written for programmers in very technical language, and all the non-Adobe Lua reference material is even more arcane and substantially irrelevant. Most of the documentation, and the error messages, are the stuff of a Kafkaesque nightmare and will only help you once you already know what they mean. To get an idea of the experience, read this http://records.viu.ca/~Johnstoi/kafka/beforethelaw.htm

Sorry to sound discouraging. I found Lua a very tough nut to crack, as it's a niche language unlike any others I'm familiar with, such as VB, JS, AS, PHP, XML, XPATH, SQL stored procedures.... One coder who works with device-embedded software and is a trained programmer calls it a horrid language, among other things. What's more, other than for use with LR, it is a dead end - its only other application seems to be something called World of Warcraft whatever that may be. If you still haven't decided that life is too short for such punishment, my suggestion is to go through the SDK sample files, figure them out, tweak them to do something you find useful, and then go back and see if the SDK makes more sense. The struggle itself...is enough to fill a man's heart.

John

 

[Tim Armes]

[quote author=johnbeardy link=topic=7191.msg48936#msg48936 date=124734762']
The Lua SDK is all you need, or rather it is all there is. It is written for programmers in very technical language, and all the non-Adobe Lua reference material is even more arcane and substantially irrelevant. Most of the documentation, and the error messages, are the stuff of a Kafkaesque nightmare and will only help you once you already know what they mean. To get an idea of the experience, read this http://records.viu.ca/~Johnstoi/kafka/beforethelaw.htm

Sorry to sound discouraging. I found Lua a very tough nut to crack, as it's a niche language unlike any others I'm familiar with, such as VB, JS, AS, PHP, XML, XPATH, SQL stored procedures...
John
[/quote]

Hmm,

I think it's only fair to give another point of view.

Like most other languages, you're ability to adapt to Lua will depend on your past programming experience. I personally got to grips with Lua withing a few days - there isn't that much to learn. There are a few oddities, but once you've understood them they don't represent any great technical hurdles.

The Adobe SDK is dead easy to learn, but it's very limited, as John said. You'll often need to think out of the box, and you'll hit a few bugs from time to time.

Tim

 

[Gene McCullagh]

And I'll take the middle ground

I don't think it's as hard as John describes (although I think he is waxing poetic to make a point 8) ) and I don't think it's nearly as easy as Tim writes.

The concepts in Lua and the SDK are pretty standard. However, Lua is a very unique language and I really wonder why Adobe went down that road. A more traditional language could certainly have worked. But, not being a Lua expert (yet! :icon_twisted: ) I suppose there are aspects about the language that made it the perfect choice.

Hopefully now that there has been a fairly decent display of plugin developers in the community, Adobe will open up more functionality in the SDK for version 3.

All in all, if you have an interest then it will be worth the effort to get this Lua thing into your head! Just remeber Rule #5 and you'll be OK!

 

[johnbeardy]

"I really wonder why Adobe went down that road"... because Mark was into it and wanted to show what Lua could do. OK, that's a bit sharp, but there's a grain of truth there too.

Folk in the middle of the road tend to get run over! I do stand by my assertion of the degree of difficulty, which struck me as significantly harder than AppleScript or ActionScript3 which I've also learnt in the last year or so. Both these have a huge amount of helpful documentation, whereas as soon as you want to go off-piste in Lua you're into a tiny programmers-only world. Even within the SDK, I didn't even mention the complete absence of code examples for a very large proportion of the SDK's features. It's not a learning curve, more like that El Capitan in Yosemite - damn easy and the documentation all makes sense once you've got up there, but a horrible vertical climb before then.

John

 

[Gene McCullagh]

Hey John! You won't get run over if you stand on the grassy median!

 

[Tim Armes]

[quote author=Gene McCullagh link=topic=7191.msg49152#msg49152 date=12476''648]
And I'll take the middle ground

I don't think it's as hard as John describes (although I think he is waxing poetic to make a point 8) ) and I don't think it's nearly as easy as Tim writes.
[/quote]

I never said it was easy, I said it depends on your past programming experience.

I'm a software engineer and I've programmed in many types of languages, so I didn't find it hard to adapt. If you have little experience with this sort of language then the learning curve will be steeper.

Tim

 

[clee01l]

From this and other threads, I've determined that Virtual Copies and Named final presets will get me most of what I want. What remains are these two bullets"

• [li]I'd like to embed these named presets in the keyword list [/li]
  [li]I would like to manage this without having to leave the Develop module or the Export Module[/li]

Over the years I've developed applications in a number of languages from COBOL, and FORTRAN to C++, Java, Coldfusion, PHP, and most of the scripting languages. All languagee seem to differ mostly in Reserved words and the syntax of conditional statements like "IF, THEN, ELSE" etc. If my two remaining bullet points can be done, I think I can figure out the lua to make it work.

 

[johnbeardy]

I did answer those questions in my 2nd paragraph, Cletus. SDK access to keywords is read-only, and you have no access to any Develop information such as presets, so on two counts you can't write presets to the keywords - even if that was a smart idea anyway. The second point becomes moot, doesn't it?

I'll give you an example of why it's a bad idea to pollute keywords with non-descriptive metadata. The other day I was adding a feature to someone's export plug-in and noticed that the plug-in incorrectly exported keywords marked in LR as not being for export. Sooner or later, if you're too clever with keywords and use them for things other than describing an image, something will catch you out.

John

 

[clee01l]

[quote author=johnbeardy link=topic=7191.msg49231#msg49231 date=1247677'85]
I did answer those questions in my 2nd paragraph, Cletus. SDK access to keywords is read-only, and you have no access to any Develop information such as presets, so on two counts you can't write presets to the keywords - even if that was a smart idea anyway. The second point becomes moot, doesn't it?

I'll give you an example of why it's a bad idea to pollute keywords with non-descriptive metadata. The other day I was adding a feature to someone's export plug-in and noticed that the plug-in incorrectly exported keywords marked in LR as not being for export. Sooner or later, if you're too clever with keywords and use them for things other than describing an image, something will catch you out.

John
[/quote]I meant to amend that bullet with "...keyword list or some other appropriate metadata field." I think the Job Identifier IPTC field may be the best and most appropriate location to store the final Develop Preset.
To that extent, I have already created an Export Template that uses Jeffrey Friedl's Flickr template and places everything I need in the Flickr Caption using tokens:
{Caption}<br>SourceFile: {FILENAME},{CopyName}<br>Preset: {JobIdentifier}

So, now all that remains is a mechanism for extracting the Named Preset from wherever Lightroom stores it and putting it into the JobIdentifier field of the IPTC meadata where the Export Template can do it's thing.
Here is an example of a file that I uploaded to Flickr. It contains all the valid information in the Flickr description (Caption)
http://www.flickr.com/photos/cletuslee/3723832769/

Now This Plug-in does not have to run in the Develop Module , but could be a part of the Export Module Perhaps as an enlargement to Jeffrey Friedl's Plugin

 

[johnbeardy]

If you want to do that you'll have to enter the preset manually since, as I also said before, you can't access any Develop information. You could capture Edit Time, but that would include Library changes too, and you could also capture the current time which you might subsequently compare to the history steps.

Also, as I understand it, the Develop preset is calculated dynamically. If the adjustments match a preset, then that preset's name is displayed in the drop down box. That can happen by applying the preset, or by dragging the sliders to matching values.

Of course, providing you just leave the VC alone after creating it for the export, you're done - you've got the edit state @ export preserved. And you have the Copy Name too, which you can target in Smart Collections. Isn't your method going to leave you with one Develop preset per export?

John

 

[clee01l]

[quote author=johnbeardy link=topic=7191.msg49248#msg49248 date=1247687954]
If you want to do that you'll have to enter the preset manually since, as I also said before, you can't access any Develop information...
Also, as I understand it, the Develop preset is calculated dynamically. If the adjustments match a preset, then that preset's name is displayed in the drop down box. That can happen by applying the preset, or by dragging the sliders to matching values.

Of course, providing you just leave the VC alone after creating it for the export, you're done - you've got the edit state @ export preserved. And you have the Copy Name too, which you can target in Smart Collections. Isn't your method going to leave you with one Develop preset per export?

John
[/quote]In the Library, I can filter on Develop Presets. The result is either a Named Preset, Unavailable Preset, Default Settings or Custom. It would seem inefficient for Lightroom to calculate each image to see if it matches a Named preset to include it in a filter. Since the Library works on the Catalog and not the image, it is my guess that there is a Preset field in the database. If you note one of the filter choices was 'Unavailable Preset' When I run this filter on all my 23K photos, I get several singlets and doublets listed as 'Unavailable Preset' When I checked these images in Develop, I find that the Named Preset in History was a Named User Preset that I deleted. Lightroom still knows the Name even though it no longer appears in the Preset Dropdown list.

As for one preset per export. I might create more preset templates than the typical Lightroom user, but not that many. After a study of the free downloadable prersets, it is apparent that theres really are only a handful (1''s rather than 1'''s) of useful preset adjustments. Tone curves being one area where ther can be a lot of variation. For a given day shoot, I may only use two or three. So, I think I can keep my preset list under 1''.

 

[johnbeardy]

It'll be a background process, run after making the adjustment rather than when the filter calculates.

John

 

[Mark Sirota]

[quote author=clee'1l link=topic=7191.msg49246#msg49246 date=1247687'68]
So, now all that remains is a mechanism for extracting the Named Preset from wherever Lightroom stores it and putting it into the JobIdentifier field of the IPTC meadata where the Export Template can do it's thing.
Here is an example of a file that I uploaded to Flickr. It contains all the valid information in the Flickr description (Caption)
http://www.flickr.com/photos/cletuslee/3723832769/

Now This Plug-in does not have to run in the Develop Module , but could be a part of the Export Module Perhaps as an enlargement to Jeffrey Friedl's Plugin
[/quote]

You could do this using Jeffrey's Run Any Command export plugin, using exiftool to extract the caption and then running it through a little script to parse the text, then back to exiftool to stick it into JobID.

 

[clee01l]

[quote author=Mark Sirota link=topic=7191.msg49474#msg49474 date=124797119']
[quote author=clee'1l link=topic=7191.msg49246#msg49246 date=1247687'68]
So, now all that remains is a mechanism for extracting the Named Preset from wherever Lightroom stores it and putting it into the JobIdentifier field of the IPTC meadata where the Export Template can do it's thing.
Here is an example of a file that I uploaded to Flickr. It contains all the valid information in the Flickr description (Caption)
http://www.flickr.com/photos/cletuslee/3723832769/

Now This Plug-in does not have to run in the Develop Module , but could be a part of the Export Module Perhaps as an enlargement to Jeffrey Friedl's Plugin
[/quote]

You could do this using Jeffrey's Run Any Command export plugin, using exiftool to extract the caption and then running it through a little script to parse the text, then back to exiftool to stick it into JobID.
[/quote]I believe you are thinking of the process backwards. I tried to think of a way to implement the 'Run Any Command' plugin, but the way I have it working in the Flickr Plugin, it already resides in the Job ID. The Flickr Plugin just appends other fields on to the Caption field for the Flickr 'Description' Currently, I am manually writing it to the JobID. Until I do that, it does not exist in the metadata. I think it only exists in the catalog. Because in the Library, I can run a filter on Develop Preset. There I can select a unique preset and manually update the JobID field in that filtered result. The only way I know to find which preset was used to create which image for export is to run this Library Filter

 

[Mark Sirota]

Yes, sorry, looks like I misunderstood.