Better support for community use cases (UI dialogs, parameters, etc.)

[imagejan]

Several projects, after using SciJava parameters and UI at first, switched back to custom, IJ1-backed implementations because SciJava couldn't offer all they needed.

I suggest to try to collect here some use case examples illustrating the "community demand":

• ilastik4ij - IlastikImportDialog (see e.g. the discussion in Drag and drop support for the hdf5 import ilastik/ilastik4ij#71)
• bigDataProcessor2 - HDF5DatasetDialog and the classes implementing AbstractProcessingDialog
• ...

@k-dominik @tischi if you find the time to summarize the reasons why you had to develop your own UI, what was missing in SciJava etc. (maybe with a screenshot of your current dialog), that would be great!

[k-dominik]

Dear @imagejan,

our main concern was usability. First we wanted to get away from this "two step" approach, where you select a file in one dialog and then open a second dialog to adjust your settings we had previously. We wanted to auto-populate fields with datasets, such that users wouldn't have to know about internal paths, if they import a file exported from ilastik that usually only has a single internal path but also be flexible enough for the case of multiple datasets. Also axistags are read and updated accordingly. All with the goal of staying macro backwards-compatible.

So in short, more dynamic "parameters" that are updated on other parameter changes were the main reason.

CC: @m-novikov

[tischi]

Dear @imagejan,

I wrote a whole novel this morning as an answer and now I see it did not save it 😦

Well, I try to reproduce:

First of all I ❤️ SciJava commands. They are very useful! Also the IJ1 dialogs don't do the job for me so I am using a combination of Swing and SciJava Commands.

Main aim essentially is to both have a concise and interactive UI and macro recording!

Towards this aim currently I think there are the following limitations in SciJava Commands:

1. When one resolves input fields programmatically they are not macro recorded, thus the UI cannot preset these fields without destroying the recording.
2. One cannot programmatically preset fields with suggested values, which the UI state may know.
3. If a Command is Interactive there is no macro recording.
4. There are no help buttons (like the ? in CellProfiler) for each parameter.

And there is one more specific request: Sometimes I only want to record the Command macro without actually running it. The reason is that I increasingly develop code where the ImageJ UI just serves to produce a macro that can then be batch executed on a computer cluster.

Thanks for opening this thread and I am looking forward to future developments!

[m-novikov]

To expand a bit on @k-dominik answer.

In Scijava case initial implementation of the dialog is convenient and simple, it becomes increasingly more complex once you want to do any kind of non-trivial interaction between widgets. On the other hand swing documentation and examples are readily available and it generalizes better for our use cases allowing more fine-grained control over UX.
ilastik4ij project is about providing simple and consise way to integrate imagej with ilastik, so good UI is main feature of this project.

Some examples of what we want to do in the future:

• Add list selection with recently created ilastik project files
• Generalized command for ilastik project file that adds elements to form based on the type of the project.
• Better validation for fields in the relation to project file (e.g. provide a list of datasets matching selected project file dimensions, show the red border, and hint if axis tags don't match shape)
• I want to have my dialog entries to be grid-aligned 👀

Some problems that we hit:

• Impossible to have optional parameter with combo box containing Fiji datasets (e.g. choices should be "", "Dataset 1", "Dataset 2")
• Syncing datasets between ij1 and scijava is delayed
• Using Command facilities for dynamic forms, field population not always triggers form update

One approach to solving some of our problems would be to provide more customization for dialog fields and more hooks in command, but then it would be trying to do the job of the proper UI framework which I don't think is a good route.

I think scijava does a really good job to autogenerate dialogs, but at some point, if it's deemed important enough by the project it would be nice to be able to generate form yourself while preserving command structure with paramters annotations.

[tischi]

Another question I have is whether programmatic access to Command parameters could be improved.
Currently, I think, this needs to be done by via a parameter map nameToValue = Map< String, Object >(), where the String must be the exact name of the Java variable.

To make this typo safe what I am now almost always doing is:

@Parameter
private String yourName = "Tischi";
public static String YOUR_NAME_PARAMETER = "yourName";

Like this I can access the YOUR_NAME_PARAMETER, using above map, without having to remember (or check) the exact spelling. Obviously this is tedious and does not look very pretty. But I don't know whether technically there is any better solution.

[tischi]

...I don't know if that would work in terms of the whole SciJava architecture, but technically I could at least imagine something like:

class myCommand implements Command
{ 
   @Parameter
   public String yourName;
}

and then

mc = new MyCommand();
mc.yourName = "Tischi";
commandService.run( mc );

So the commandService would run an actual instance of the Command and thus have access to potentially preset parameters.

[frauzufall]

@tischi just a note here, I use the ModuleService for this, it let's me access inputs and outputs programmatically:

CommandInfo commandInfo = commandService.getCommand(MyCommand.class);
Module module = commandService.moduleService().createModule(commandInfo);
module.setInput("input", img);
module.resolveInput("input");
commandService.moduleService().run(module, true);

[tischi]

ok, but still this is neither typo nor type safe and quite verbose.
if at all possible I would thus still think a solution like below would help the adoption of SciJava commands by a broader community:

MyCommand command = new MyCommand(); // concise 
command.yourName = "Tischi"; // typo and type safe 
commandService.run( command, true );

[tischi]

@frauzufall

What would happen in this code?

CommandInfo commandInfo = commandService.getCommand(MyCommand.class);
Module module = commandService.moduleService().createModule(commandInfo);
module.setInput("myName", "Deborah");
// module.resolveInput("myName");
commandService.moduleService().run(module, true);

Would that do anything to the myName field? I am asking because of my above wish to be able to preset parameters with suggested values (without resolving them). Maybe one can already?

[frauzufall]

@tischi actually, this works too (EDIT: true for DynamicCommands, not all Commands):

MyCommand mc = new MyCommand();
mc.setMyName("Tischi");
commandService.context().inject(mc);
commandService.moduleService().run(mc, true);

The resolveInput method, as far as I know, can be used e.g. by the input harvester to figure out if this is a parameter that should be displayed as a widget or not. So you can set the value, and you can also choose if the user can change it or not. But please test it ^^

[tischi]

OK, so maybe some things are actually possible...
I think this asks for some sort of learn-/hack-athon to explore these things together in an efficient way, with all the knowledgable people around (@frauzufall, @imagejan) to help and make suggestions what to try. I think one day could be enough for this. After such a day we would probably know much better what the status and wish-list is.

[frauzufall]

Happy to meet (virtually)! I have not seen you all for too long :) Anything we come up with should become part of the howto's. In case you have a minute, maybe you can make a PR with this snippet :) Otherwise I will do it later.

[tischi]

In case you have a minute, maybe you can make a PR with this snippet :)

I think that's partly my point: this is much more efficient during a small hackathon.

Happy to meet (virtually)!

Do you feel like organising this? I could help with providing e-mail addresses of people that may want to join.

[tischi]

@imagejan @frauzufall
I have another question/request: Is it possible to suppress macro recording of Commands in scripting mode?
Normally that results in those IJ.run(...) recordings.

My use-case is that I had to implement custom recordings of jython code using the actual Java API of my code, which works, but now it records both (i) my jython code and (ii) the IJ.run(...) calls. Do you know if it is possible to suppress the IJ.run(...) recordings? Please note that I am calling the Commands programmatically, so I could use something like ModuleService in case this helps.
Thank you for your help!

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/access-to-imagej2-input-harvester-generated-elements/49338/3

[imagejan]

Relevant discussion also in imagej/imagej-legacy#239