User-configurable Keymaps

While developing the BDV Preferences dialog, a “pattern” has emerged of how we wire up the shortcut and action definitions. This tutorial explains the current recommended way of doing that. We give some background about using ui-behaviour etc. Feel free to just skip to the end for the recommended pattern.

Introduction

In BigDataViewer 10.4 we added a Preferences dialog. This makes settings more user accessible, that previously could only be made through editing config files. In particular, users can now easily override BigDataViewer keybindings to their liking.

It is also possible to define and switch between multiple sets of keybindings. For example, in Mastodon, we have predefined keymaps that have * basic BDV key bindings, but many shortcuts remapped to navigate along a cell lineage, or * full BDV key bindings, at the expense of more complicated shortcuts for cell lineage navigation.

On top of these users can define their own completely customised keymaps.

This is all based on ui-bahaviour, which several tools (BDV-based and otherwise) already use for managing shortcuts. While developing the Mastodon Preferences dialog, and now carrying over to BigDataViewer, a pattern has emerged of how we wire up the shortcut and action definitions. It would be great if this would become a blueprint for actions in other tools, because a) that will make the code easier to understand and b) facilitate reuse of action definitions across projects.

We work towards the recommended pattern, from scratch, in a series of examples that you can also find on github.

%%loadFromPOM
<repository>
    <id>scijava.public</id>
    <url>https://maven.scijava.org/content/groups/public</url>
</repository>
<dependency>
    <groupId>sc.fiji</groupId>
    <artifactId>bigdataviewer-core</artifactId>
    <version>10.4.3</version>
</dependency>
<dependency>
    <groupId>org.scijava</groupId>
    <artifactId>ui-behaviour</artifactId>
    <version>2.0.7</version>
</dependency>

Setting up shortcuts through ui-behaviour

Lets look at a basic example of integrating ui-beahviour in a AWT/Swing application.

We need a minimal application to play with: MainPanel is a JPanel containing (only) a single JLabel displaying the text "hello". The displayed text can be changed by the setText(String) method. We will use this to define different mock “actions”.

/*
#| include: false
*/
import java.awt.BorderLayout;
import java.awt.Dimension;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.border.EmptyBorder;

public class MainPanel extends JPanel
{
    private final JLabel label;

    public MainPanel()
    {
        setLayout( new BorderLayout() );
        setBorder( new EmptyBorder( 0, 20, 0, 0 ) );
        setFocusable( true );

        label = new JLabel( "hello" );
        add( label, BorderLayout.CENTER );
    }

    public void setText( final String text )
    {
        label.setText( text );
    }
}

Let’s instantiate a MainPanel and show it in a JFrame.

var frame = new JFrame( "Keymaps Demo" );
var panel = new MainPanel();
frame.add( panel );
frame.setPreferredSize( new Dimension( 200, 100 ) );
frame.pack();
frame.setVisible( true );

MainPanel showing text “hello”

/*
#| include: false
*/
import javax.swing.JComponent;
import javax.swing.SwingUtilities;

To set up ui-behaviour for the panel, we first need an instance of InputActionBindings

import org.scijava.ui.behaviour.util.InputActionBindings;

var bindings = new InputActionBindings();

InputActionBindings bind inputs to actions.

This is of course exactly what AWT/Swing’s Key Bindings framework (InputMap, ActionMap) does. InputActionBindings adds very little over that; basically only more convenient InputMap chaining.

Side note: The initial purpose of ui-behaviour was to offer a similar framework for mouse clicks, scrolls, drags, etc. Modeled after InputMap and ActionMap, there are InputTriggerMap and BehaviourMap. Analogous to InputActionBindings there is TriggerBehaviourBindings.

Anyway, we connect the InputActionBindings instance to our MainPanel as follows.

SwingUtilities.replaceUIActionMap(
    panel,
    bindings.getConcatenatedActionMap() );
SwingUtilities.replaceUIInputMap(
    panel, JComponent.WHEN_ANCESTOR_OF_FOCUSED_COMPONENT,
    bindings.getConcatenatedInputMap() );

InputActionBindings manages a chain of InputMap/ActionMap pairs. An Actions object encapsulates one such pair, and feeds new action definitions into it. We create a new Actions (the constructor arguments don’t matter for now) …

import org.scijava.ui.behaviour.io.InputTriggerConfig;
import org.scijava.ui.behaviour.util.Actions;

var actions = new Actions( new InputTriggerConfig(), "demo" );

… and we add the pair to our InputActionBindings under the name “actions”.

actions.install( bindings, "actions" );

(We could use the name later to remove, replace, or temporarily block the InputMap/ActionMap pair.)

The actions instance is now connected to the panel via bindings. We can finally use it to add new shortcuts.

actions.runnableAction(
    () -> panel.setText( "Action A triggered" ),
    "Action A",
    "SPACE", "A" );

The actions.runnableAction method takes the following arguments

public void runnableAction(
    final Runnable runnable,
    final String name,
    final String... defaultKeyStrokes )

1. A Runnable to run when the action is triggered.
2. A unique name for the action (this will be used as the actions key in the underlying InputMap/ActionMap.
3. Zero or more keystrokes that should trigger the action.

Here for example, the Runnable sets the text “Action A triggered” in the panel label. It is added under the name “Action A”, and triggered by the “SPACE” key, or the “A” key by default. The syntax for key strokes is described here.

Let’s add a few more actions.

actions.runnableAction(
    () -> panel.setText( "Action B triggered" ),
    "Action B",
    "B", "shift B" );
actions.runnableAction(
    () -> panel.setText( "Action C triggered" ),
    "Action C",
    "1", "2", "3", "4", "5", "6", "7", "8", "9", "0" );

Now we can use these defined shortcuts to run these three actions (which will change the text label to “Action A/B/C triggered”.

You can find the full example on github.

Making shortcuts configurable

Another goal of ui-behaviour is to make mouse and key bindings easily configurable by the user (for example through config files).

This is the purpose of the Actions constructor arguments

var action = new Actions( new InputTriggerConfig(), "demo" );

The first argument is a InputTriggerConfig, and after that one or more String contexts are given (more on that later).

The InputTriggerConfig contains is basically a map from action names to key bindings. When adding a new action, for example like this:

actions.runnableAction(
    () -> mainPanel.setText( "Action B triggered" ),
    "Action B",
    "B", "shift B" );

then actions will first look into its InputTriggerConfig to check whether any key binding is associated with the respective action name (“Action B”). If nothing is defined in the InputTriggerConfig then (and only then) the specified default key bindings will be used ("B" and "shift B").

Loading shortcuts from a config file

So far, we just used a new, empty InputTriggerConfig, meaning we just get the specified defaults, which is exactly what we want for prototyping. If the project becomes more mature, and we want to change the config from outside, we can load the InputTriggerConfig from a config file.

import org.scijava.ui.behaviour.io.yaml.YamlConfigIO;

Reader reader = new FileReader( "config.yaml" );
var config = new InputTriggerConfig( YamlConfigIO.read( reader ) );

The config.yaml file looks like this:

---
- !mapping
action: Action A
contexts: [demo]
triggers: [SPACE, A]
- !mapping
action: Action B
contexts: [demo]
triggers: [N]

The format should be more or less self-explanatory.

The loaded config should now map the String "Action A" to the Set of Strings {"SPACE", "A"}, and "Action B" to {"N"}. We could set up actions with the loaded config in the constructor, and then define the same actions as in the previous example.

Alternatively, we can just update the existing Actions with the new config.

actions.updateKeyConfig(config, false);

The config contains bindings for “Action A” and “Action B”. These will override the specified default bindings. So “Action A” will be triggered by the “SPACE” or “A” keys, and “Action B” will be triggered by “N”.

The config doesn’t specify anything for “Action C”, so that will be triggered by the programmatically specified defaults, that is, “1”, “2”, etc.

Action context

Besides the InputTriggerConfig, the Actions constructor also requires one ore more String... context arguments.

The idea is that the same action (or at least action name) might occur in different contexts, that is, different tools, different windows of the same tool, etc. For example, an action named “Undo” could occur in many contexts and it would be nice to be able to assign different shortcuts, depending on context.

Therefore, an InputTriggerConfig does not directly map action to shortcuts, but rather maps (action, context) pairs to shortcuts, where action and context are both Strings. So, for example, ("Undo", "bdv") can map to a different shortcut than ("Undo", "paintera").

The context arguments given in the Actions constructor specify which subsets of key bindings defined in the InputTriggerConfig should be considered. In the above example, we have

var actions = new Actions( config, "demo" )

This actions will pick up bindings for ("Undo", "demo") from the config, but not ("Undo", "bdv") for example.

Disabled actions

There is a special trigger "not mapped" that can be used to specify that a particular action should not be associated to any shortcut. For example, if we add

- !mapping
action: Action C
contexts: [demo]
triggers: [not mapped]

to the config.yaml file, then “Action C” will be disabled, that is, the programmatic defaults “1”, “2”, etc., will not be used.

You can find the full example on github.

Configuring shortcuts through the UI

Being able to define shortcuts through a config file is useful. The config files can be edited, and distributed between different users or computers.

Even more comfortable is to be able to modify shortcuts directly through the UI, at runtime.

Preferences dialog

For this, we use bdv.ui.settings.SettingsPanel. This panel implements a typical Preferences layout (like it’s used in Eclipse, for example) with a tree of preferences sections on the left, the selected section on the right, and Apply, Ok, Cancel buttons on the bottom.

The following PrefererencesDialog contains only the SettingsPanel, and a method addPage() to adds new sections (bdv.ui.settings.SettingsPage) to the preferences tree.

/*
#| include: false
*/
import java.awt.Frame;
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;

import javax.swing.JDialog;
import javax.swing.WindowConstants;

import bdv.ui.settings.SettingsPage;
import bdv.ui.settings.SettingsPanel;

public class PreferencesDialog extends JDialog
{
    private final SettingsPanel settingsPanel;

    public PreferencesDialog( final Frame owner )
    {
        super( owner, "Preferences", false );
        settingsPanel = new SettingsPanel();
        settingsPanel.onOk( () -> setVisible( false ) );
        settingsPanel.onCancel( () -> setVisible( false ) );

        setDefaultCloseOperation( WindowConstants.HIDE_ON_CLOSE );
        addWindowListener( new WindowAdapter()
        {
            @Override
            public void windowClosing( final WindowEvent e )
            {
                settingsPanel.cancel();
            }
        } );

        getContentPane().add( settingsPanel, BorderLayout.CENTER );
        pack();
    }

    public void addPage( final SettingsPage page )
    {
        settingsPanel.addPage( page );
        pack();
    }
}

Let’s instantiate a PreferencesDialog for our example, and add a keyboard shortcut (command-comma or control-comma) to show it.

var preferencesDialog = new PreferencesDialog( frame );
actions.runnableAction(
    () -> preferencesDialog.setVisible( !preferencesDialog.isVisible() ),
    "Preferences",
    "meta COMMA", "ctrl COMMA" );

Next, we want to add a preferences section for configuring shortcuts. There is bdv.ui.keymap.KeymapSettingsPage that we can readily use. In the end this will give us something like this: What remains to be done is to fill the settings page with a list of configurable actions.

CommandDescriptions

Specifially, we need to supply the KeymapSettingsPage with a list of existing actions, with short textual descriptions. This is done by creating a CommandDescriptions object and adding the configurable actions.

import org.scijava.ui.behaviour.io.gui.CommandDescriptions;

var descriptions = new CommandDescriptions();

descriptions.setKeyconfigContext( "demo" );

descriptions.add( "Action A", new String[] { "SPACE" }, "trigger Action A" );
descriptions.add( "Action B", new String[] { "B", "shift B" }, "trigger Action B" );

For each action, we add its name and default shortcuts in the same way we did when creating the action, and a short description (this is just for showing to the user, so can be left empty if you’re lazy…).

The other thing we need to supply to the KeymapSettingsPage is a KeymapManager. KeymapManager maintains a set of named Keymaps (some built-in, some user-defined). A Keymap is a simple container for a InputTriggerConfig, adding just a name and support for listeners to be notified when the InputTriggerConfig changes.

Our KeymapManager extends the existing AbstractKeymapManager base class. The only thing that needs to be done is providing one or more default Keymaps. We can build a default keymap from the above descriptions. (But they could also be loaded from resources, build manually, …)

import bdv.ui.keymap.AbstractKeymapManager;
import bdv.ui.keymap.Keymap;

var defaultKeymap = new Keymap( "Default", descriptions.createDefaultKeyconfig() );

/**
 * Manages a collection of {@link Keymap}.
 */
public class KeymapManager extends AbstractKeymapManager< KeymapManager >
{
    @Override
    protected List< Keymap > loadBuiltinStyles()
    {
        return Collections.singletonList( defaultKeymap );
    }

    @Override
    public void saveStyles()
    {
        // not implemented.
        // Here we would save user defined keymaps to YAML files, for example.
    }
}

We create a KeyMapManager instance and add it to the Preferences dialog (via KeymapSettingsPage).

import bdv.ui.keymap.KeymapSettingsPage;

var keymapManager = new KeymapManager();
preferencesDialog.addPage(
        new KeymapSettingsPage( "Keymap", keymapManager, new KeymapManager(), descriptions ) );

The KeyMapManager (via its base class) exposes the user-selected keymap. We set that for our actions object. We also add a listener that refreshes actions keybinding when that keymap changes.

var keymap = keymapManager.getForwardSelectedKeymap();
actions.updateKeyConfig( keymap.getConfig(), false );
keymap.updateListeners().add(
    () -> actions.updateKeyConfig( keymap.getConfig(), false )
);

true

That’s it. The user can now use the Preferences dialog to define custom keymaps with shortcuts to their liking, and switch between different keymaps. (Use command-comma or control-comma to show the preferences dialog).

You can find the full example on github.

Making action descriptions discoverable

Keeping the list of existing actions (that is, the CommandDescriptions) up to date is tedious. Actions that should appear in the config dialog may be scattered through your own code and dependencies. This can be somewhat automated with CommandDescriptionProviders. These are scijava @Plugins that can be discovered at runtime.

import org.scijava.plugin.Plugin;
import org.scijava.ui.behaviour.io.gui.CommandDescriptionProvider;

var DEMO_SCOPE = new CommandDescriptionProvider.Scope( "tpietzsch.keymap" );
var DEMO_CONTEXT = "demo";

/*
 * Command descriptions for all provided commands
 */
@Plugin( type = CommandDescriptionProvider.class )
public static class MyActionDescriptions extends CommandDescriptionProvider
{
    public MyActionDescriptions()
    {
        super( DEMO_SCOPE, DEMO_CONTEXT );
    }

    @Override
    public void getCommandDescriptions( final CommandDescriptions descriptions )
    {
        descriptions.add( "Action A", new String[] { "SPACE" }, "trigger Action A" );
        descriptions.add( "Action B", new String[] { "B", "shift B" }, "trigger Action B" );
    }
}

For discovery, we use a CommandDescriptionsBuilder

import org.scijava.Context;
import org.scijava.plugin.PluginService;
import org.scijava.ui.behaviour.io.gui.CommandDescriptionsBuilder;

var context = new Context( PluginService.class );
var builder = new CommandDescriptionsBuilder();
context.inject( builder );

builder.discoverProviders( DEMO_SCOPE );

Note the use of DEMO_SCOPE here. The same scope is also given in the MyActionDescriptions constructor. The discoverProviders() method takes an optional scope argument, and will only discover CommandDescriptionProvider that match this scope. If no scope is given, all CommandDescriptionProvider on the classpath will be discovered. For example within Fiji, that would include actions from Mastodon and BigDataViewer.

Unfortunately, the @Plugin annotations do not work for classes defined in JShell (used by this notebook). As a workaround, we can add MyActionDescriptions manually.

builder.addManually( new MyActionDescriptions(), DEMO_CONTEXT );

After we add everything we need to the builder, we can get the Descriptions.

var descriptions = builder.build();

You can find the full example on github.

Recommended pattern for defining actions

Action definitions in BigDataViewer and Mastodon are organized in the following way.

A set of related actions is collected into a MyActions (for example) class. Action names and default shortcuts are defined as public static final constants, because they are used both for defining the actions, and for creating action Descriptions.

The actions contained in MyActions are described in a public static inner class Descriptions extends CommandDescriptionProvider.

In the Descriptions constructor, we give a scope for the respective library / tool. Ideally, the scope should be defined public static somewhere so that is can easily used outside the component to discover its actions. For example, BigDataViewer uses this scope. If another tool (BigStitcher, BigWarp, etc.) wants to include BDV shortcuts into its customizable keymaps, they can be easily discovered like that.

import org.scijava.plugin.Plugin;
import org.scijava.ui.behaviour.io.gui.CommandDescriptionProvider;

final var DEMO_SCOPE = new CommandDescriptionProvider.Scope( "tpietzsch.keymap-idiom" );
final var DEMO_CONTEXT = "demo";

public class MyActions
{
    // define action name constants
    public static final String ACTION_A = "Action A";
    public static final String ACTION_B = "Action B";
    public static final String PREFERENCES = "Preferences";

    // define default shortcut constants
    public static final String[] ACTION_A_KEYS = { "SPACE" };

    public static final String[] ACTION_B_KEYS = { "B", "shift B" };
    public static final String[] PREFERENCES_KEYS = { "meta COMMA", "ctrl COMMA" };


    /*
     * Command descriptions for all provided commands
     */
    @Plugin( type = CommandDescriptionProvider.class )
    public static class Descriptions extends CommandDescriptionProvider
    {
        public Descriptions()
        {
            super( DEMO_SCOPE, DEMO_CONTEXT );
        }

        @Override
        public void getCommandDescriptions( final CommandDescriptions descriptions )
        {
            descriptions.add( ACTION_A, ACTION_A_KEYS, "trigger Action A" );
            descriptions.add( ACTION_B, ACTION_B_KEYS, "trigger Action B" );
            descriptions.add( PREFERENCES, PREFERENCES_KEYS, "Show the Preferences dialog." );
        }
    }

    
    /**
     * Install into the specified {@link Actions}.
     */
    public static void install( final Actions actions, final MainPanel mainPanel, final PreferencesDialog preferencesDialog )
    {
        actions.runnableAction( () -> mainPanel.setText( "Action A triggered" ),
                ACTION_A, ACTION_A_KEYS );
        actions.runnableAction( () -> mainPanel.setText( "Action B triggered" ),
                ACTION_B, ACTION_B_KEYS );
        actions.runnableAction( () -> preferencesDialog.setVisible( !preferencesDialog.isVisible() ),
                PREFERENCES, PREFERENCES_KEYS );
    }
}

MyActions contains one install method that installs all actions into a provided Actions argument. Ideally, MyActions is stateless, and install method is static.

The remaining arguments to install are whatever is needed to create the actions. In the example, the mainPanel is needed to create “Action A” and “Action B”, and the preferencesDialog is needed to create the action to show/hide it.

So, MyActions.install(...) is called to install into a provided Actions. Usually every frame/panel in the application should have an Actions instance, which is linked to the KeymapManager so that keymap updates propagate correctly.

And that’s it… This is currently the recommended way to structure and bundle action definitions. You can find the full example on github.

See BigDataViewer’s NavigationActions as an example “in the wild”. For behaviours (mouse gestures, etc.) the structure is the same. See BigDataViewer’s TransformEventHandler2D for example.