.. _context-menu:

Context menus
=============

To enable the user to configure a state machine in Desk a context menu is
defined, which leads the user step-by-step through the configuration. Those
context menus can be implemented using HTML5, CSS3 and JavaScript. To support
dynamic markup, `KnockoutJS <http://knockoutjs.com>`_ with its data binding and
component system is utilized. The basics about employed mechanisms like binding
handlers and the binding context can be found in `the knockout documentation
<http://knockoutjs.com/documentation/introduction.html>`_.

To attach a context menu to an app, the ``contextMenu`` field of the app's
``clientData`` has to be set. Any HTML markup in this ``clientData`` field will
then be rendered as a context menu in Desk. Context menus are mainly navigable
via the pilot-buttons on the robot, i. e. ``cross``, ``circle``, ``check`` and
arrow-buttons in four directions. Navigation via mouse/keyboard and touch on a
tablet computer is also possible.

Steps divide the process of configuring a state machine in small chunks. Steps
are combined in a tree like structure. The following code-block depicts a basic step.

.. code-block:: html

  <step id="id"></step>

A step needs to have an ``[id]``, but more attributes can be provided (see
:ref:`step-api`).

.. note::
  All step ids - except for the top level steps - get postfixed
  with a consecutive number, e.g. ``pick-pose-0``. This has to be taken into
  account when referring to a step via its id, e.g. for styling purposes.

.. note::
  Use only classes for CSS styling. ``[style]`` attributes and
  ``[id]`` selectors should be avoided. This ensures that the context menu API can
  adapt the styling correctly to a common look (due to selector specifity).


The first level of steps - so called *top level steps* - are
rendered as tabs of the context menu and are navigable via ``check`` and
``cross``.

A top level step can contain further steps - so called sub steps. If a top level
step is focused, its sub steps are recursively focused until the first leaf step
is reached. Inside a top level step navigation is possible via the
arrow-buttons, where ``right`` and ``down`` focus the next leaf step and
``left`` and ``up`` focus the previous leaf step. However, it is possible to
override this behavior if needed (see :ref:`pilot-navigation`)

When creating a context menu, we first have to define the top level steps
(i.e. divide the parameters to be configured into meaningful chunks). A good
strategy is to cover all the parameters which can be taught via guiding in the
beginning, such as robot poses and gripper widths.

For our ``Tutorial`` app (see :ref:`pick-place-tutorial`), we have to
parameterize the approach and retract poses along with the gripper width for
the opened and closed gripper. For picking the object, the parameters are set
in a step with the id ``#pick-motion``. A step with the id ``#place-motion``
is added for placing the object. Furthermore, the collision area and the
velocity have to be parameterized. The structure of our context menu for the
``Tutorial`` app will therefore look like this:

.. code-block:: html

  contextMenu : @{
    <step id="pick-motion"></step>
    <step id="place-motion"></step>
    <step id="collision"></step>
    <step id="velocity"></step>
  }@;

The following image depicts the context menu we want to create for our
``Tutorial`` app. The just defined top level steps are visible above the
content area.

.. image:: _static/menu.png

Parameter modification
----------------------

Parameters of the surrounding state machine are accessible via ``parameter()``.
To access only a subtree of the parameter, an expression has to be provided.
``parameter('velocity')`` accesses e.g. the ``velocity`` parameter in the
``Tutorial`` app, ``parameter('pick_pose.joint_angles')`` the joint angles
of the pick pose. Sometimes, numerical values are internally handled in a unit
which is not comfortable to modify for the user. Therefore several modifiers
are provided for the parameters, which enable conversion. Those modifiers are
``add``, ``multiply`` and ``round``. To convert e.g. a temperature from degree
Celsius to degree Fahrenheit with one decimal place, one would write

.. code-block:: html

  parameter('temperature').add(-32).multiply(5/9).round(1)


To enable the user to modify the parameters, a variety of input components is
provided. Numerical values can be modified via sliders. Choosing between two or
more options is enabled via ``checkbox-slider``, ``alternative-selector`` and
``drop-down-menu``. One or several poses can be taught via ``robot-pose`` and
``robot-pose-carousel``, respectively. Teaching the gripper-width isolated from
the pose is obtained with ``gripper-control``. Components are used as follows:

.. code-block:: html

  <component-name params="
    ...
    step: step
  "/>

Each component has specific parameters, which have to be passed to the ``[params]``
attribute.

.. note::
  The components require you to also pass the ``step`` from the
  binding context into the component. This is necessary to allow for pilot
  navigation and correct visualization on acquiring focus.


.. _linear-slider:

Linear slider
`````````````

Allows for setting a value within a range

.. image:: _static/linear_slider.png

**Parameters**

* ``value`` [mandatory] (Number): the app parameter which should be modified
* ``max`` [mandatory] (Number): the maximal value
* ``min`` [default=0] (Number): the minimal value
* ``increment`` [default=1] (Number): the amount by which the value is increased or
  decreased by pressing the arrow buttons or dragging the slider knob
* ``initial`` [default=min] (Number): the initial value
* ``unit`` (String or Array[String]): the unit of the parameter, either as a single string, e.g.
  ``'%'`` or as an array of strings, e.g. ``['second', 'seconds']``, where the first entry is
  displayed if the value is ``1`` and the last entry otherwise

**Code Example**

.. code-block:: html

  <linear-slider params="
    value: parameter('velocity'),
    initial: 65,
    increment: 5,
    max: 100,
    unit: '%',
    step: step
  "/>

Toggle slider
`````````````

Allows for modifying several numerical values at once.

.. image:: _static/toggle_slider.png

**Parameters**

An array is passed to the ``toggle-slider`` which contains the parameters for
all ``linear-slider`` inside, as listed in :ref:`linear-slider`, plus an
additional ``label`` parameter, which is displayed in the button bar on the
left.

**Code Example**

.. code-block:: html

  <toggle-slider params="
    parameters: [
      {value: parameter('safety.force_thresholds.x'), label: 'x', min: 3, max: 100, unit: 'N'},
      {value: parameter('safety.force_thresholds.y'), label: 'y', min: 3, max: 100, unit: 'N'},
      {value: parameter('safety.force_thresholds.z'), label: 'z', min: 3, max: 100, unit: 'N'},
      {value: parameter('safety.force_thresholds.x_rot'), label: 'x-rot', min: 1, max: 30, unit: 'Nm'},
      {value: parameter('safety.force_thresholds.y_rot'), label: 'y-rot', min: 1, max: 30, unit: 'Nm'},
      {value: parameter('safety.force_thresholds.z_rot'), label: 'z-rot', min: 1, max: 30, unit: 'Nm'},
    ],
    step: step
  "/>

Arc slider
``````````
Allows for a circular representation of the value space.

.. image:: _static/arc_slider.png

**Parameters**

* ``value`` [mandatory] (Number): the app parameter which should be modified
* ``min`` [default=0] (Number): the minimal value
* ``max`` [default=Infinity] (Number): the maximal value
* ``increment`` [default=1] (Number): the amount by which the value is increased or decreased
* ``zeroValue`` [default=0] (Number): the start of a circle
* ``fullValue`` [default=360] (Number): the end of a circle
* ``initial`` [default=min] (Number): the initial value
* ``unit`` (String or Array[String]): the unit of the parameter, either as a single string,
  e.g. ``'%'`` or as an array of strings, e.g. ``['second', 'seconds']``, where the first entry is
  displayed if the value is ``1`` and the last entry otherwise

**Code Example**

.. code-block:: html

  <arc-slider params="
    value: parameter('duration'),
    initial: 20,
    fullValue: 60,
    unit: 's',
    step: step
  "/>

Checkbox slider
```````````````

Allows for modifying booleans.

.. image:: _static/checkbox.png

**Parameters**

* ``value`` [mandatory] (Boolean): the app parameter which should be modified
* ``unchecked`` [default='OFF'] (String): a string displayed on the left side of the ``checkbox-slider``
* ``checked`` [default='ON'] (String): a string displayed on the right side of the ``checkbox-slider``

**Code Example**

.. code-block:: html

  <checkbox-slider params="
    value: parameter('clockwise'),
    unchecked: 'CW',
    checked: 'CCW',
    step: step
  "></checkbox-slider>

Alternative selector
````````````````````

Allows for choosing between multiple values which are displayed as an image
with a label.

.. image:: _static/alternative.png

**Parameters**

* ``parameter`` [mandatory] (String): the app parameter which should be modified
* ``alternatives`` (Array[Object]): an array of alternative objects, defined by

  - ``name`` (String): the displayed name of the alternative
  - ``image`` (String): an image displayed above the name
  - ``value`` [mandatory] (String): the value which is written into ``parameter`` when
    the corresponding alternative is selected

**Code Example**

.. code-block:: html

  <alternative-selector params="
    parameter: parameter('velocity_profile.plane'),
    alternatives: [
      { name: 'XY', image: plane_xy, value: 'xy' },
      { name: 'XZ', image: plane_xz, value: 'xz' },
      { name: 'YZ', image: plane_yz, value: 'yz' }
    ],
    step: step
  "></alternative-selector>

Drop down menu
``````````````

Allows for choosing between multiple values.

.. image:: _static/dropdown.png

**Parameters**

* ``parameter`` [mandatory] (String): the app parameter which should be modified
* ``items`` [mandatory] (Array[Object]): an array of item objects, defined by

  - ``value`` [mandatory] (String): the value which is written into ``parameter`` when
    the corresponding item is selected
  - ``text`` (String): the displayed name of the item

* ``default`` (String): the value of the item selected by default

**Code Example**

.. code-block:: html

  <drop-down-menu params="
    parameter: parameter('plug_type'),
    default: 'hdmi',
    items: [ {value: 'USB', text: 'USB'},
             {value: 'hdmi', text: 'HDMI'},
             {value: 'VGA', text: 'VGA'},
             {value: 'dellPower', text: 'Dell Power'},
           ],
    step: step
  "></drop-down-menu>

.. _robot-pose:

Robot pose
``````````

Allows for writing the current pose of the robot into a parameter. This component
allows to simultaneously write the gripper width by passing a parameter to
``gripper_open`` or ``gripper_closed``. Can be used to teach a single pose or
multiple related poses. If multiple related poses are taught, one of the poses
has to be marked as reference.

Markup passed to the component will be rendered inside the context menu to
represent the pose. Refer to :ref:`scene` for a detailed description to proper
visualization.

**Parameters**

.. note::
  Besides the step also the path and the componentProviderApi has to be
  passed from the binding context.

* ``pose`` [mandatory] (Object): the app parameter which should be modified
* ``gripper_open`` (Object): if passed an app parameter, it
  will be written simultaneously with the pose
* ``gripper_closed`` (Object): if passed an app parameter, it
  will be written simultaneously with the pose
* ``isRelativeToFirst`` (Boolean): refer to :ref:`grid-points` for a detailed description
* ``isBottomRightFirst`` (Boolean): refer to :ref:`grid-points` for a detailed description

If more than one robot pose is used:

* ``reference`` (Boolean): set to true, if this is the pose which requires the most accuracy,
  e.g. if there is an approach pose and a pick pose, set true for the pick pose
* ``relatedPoses`` (Object(Object)): contains all related poses with the id of the step as
  key and the parameter as value


**Code example**

.. code-block:: html

  <robot-pose params="
    pose: parameter('pick_pose'),
    reference: true,
    isBottomRightFirst: false,
    relatedPoses: {
      'pick-pose': parameter('pick_pose'),
      'approach-pose': parameter('pick_approach')
    },
    step: step,
    path: path,
    componentProviderAPI: componentProviderAPI
  ">
    <div class="pose-content">
      ...
    </div>
  </robot-pose>

Robot pose carousel
```````````````````

Allows for teaching an arbitrary amount of poses.

.. image:: _static/robot-pose-carousel.png

Displays the first and last pose, when not editing.

.. image:: _static/robot-pose-carousel-edit.png

**Parameters**

.. note::
  Besides the step also the path and the componentProviderApi has to be
  passed from the binding context.

* ``poses`` [mandatory] (Array[Object]): the app parameter which should be modified
* ``maximumPoses`` (Number): limit the amount of teachable poses
* ``motionParams`` (Array[Object]): allows to define expert settings per pose in `poses`

  - ``pose_name`` (String): custom name of the pose
  - ``use_custom_parameters`` (Boolean): activate the custom settings for the pose
  - ``custom_acceleration``: (Number): custom acceleration for motion to this pose
  - ``custom_deceleration``: (Number): custom deceleration for motion to this pose
  - ``custom_speed``: (Number): custom speed for motion to this pose
  - ``point2point`` (Boolean): go to exact pose (no spline interpolation)

* ``motionType`` [default="cartesian"] (String): cartesian or joint motion. For
  "joint" the point2point and deceleration options are hidden.
* ``relatedPoses`` (Object(Object)): contains all related poses with the id of
  the step as key and the parameter as value

**Code Example**

.. code-block:: html

  <robot-pose-carousel params="
    poses: parameter('motion'),
    maximumPoses: 100,
    motionParams: parameter('motion_params'),
    relatedPoses: {
      points: parameter('motion'),
    },
    step: step,
    path: path,
    componentProviderAPI: componentProviderAPI
  "></robot-pose-carousel>


Gripper control
```````````````

Allows for teaching the gripper width independent from the pose. It displays the
actual gripper width (grey indicator) and the current set width (blue
indicator) and enables the user to adopt the current value.

.. image:: _static/gripper.png

**Parameters**

* ``width`` [mandatory] (Number): the app parameter which should be modified

**Code Example**

.. code-block:: html

  <gripper-control params="
    width: parameter('gripper_open_width'),
    step: step
  "></gripper-control>


.. _scene:

Visualizing robot poses
-----------------------

Typically, apps demand for teaching several related poses. These poses should be
depicted in a scene, which represents the poses in relation to the workspace.
Suitable images have to be passed to the ``robot-pose`` component, such that the
current teaching situation is visualized in the context menu. Markup passed to
the component will be rendered inside the context menu to represent the pose.
Typically it consists of one or more wrapped up SVG images.

.. note::
  The markup passed to the robot-pose has to be sized properly, such that
  other elements inside the context-menu will not be overlapped by the overflow of
  the element's box.

For a ``#pick_pose`` we could display the gripper grasping an object as
depicted in the following image, consisting of an SVG image of the gripper and
an SVG image of the object (see :ref:`svg`).

.. image:: _static/pick_pose.png

Both SVG images are passed to the ``robot-pose``.

.. code-block:: html

  <robot-pose params="...">
    <svg><use href="object.svg#icon"></svg>
    <svg><use href="gripper_opened.svg#icon"></svg>
  </robot-pose>

.. _positioning:

Positioning
```````````

There are several classes provided to arrange the passed markup, i.e.

================  ==================  =================
``.left-top``     ``.center-top``     ``.right-top``
``.left-center``  ``.center-center``  ``.right-center``
``.left-bottom``  ``.center-bottom``  ``.right-bottom``
================  ==================  =================

As we have two elements, we arrange them centered at the top and bottom.

.. code-block:: html

  <robot-pose>
    <div class="pose-content" style="width: 80px; height: 100px">
      <svg class="center-top"><use href="object.svg#icon"></svg>
      <svg class="center-bottom"><use href="gripper_opened.svg#icon"></svg>
    </div>
  </robot-pose>

Notice that we added a ``.pose-content`` wrapper around the SVGs. The classes
provided above add ``position: absolute`` to the elements such that they do not
influence the size of the surrounding container. We therefore define ``width``
and ``height`` on the ``.pose-content``.

.. _coloring:

Coloring and visibility
```````````````````````

The coloring and visibility of the pose-content depends on the current focus.
The content of leaf steps which prior siblings are not configured is hidden.
This behavior can be customized by adding the classes ``.visible`` or
``.hidden`` to elements.

There are three color schemes

* ``light-gray``: default for focusable but not focused steps
* ``dark-gray``
* ``highlighted``: default for focused steps

as depicted in the following image from left to right.

.. image:: _static/colors.png

To adjust the appearance of elements inside a focused step the classes
``.light-gray`` or ``.dark-gray`` can be added. To enforce a certain color
scheme no matter if the step is focused or not, the color classes can be
prefixed with ``static-``, i.e. ``.static-highlighted``.

For the ``#pick-pose``, we want the object to be visible all the time in a neutral
color. Therefore the classes ``.visible`` as well as ``.static-dark-gray`` are added.

.. code-block:: html

  <robot-pose>
    <div class="pose-content">
      <svg class="center-top"><use href="object.svg#icon"></svg>
      <svg class="center-bottom static-dark-gray visible">
        <use href="gripper_opened.svg#icon">
      </svg>
    </div>
  </robot-pose>

.. _svg:

Guidelines for SVG images
-------------------------

To get good looking results, the SVG images have to fulfill certain
prerequisites. To address the SVG image like in the example above, an ``[id]``
has to be set. It is recommended to remove all ``[height]`` or ``[width]``
attributes from the SVG image and provide only a ``[viewbox]``. Sizing of the
SVG images should take place inside the app, e.g. by adding a ``<style>``-tag or
``[style]``-attribute. Furthermore, inside the SVG image no colors should be
set, as the color schemes above would be overwritten. If unintended outlines or
lines are visible in the SVG image, it is recommended to set ``[stroke]`` as
``"none"``. If those prerequisites are fulfilled, the SVG will be displayed in
the primary color of the color scheme. To add depth to the image by coloring
certain areas darker, ``fill="currentColor"`` has to be set.

.. image:: _static/gripper_svg.png

The depicted gripper image has for example the following structure

.. code-block:: html

  <svg id="icon" xmlns="http://www.w3.org/2000/svg" stroke="none" viewBox="0 0 123.65 81.91">
      <g id="gripper-front">
          <path id=".." d="..."/>
      </g>
      <g id="gripper-back" fill="currentColor">
          <path id=".." d="..."/>
          <path id=".." d="..."/>
          <path id=".." d="..."/>
          <path id=".." d="..."/>
          <path id=".." d="..."/>
      </g>
  </svg>

SVGs can be used inside the context menu via

.. code-block:: html

  <svg><use href="bundles/tutorial/gripper.svg#icon"></use></svg>

The link to the image assumes, that it is placed within a ``resources`` folder
inside the ``tutorial`` bundle. The structure would then look like the
following:

.. code-block:: none

  .
  └── bundles
    └── tutorial
        ├── migrations
        ├── resources
        │  ├── logo.svg
        │  ├── gripper_holding.svg
        │  ├── gripper_opened.svg
        │  └── object.svg
        ├── sources
        │  └── Tutorial.lf
        └── manifest.json

.. important::
  The ``<svg>`` tag within the SVG file requires an ``[id]`` attribute to be
  addressable. This ``[id]`` then has to be appended to the URL with a ``#``,
  like in the example above. For app icons this ``[id]`` needs to be ``icon``.

.. _steps:

Positioning and spacing of steps
--------------------------------

Finally, the steps themselves have to be arranged to depict the workspace. The
*positioning* via the *CSS Flexbox Layout* can be achieved by using

* ``.flex-column``
* ``.flex-row``

either on the steps or surrounding containers. Those classes can be extended by

* ``.center-content-x``
* ``.center-content-y``

to achieve vertical and horizontal alignment respectively.

Another convenient way for positioning is the *CSS Grid Layout*, as it enables
two-dimensional arrangement of the content.

A convenient way to modify the *spacing* via margins is by using the classes

* ``.margin-top-``
* ``.margin-bottom-``
* ``.margin-left-``
* ``.margin-top-``

postfixed with the margin in pixels. The classes are available for margins from
20 to 300 pixels in 20 pixel increments, e.g. ``.margin-top-40`` or
``.margin-left-280``.

To get an example have a look at the code of the ``Tutorial`` app. There
we add ``.flex-column`` to the ``#pick-motion`` step, which contains two
containers with the ``#pick-approach`` and ``#approach-width`` and the
``#pick-pose`` and ``#pick-width`` respectively. Both containers have the
``.flex-row`` class. As now all components are close together, we add
``.margin-bottom-60`` to the first container and ``.margin-right-60`` to the
``#pick-approach`` and ``#pick-pose`` steps.

The ``#pick-motion`` step looks now like this.

.. code-block:: html

  <step id="pick-motion" name="Pick Motion" class="flex-column">
    <div class="flex-row margin-bottom-60">
      <step id="pick-approach" class="margin-right-60">
        <robot-pose params="...">
          <div class="pose-content">
            <svg class="center-top"><use href="gripper_opened.svg#icon"></use></svg>
          </div>
        </robot-pose>
      </step>
      <step id="approach-width">
        <gripper-control params="..."></gripper-control>
      </step>
    </div>
    <div class="flex-row">
      <step id="pick-pose" class="margin-right-60">
        <robot-pose params="...">
          <div class="pose-content">
            <svg class="center-bottom static-dark-gray visible">
              <use href="object.svg#icon"></use>
            </svg>
            <svg class="center-top">
              <use href="gripper_holding.svg#icon"></use>
            </svg>
          </div>
        </robot-pose>
      </step>
      <step id="pick-width">
          <gripper-control params="..."></gripper-control>
      </step>
    </div>
  </step>

If the context menu of the ``Tutorial`` app is opened for the first time, the
following scene is visible to the user

.. image:: _static/pick_top.png

After teaching the approach pose and approach width, the scene changes to the
following image.

.. image:: _static/pick_bottom.png

Providing descriptions
----------------------

Providing only images is in most cases not sufficient to guide the user through
the teaching process. Therefore it is possible to add a ``[name]`` and a ``[label]``
attribute to steps, which provide textual information to the user. The name
shortly titles the step, whereas the label provides one or more sentences to
describe the step in more detail. The names of the top level steps are visible
all the time as the titles of the tabs at the top of the context menu.

``robot-pose``, ``robot-pose-carousel`` and ``gripper-control`` provide
``step-labels``, which display title and label of the surrounding step. If a
step is focusable - i.e. all previous steps are configured - the content of the
step and its name are visible. If focused, the name as well as the label are
visible next to the step as depicted in the following image.

.. image:: _static/labels.png

Step labels can also be added manually by placing ``<step-label params="step:
step" />`` inside a step. Step labels appear by default on the right side of the
step. By adding the class ``.left`` to the step-label or ``.left-step-label`` to
the step, the step-label will be moved to the left side.

We therefore extend our ``#motion`` step with names and labels.

.. code-block:: html

  <step id="pick-motion" name="Pick Motion">
    <step
      id="pick-approach"
      name="Position above"
      label="Position the Hand above the object."
    >...</step>
    <step
      id="approach-width"
      name="Open fingers"
      label="Set the width between the fingertips such that it is larger than the object at its picking points."
    >...</step>
    <step
      id="pick-pose"
      name="Pick position"
      label="Position the Hand such that the object is between the fingertips."
    >...</step>
    <step
      id="pick-width"
      name="Close fingers"
      label="Close the fingers such that the object is firmly grasped."
    >...</step>
  </step>

Furthermore it is possible to add descriptions via the html elements ``h1``,
``h2`` and ``p`` or ``label``. In our example we add ``<h2/>`` with the step
label to add a description to the ``linear-sliders`` in the ``#collision`` and
``#velocity`` step.

.. code-block:: html

  <step
    id="collision"
    name="Collision Area"
    class="flex-column"
    label="Set the collision area in which a collision should be detected."
  >
    <h2 data-bind="text: step.label"/>
    <linear-slider params="..."/>
  </step>
  <step
    id="velocity"
    name="Velocity"
    class="flex-column"
    label="Set the velocity of the robot's motion."
  >
    <h2 data-bind="text: step.label"/>
    <linear-slider params="..."/>
  </step>

The resulting ``#velocity`` step can be seen in the following image

.. image:: _static/label_slider.png

.. note::
  The complete context menu for the ``Tutorial`` app example can be viewed
  :download:`here <_static/Tutorial.lf>`. For a full app creation tutorial
  check out the section :ref:`pick-place-tutorial`.

Advanced Topics
---------------

.. _step-api:

Step API
````````

Conditional steps
^^^^^^^^^^^^^^^^^

Sometimes, steps should only be available, if certain prerequisites are
fulfilled. This can be obtained by setting the ``[if]`` attribute of the step,
e.g.

.. code-block:: html

  contextMenu : @{
    <step id="..." if="parameter('show_step')() == true">...</step>
  }@;

Initial
^^^^^^^

To specify a different entry point when focusing a step, add ``[initial]`` to one of
its children:

.. code-block:: html

  <step id="..." initial></step>


Only navigate
^^^^^^^^^^^^^

If a previous step is done, the next sub step will automatically be focused.
To remove a step from the next considered steps, add ``[only-navigate]`` to the
specific step:

.. code-block:: HTML

  <step id="..." only-navigate></step>

The step will still be focusable by directly navigating using
``left`` or ``right``, or by clicking on it.

Configured
^^^^^^^^^^

Steps which need no more intervention by the user are *configured*. A step can
only be configured if all of its children are configured as well. Not yet
configured top level steps are marked in the context menu. In general, the
configured handling is enforced by the components, which check if the passed
parameter is still ``nil``. The configured handling can be customized by adding
the ``[configured]`` attribute to the step.

.. code-block:: HTML

  <step id="..." configured="parameter('configured')() == true" ></step>

Light-weight viewmodels
```````````````````````

The light-weight viewmodel mechanism can be used to inject a custom viewmodel
into a step. This allows to hold values in lexical scope, provide a cleaner
interface to the markup / binding handlers while retaining access on the full
binding context (with which the ``<viewmodel>`` content is invoked):

.. code-block:: HTML

  <step id="repetitions" label="Repetitions" configured="parameter('times')">
    <viewmodel>
      var ko = require("knockout")
      var value = parameter('times')

      var element = $element
      element.addEventListener("blur", function() {
        value(element.value)
      })

      return ko.pureComputed(function() {
        return value() + " times"
      })
    </viewmodel>
    <input type="number" data-bind="
      value: viewmodel,
      attr: { min: 1, max: 10, step: 1 }
    " />
    <div data-bind="text: viewmodel">
  </step>


.. _components:

Custom components
`````````````````

Creating components
^^^^^^^^^^^^^^^^^^^

Components are declared using ``<component>`` tags in the ``components``
``clientData`` entry in the app.

A component has one or more names and consists of two parts: ``<template>``,
which includes the component markup, and ``<script>``, which includes the
viewmodel javascript code. The contents of these tags can be
either inlined or sourced from another file, e.g.:

.. code-block:: HTML

  components : @{
    <component name="custom-component also-custom-component">
      <template src="custom_component.html"></template>
      <script src="custom_component.js"></script>
    </component>
  }@

This markup declares a new component available as ``custom-component`` and
``also-custom-component``. All component names are **required** to contain a dash.

Global components
^^^^^^^^^^^^^^^^^

Components can also be made available globally by specifying them in the
``globalComponents`` field of the manifest, e.g.:

.. code-block:: json

  {
    "name": "my_bundle",
    "version": "1.0",
    "dependencies": [],
    "globalComponents": [
      "<component name=\"custom-component also-custom-component\"><template src=\"custom_component.html\"></template><script src=\"custom_component.js\"></script></component>"
    ]
  }

Global components are available to all apps and groups.

.. _template:

Template
^^^^^^^^

Whenever possible, it is recommended to use the introduced classes in
:ref:`steps`. However, templates can contain ``<style>`` tags to style the
template elements accordingly. It is recommended to use the name of the
components inside the css rules to avoid leaking, as depicted in the following
code-block.

.. code-block:: HTML

  <style>
    custom-component also-custom-component input {
      width: 50px;
    }
  </style>

To be able to style the component dependent on the current focus, the step class
``.focused`` can be used, as depicted in the following code-block.

.. code-block:: HTML

  <style>
    step.focused custom-component {
      border-width: 2px;
    }
  </style>


There are a number of variables for colors which get replaced
to pre-defined values. It is recommended to use these over hardcoded values to
have consistent styling. Those are

* ``$error.color``
* ``$warning.color``

* ``$circle.color``
* ``$check.color``
* ``$cross.color``

Similar to the ``highlighted`` color scheme

* ``$focused.color``
* ``$focusedAccent.color``

Similar to the ``light-gray`` color scheme

* ``$inactive.color``
* ``$inactiveAccent.color``
* ``$inactiveSubtle.color``

Similar to the ``dark-gray`` color scheme

* ``$active.color``
* ``$activeAccent.color``
* ``$activeSubtle.color``

The following example template contains the HTML5 ``<input>`` element and a
button bound to the component's click method.

.. code-block:: HTML

  components : @{
    <component name="my-custom-component">
      <template>
        <style>
          custom-component input {
            width: 50px;
          }
          step.focused custom-component {
            border-color: $focused.color;
          }
        </style>
        <input type="number" data-bind="
          value: value,
          attr: { min: min, max: max, step: increment }"/>
        <div data-bind="click: click">OK</div>
      </template>
      <script>
        return {
          click: () => {console.log('ok clicked!')},
          value: 41,
          increment: 1,
        }
      </script>
    </component>
  }@

Viewmodel
^^^^^^^^^

The ``<viewmodel>`` tag is loaded as CommonJS module with a variety of
requirable modules. However, the code is loaded without context, so everything
has to be required. The declared exports or returned value is used as
constructor for the component viewmodel, so make sure you export or return a
function.

Requirable modules are

* `lodash 3.10.1 <https://lodash.com/docs/3.10.1>`_
* `jquery 3.4.1 <http://api.jquery.com/>`_
* `knockout 3.4.2 <http://knockoutjs.com/documentation/introduction.html>`_
* :ref:`matrix <matrix>`
* :ref:`component_util <component_util>`

The viewmodel constructor is provided with (as in Knockout components) the
passed ``params`` and additional arguments:

* ``api`` - The Desk API to play sound or retrieve robot poses
* ``source`` - The element API for the  skill/group which declared the
  component. This is used to provide components which not only parameterize the
  instantiating but also the providing skill.
* ``element`` - A reference to the DOM element holding the rendered template.

The instance created by invoking the constructor with new will then be available
as ``$data`` and ``$component`` in the binding context.

.. code-block:: javascript

  var _ = require("lodash")

  var NumberInput = module.exports = function(params, api, source, element) {

    this.value = params.value
    this.min = params.min || 0
    this.max = params.max
    this.increment = params.increment || 1

    this.click = function() {
      params.step.done()
    }
  }

.. _matrix:

Matrix
^^^^^^

The module ``matrix`` provides several matrix utility functions for
transformation matrices, poses and trajectories. Transformation
matrices are stored in column-major order in arrays of
length 16. Poses are objects containing the transformation matrix and
the joint angles as an array of length 7, i.e. ``{ pose: Array[16],
joint_angles: Array[7] }``. Trajectories are arrays containing poses.

* ``Matrix.transpose(m)``: transposes a matrix

  * *arguments*: ``m (Array)`` the matrix to process
  * *returns*: ``(Array)`` the transposed matrix

* ``Matrix.invert(m)``: inverts a matrix

  * *arguments*: ``m (Array)`` the matrix to process
  * *returns*: ``(Array)`` the inverted matrix

* ``Matrix.dot(m1, m2)``: multiplies two matrices

  * *arguments*: ``m1 (Array), m2 (Array)`` the matrices to process
  * *returns*: ``(Array)`` the result of the multiplication

* ``Matrix.identity()``: returns a new identity matrix

  * *returns*: ``(Array)`` identity matrix

* ``Matrix.translate(m, x, y, z)``: translates the transformation matrix

  * *arguments*: ``m (Array)`` the transformation matrix, ``x (Number), y (Number), z (Number)`` translation vector
  * *returns*: ``(Array)`` the result of the translation

* ``Matrix.moveTrajectory(traj, a, b)``: moves a trajectory from a to b

  * *arguments*: ``traj (Array(Objects))`` the trajectory to be moved, ``a (Object), b (Object)`` the start
    (a) and end (b) pose as
  * *returns*: ``(Array(Objects))`` the moved trajectory

* ``Matrix.movePose(pose, a, b)``: moves a pose from a to b

  * *arguments*: ``pose (Object)`` the pose to be moved, ``a (Object), b (Object)`` the start
    (a) and end (b) pose as
  * *returns*: ``(Object)`` the moved trajectory

* ``Matrix.matricesEqual(a, b, [p])``: checks if two matrices are equal given a certain precision

  * *arguments*: ``a (Array), b (Array)`` the matrices to be compared, ``p [optional] (Number)`` the precision
  * *returns*: ``(Boolean)`` returns ``true`` if matrices are equivalent

* ``Matrix.trajectoriesEqual(a, b, [p])``: checks if two trajectories are equal given a certain precision

  * *arguments*: ``a (Array(Object)), b (Array(Object))`` the trajectories to be compared, ``p [optional] (Number)`` the precision
  * *returns*: ``(Boolean)`` returns ``true`` if trajectories are equivalent

.. _component_util:

Component util
^^^^^^^^^^^^^^

The module ``component_util`` provides several utility functions.

* ``Util.or([args])``: returns the first argument which is not ``undefined`` or ``null``

  * *arguments*: ``args (...)`` the arguments to check for ``undefined`` and ``null``
  * *returns*: ``(...)`` the first arguments which is not ``undefined`` or ``null``

* ``Util.clamp(value, min, max)``: clamps value within the lower and upper bounds

  * *arguments*: ``value (Number)`` the value to clamp, ``min (Number)`` the lower bound, ``max (Number) the upper bound``
  * *returns*: ``(Number)`` the clamped value

* ``Util.pluralize(unit, value)``: applies correct pluralization to unit, e.g. ``"second"`` instead of ``"seconds"`` if the value is 1

  * *arguments*: ``unit (String or Array(String))`` the unit(s), ``value (Number)`` the current value
  * *returns*: ``(String)`` unit with correct number

Custom groups
^^^^^^^^^^^^^
You can create your own groups, similar to `points` and `grid`. A ``.lf`` file is recognized as a group when it defines ``type: "group"`` within the ``clientData``.

The group can modify the behavior of child apps in many ways. One is by registering their own components like ``robot-pose`` and thus overwriting the default ``robot-pose``:

.. code-block:: html

  components: @{
    <component name="robot-pose">
      <template src="custom_group/robot_pose.html"></template>
      <script src="custom_group/robot_pose.js"></script>
    </component>
  }@;

The icons displayed on the left and right can be defined by adding two more ``svg``
elements to the ``image`` ``clientData`` with the ids ``open`` and ``close``.

.. code-block:: html

  clientData {
    ...
    image: @{
      <svg id="icon"><use href="bundles/custom_group/logo.svg#icon"></use></svg>
      <svg id="open"><use href="bundles/custom_group/open.svg#open"></use></svg>
      <svg id="close"><use href="bundles/custom_group/close.svg#close"></use></svg>
    }@;
  }

.. _reacting-to-changes-using-pure-computed:

Reacting to changes of observable values using ``ko.pureComputed``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Many values provided by the viewmodel apis implement an observer pattern to
always provide up-to-date values to viewmodels. For instance, it is possible
to check if the current step is focused by calling ``params.step.focused``:

.. code-block:: javascript

  const MyViewmodel = function(params, api, source, element) {
    console.log(params.step.focused()) // will output if the step is focused at
                                       // the moment of invocation
  }

To always display current data to users, it is possible to wrap observable
values in a knockout ``ko.pureComputed``. A ``ko.pureComputed`` will keep track
whether other observers are subscribed to its value, update if its dependencies
change and it is being observed, and will dispose of its own subscriptions to
observables automatically on teardown of the viewmodel. This allows for a number
of patterns:

.. code-block:: javascript

  const ko = require('knockout');

  const MyViewmodel = module.exports = function(params, api, source, element) {
    this.isStepFocused = ko.pureComputed( () => {
      // simply return the observable value
      return params.step.focused();
    });
    this.isStepFocusedAndConfigured = ko.pureComputed( () => {
      // combine the value of multiple observables
      return params.step.focused() && params.step.configured();
    });
    this.gripperClosedLabel = ko.pureComputed( () => {
      // transform an observable value to another type
      return api.relativeGripperWidth() === 0
        ? "The gripper is closed."
        : "The gripper is not closed!";
    });
  }

There values can then be used in the HTML markup of the viewmodel:

.. code-block:: HTML

  <!-- ko if: isStepFocused >
    <span>The step is focused</span>
  <!-- /ko >
  <!-- ko if: isStepFocusedAndConfigured >
    <span>The step is focused and configured</span>
  <!-- /ko >
  <span data-bind="text: gripperClosedLabel"></span>

.. _using-subscribe:

Usage of ``subscribe`` and ``ko.computed`` with observable values
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It is possible to observe for changes in an observable by invoking the knockout
``subscribe`` method. This is especially interesting if it is necessary to run
side-effects as a reaction to a change in an observable value. The ``subscribe``
method will invoke the provided callback as soon as the observed value changes,
which allows running side effects as a result of value changes.

However, using the ``subscribe`` function can cause `memory leaks` and, when
improperly used in context menu viewmodels, `loss of data`, if the subscriptions
are not disposed of properly. When using subscribe, always invoke the
``dispose`` function on the returned object when disposing the viewmodel:

.. code-block:: javascript

  const MyViewmodel = module.exports = function(params, api, source, element) {

    // creating an empty list to store all created subscriptions
    this.subs = []

    // adding return values of ``subscribe'' to the list of subscriptions
    this.subs.push(
      params.step.focused.subscribe(function (focused) {
        if (focused) console.log('My step is currently focused!')
      }),
      params.step.configured.subscribe(function (configured) {
        if (configured) console.log('My step is currently configured!')
      })
    )
  }

  // the dispose function is called when the viewmodel is unloaded
  MyViewmodel.prototype.dispose = function() {
    // dispose of all registered subscriptions
    this.subs.forEach( (subscription) => subscription.dispose() )
  }

A similar pattern is possible and should be followed in *lightweight viewmodels*:

.. code-block:: html

  <viewmodel>
    // creating an observable
    const myObservable = ko.observable('some value');
    // an array to store all subscriptions
    const subs = []

    // each time when creating a subscription, add it to the array of subscriptions
    subs.push(myObservable.subscribe(console.log))

    return {
      dispose: function () {
        // dispose each subscription when disposing the viewmodel
        subs.forEach((subscription) => { subscription.dispose() })
      }
    };
  </viewmodel>

The same principle applies to ``ko.computed`` values, and they should be disposed
in the same way.

**Use of the** ``subscribe`` **and** ``ko.computed`` **functions is heavily discouraged.**
If you wish to display values in your viewmodel template that are based on other
observable values, prefer using a knockout ``ko.pureComputed`` instead (check
:ref:`the corresponding section<reacting-to-changes-using-pure-computed>` for
details).

Interaction
```````````
.. _pilot-navigation:

Customizing pilot navigation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The context menus are usually directly interacted with from the pilot interface
mounted on the robot. Sometimes the provided default implementation of the pilot
navigation is not suitable. It can be customized by passing an object to the
``step.update()`` function as depicted below

.. code-block:: JavaScript

  step.update(function() {
    var pilot = {
      circle: { label: "...", click: function() { ... } },
      check: { label: "...", click: step.done }
    }
    return {
      pilot: pilot
    }
  })

For every button on the pilot interface, a configuration can be passed to
``step.pilot``, i.e.

.. code-block:: JavaScript

  {
    check: {...},
    circle: {...},
    cross: {...},
    down: {...},
    left: {...},
    right: {...},
    up: {...}
  }

If an entry is specified, it has to be an object. An empty object `{}` means the
button shall be disabled explicitly. To use the button, a `click` handler needs
to be specified. Also buttons can be labeled and highlighted:

* ``click`` (Function) [mandatory]: called when the button is pressed
* ``label`` (String): label of the button in the context menu
* ``highlight`` (Boolean): button on the pilot will be colored


The values of all currently focused steps plus the context-menu's default
semantics and colors (with lowest priority) are merged together to form the
actually available pilot buttons with their click handlers, labels and
highlighting.

Interaction via mouse
^^^^^^^^^^^^^^^^^^^^^

Interaction with components is only possible, if the surrounding step is
focused. To avoid the necessity to click twice on an element to first focus the
step and afterwards actually click the element, the class ``.clickable`` can be
added. Furthermore the class ``.pointer`` is provided, to change the cursor to a
pointer when hovering an element with named class.

.. _grid-points:

Robot poses in ``pattern`` and ``points``
`````````````````````````````````````````

Examples for groups could be ``points`` and ``pattern`` groups. They enable
the execution of multiple similar robot movements by teaching a reference pose
multiple times. To use an app within the groups, one of the poses has to be
marked as the *reference*, by setting the ``reference`` parameter to true on
the corresponding component. Non-reference poses listed in the ``relatedPoses``
parameter have to be taught only once and are then shifted according to the
reference. By default, poses are relative to the reference pose which is taught
first. However, if the parameter ``isRelativeToFirst`` is set to false, they
are relative to the reference pose which is taught last. It is recommended to
set ``isRelativeToFirst`` to false for poses, which are taught after the
reference pose(s).

Points
^^^^^^

Robot poses inside an app inside a ``points`` group will be displayed in a list
similar to the ``robot-pose-carousel`` as depicted in the following image.

.. image:: _static/points.png

For the positioning of the steps inside the context menu it has to be taken into
account to which pose (the first or the last) the non-reference poses are relative.
In the image above, the *Position above* is relative to the first taught
reference pose.

Pattern
^^^^^^^

In a ``pattern`` group, the dimensions of a pattern and three corners are
taught. The ``robot-pose`` will be replaced with a ``grid-widget``, as depicted
in the following image.

.. image:: _static/grid.png

The markup passed to the ``robot-pose`` has to be positioned to fit the corners,
i.e. the center of the image(s) has to be specified.
In our example, we add a ``[top]`` and ``[left]`` attribute to the ``.pose-content``.

.. code-block:: html

  <robot-pose params="reference: true, ... ">
    <div class="pose-content" top="95px" left="65px">
      ...
    </div>
  </robot-pose>

By default, the corners will be taught in the following order

* top-left
* top-right
* bottom-right

By setting the parameter ``isBottomRightFirst`` to true, the order will change to

* bottom-right
* bottom-left
* top-left