Tool APIs

VisionAppster Tool APIs are used for implementing new tools and integrating existing ones to the VisionAppster platform. The APIs make it easy to build dynamically loadable libraries that add new tools that can then be used as building blocks in vision applications. All tool APIs are built on top of a C API.

An important thing to note is that unlike the tools built into the VisionAppster platform, user-defined tools are not automatically available everywhere. It is the developer's responsibility to make sure the tools are built for all target platforms and installed there.

Terminology

Tool
A wrapper for a low-level function. Defines an arbitrary number of input and output parameters that work as connection points in a processing graph.
Processing graph
A directed acyclic graph that declares dependencies between processing steps.
Tool instance
A node in the processing graph. Different instances of the same tool may have different ideas of default values of arguments and the internal state of the tool, if the tool has state variables. A pointer to the tool instance is passed to the underlying function. By default, the instance only contains the current parameters, but state variables can also be added if needed.
Input parameter
A parameter that is passed to the tool.
Fixed input parameter
An input parameter that is user-adjustable but not connectable in the processing graph.
Optional input parameter
An input parameter that has a default value but is also connectable.
Required input parameter
An input parameter that has no default value and must be connected in the graph.
Output parameter
A parameter that is produced by calling the underlying function with the current input parameters. Output parameters can work as inputs to subsequent functions.
Plugin
A dynamically loadable machine code library that implements a known entry point function (va_get_plugin_info).
Component
A redistributable unit that may contain an arbitrary number of plugins. The VisionAppster platform uses components for sharing any content, not just plugins. The ID and the major version of a component (e.g. com.visionappster.demo/1) defines a namespace for the tools inside it.

Execution modes

The VisionAppster platform uses a pool of threads to distribute work across processing cores. Most built-in tools in the platform are reentrant. In practical terms this means that the outcome of a tool invocation only depends on its input parameters and not on any static or global non-constant data.

Whenever all input parameters required by a tool are available, the platform schedules the underlying function for execution. Since the outcome of a reentrant function depends only on the input parameters, it is always possible to execute any number of invocations to the function simultaneously. Therefore, one should always make tools reentrant if possible.

Sometimes, a tool must maintain mutable state variables. An example of such a tool is a counter that is incremented on each invocation. Such tools cannot be executed simultaneously unless the programmer manually ensures that concurrent access to the same state variables won't break the program. There are also technical limitations that may make it impossible to implement a tool in a reentrant way. For example, it may be necessary to hold state in thread-local variables, which are only visible if the call is always made from the same thread.

It is important to understand the difference between a tool implementation and a tool instance in a processing graph. Multiple instances of a tool may appear in different locations in the graph. For reentrant tools this makes no difference. If the tool needs mutable state, each of the instances has a different idea of the state of the tool.

The underlying function may be called from different threads, but each thread will maintain different state data. This means that all wrapped functions must be thread-safe even though they may not be reentrant. In practical terms, the state of a thread-safe tool must not be kept in global or static variables, but only in a storage that is specific to a tool instance.

The VisionAppster platform makes it possible to change the threading mode for each individual tool separately. If the default multi-threaded execution mode isn't suitable, there are two alternatives:

Non-threaded
No concurrent calls to the tool will be made by one tool instance. In other words, no concurrent calls with the same instance will be made. Successive calls come from arbitrary threads. Note that it may make sense to use the non-threaded mode for a reentrant tool; the tool may be so simple that queuing the input data and performing a context switch would cost more than just calling it whenever data is available.
Single-threaded
Each instance of the corresponding tool will be assigned a dedicated processing thread. No concurrent calls with the same instance will be made. Successive calls from the same tool instance always come from the same thread.

If a tool has no required input parameters and all optional input parameters are unconnected, the VisionAppster platform uses the single-threaded execution mode by default and calls the tool repeatedly in a loop.

Meta-data

To be able to call a function the VisionAppster platform only needs to know the types of its parameters. Therefore, other meta-data is not strictly required. It does however make it easier for the application programmer to use the tool in the Builder.

Meta-data are either static or dynamic values that give the Builder and other GUI tools hints on how to handle the parameter. Each input and output parameter can have an arbitrary number of meta-data entries. Each meta-data entry is identified by a text string that is composed of the name of the parameter the entry refers to, a separator and a key that identifies the entry. In the C API, underscore (_) is used as the separator. For example, the min meta-data for a parameter called value would be identified by value_min.

Meta-data entries can be either static or dynamic. The need for dynamic meta-data usually arises when the meta-data depends on the current value of a fixed input parameter. For example, it may be useful to selectively hide certain parameters.

It is important to note that meta-data entries are only hints for the GUI, and it may choose not to respect them. At run time there is no protection whatsoever: your tool implementation may receive a value that is larger than the declared min, for example. If it is important that this does not happen, the tool implementation must check the validity of its input data and return an error code if needed.

Recognized meta-data keys

The following list contains the meta-data keys most commonly used and supported by the default VisionAppster user interface components. It should be noted that no user interface is required to respect any of these and that custom components may provide additional values not listed here.

readOnly
A Boolean flag that indicates whether the value of the parameter can be written to or only read.
typeName
A hierarchical type specifier for the parameter.
doc
A text string containing documentation for the parameter. May contain HTML formatting.
enabled
A Boolean flag that indicates whether the parameter should be available or not. The user interface may hide or gray out parameters that are not currently enabled.
stored
A Boolean flag that tells whether the value of an optional or fixed parameter should be stored permanently when the processing graph is saved. By default, the current values of all optional or fixed read-write input parameters are stored.
min
Minimum allowed value. The type must be the same as that of the corresponding parameter.
max
Maximum allowed value. The type must be the same as that of the corresponding parameter. Alternatively, max can be the text string "GRAY_MAX". If the user works with images that have more than eight bits of depth, this value changes the maximum value according to the depth of the images.
step
If the value can only take discrete values, this parameter specifies the step between successive values. Usually used with min and max. The type must be the same as that of the corresponding parameter.
suggestedMin, suggestedMax, suggestedStep
Sames as min, max and step, but impose no hard limits. This is useful for parameters that usually take a certain range but can also take different values in special situations.
choices
An array of valid choices for the parameter. Usually used for enumerated parameters. Each entry in the array is an object with a name and a value key. See structured meta-data below.
systemDependent
If the tool cannot work without knowing something of the underlying system, it can provide fixed parameters that are marked system-dependent. When the user deploys an application containing system-dependent parameters, the VisionAppster platform makes sure there is a value assigned to the parameter. Typically, these parameters are addresses of attached hardware the application communicates with. The value of systemDependent is a Boolean flag.
columnNames
An array text strings that are shown in the column header of a matrix or table, replacing the auto-generated heading. If columnNames is given, it is assumed that the number of columns is fixed. For convenience, columnNames can also be given as a single, comma-separated text string. For example, a matrix representing a coordinate frame may define columnNames as "x,y,z,origin".
firstColumnIndex
The index of the first column of a matrix or table, if auto-generated column headings are used. If firstColumnIndex is given, the default alphabetical column headings will be replaced by numbers. Must be an int32_t.
columns
A fixed number of columns as an int32_t. Unnecessary if columnNames or columnDefinitions is given.
columnsEditable
A Boolean flag that enables or disables UI controls for changing the number of columns in a matrix or table.
minColumns
Minimum number of columns if columnsEditable is true. Zero by default. An int32_t.
maxColumns
Maximum number of columns if columnsEditable is true. Unlimited by default. An int32_t.
columnDefinitions
An array of object specifying meta-data for individual columns in a table. See structured meta-data below.
defaultValue
The value used for newly created table or matrix cells when new rows or columns are being added. The type of defaultValue can be anything, but it must match the typeName of the column definition object, if there is one.
firstRowIndex
The index of the first row of a matrix or table. Auto-numbering is started from one by default. Often, there is a need to change this to zero. Must be an int32_t.
rowNames, rows, rowsEditable, minRows, maxRows
Same as the corresponding meta keys for columns.

Types

Built-in types

To make all tools compatible with each other, the VisionAppster platform defines a set of common C types every function must use when passing data. To ensure matching word lengths across different architectures, the stdint types of the C99 standard are used. Compatible definitions are provided for compilers that do not support the C99 standard. Tool API implementations for different programming languages use the core C types behind the scenes.

Hierarchical type names

Often, semantically different types are represented by the same C data type by the VisionAppster platform. For example, coordinate frames, points and one-dimensional time series are all represented as matrices. To make the Builder aware of the semantics of a parameter, one can provide a type name in addition to the actual C data type.

The following types are recognized by standard user interface components:

enum
A list of mutually exclusive choices. Each choice is an object that has a name and a value that can be any data type.
enum/editable
A value that has a predefined list of choices but also allows editing the value manually.
enum/multiple
Multiple selection. Values must be integers and powers of two. The user interface will build a bit mask of selected values and pass that to the tool. This type is currently not supported by the Builder.
string/multiline
A multi-line string.
Matrix/frame
A 3D coordinate frame with translation (4-by-4 matrix).
Matrix/size
Width and height relative to a coordinate frame (1-by-2 matrix).
Matrix/point
A single point in two or three dimensions (1-by-2 or 1-by-3 matrix).
Matrix/pointset
A set of related points in two or three dimensions (N-by-2 or N-by-3 matrix).

Data type conversions

When any 3rd party library is integrated to the VisionAppster platform, the need for data type conversions arises. To make this easy for the user, the platform comes with some built-in conversions to common types. Currently, OpenCV types are supported. Generally, one should avoid making deep copies of data whenever possible.

Debugging

In rare cases, a carefully designed and meticulously implemented algorithm doesn't work exactly as intended once it compiles. If console output doesn't help, a desperate developer may reach to a debugger as a last resort. But there is a problem: you cannot easily start the VisionAppster Builder or Engine in a debugger.

What you can do, however, is to compile your tool plugin with debug symbols, deploy int normally and attach your debugger to a running VisionAppster process.

Linux

To debug a plugin in the Builder, launch the Builder normally and then attach the debugger to the va-builder process. To debug it in a locally running Engine, attach the debugger to the va-run process.

Windows

To debug a plugin in the Builder, launch the Builder normally and then attach the debugger to the va-builder.exe process.

The VisionAppster Engine service runs in the LOCALSERVICE account and administrator rights are required to debug your tool in an Engine running in it. So, launch your IDE with "Run as administrator" command and then attach the debugger to the appenginesrv.exe process. You may need to select option "Show processes from all users" to see the service process.