App APIđź”—

Every app created in the Builder has an API object that will be published as a remote object when the app is run. The API is managed using the API editor side panel.

When designing an API it is important to understand the difference between things that are shared by all clients and those that are not. When an app is started, it will run exactly one instance of the processing graph and publish exactly one instance of the API object, which is therefore shared by all clients. Any change made to the state of that object will be seen by them all and any signal sent can be captured by any of the clients.

Propertiesđź”—

Properties are used to parameterize the execution of the processing graph and thus alter the state of the graph. Once a parameter is set, the effect will be seen by all clients.

To add an input parameter as a property to the API, right-click it and select “Publish app API property” from the context menu. The Builder will create a new property and a signal that can be used to listen changes to the property.

Once you have the app with a published property running, setting the value of the property in a JavaScript client is straightforward:

const va = VisionAppster;
const apiCtrl = new va.RemoteObject('http://localhost:2015/apis/com.test/1');
apiCtrl.connect().then(api => api.propertyName.set(123));

The same can be done on the command line:

curl -X PUT -d 123 http://localhost:2015/apis/com.test/1/properties/propertyName

Signalsđź”—

Signals can be connected to by anybody, and all clients will receive the same value. Each API property has an associated signal that emits the new value of the property whenever it changes.

To manually add an output as a signal to the API object, right-click the parameter and select “Publish app API signal” from the context menu.

In JavaScript, one can connect a handler function (here, console.log) to published signals:

const va = VisionAppster;
const apiCtrl = new va.RemoteObject('http://localhost:2015/apis/com.test/1');
apiCtrl.connect().then(api => api.$signalName.connect(console.log));

There is no easy way to listen to signals on the command line as a WebSocket client is required.

Single input functionsđź”—

Connectable inputs can be individually published as functions in the API. When such a function is called, the function’s input argument will be sent to the tool’s input as if it was connected to another tool. Any client can call the function, and everybody will also be able to observe the processing results.

It is rarely useful to publish inputs in intermediate tools this way. Doing so is an easy way of locking up the processing graph. Single input functions may however come in handy in situations where you want to provide an external trigger to a processing pipeline by publishing the very first input as a function.

As a general rule you should create at most one single input function to a single processing pipeline.

To publish an input parameter as a function, select “Publish app API function” from its context menu.

Single input functions take a single input parameter and return no value. Calling them from JavaScript is easy:

const va = VisionAppster;
const apiCtrl = new va.RemoteObject('http://localhost:2015/apis/com.test/1');
apiCtrl.connect().then(api => {
  api.singleInputFunction(3.14);
});

If the input parameter requires no special encoding, a simple GET request does the same thing:

curl http://localhost:2015/apis/com.test/1/functions/singeInputFunction?3.14

Tool functionsđź”—

In applications where the data to be analyzed is sent by the client one usually wants the results to be delivered to the sending client only. In this case, signals are not the correct way of sending the results as they can be connected to by anybody.

If you intend to run an app in VisionAppster Cloud, the public API of the app must consist of 1-N tool functions and nothing else.

It is possible to make any tool in the analysis graph callable as an API function. Such functions differ from single input functions in two important ways:

  1. All input parameters are passed in one call.

  2. The results are delivered using callback functions supplied by the client to the calling client only.

Any tool can be exposed as a function by simply checking the “Publish tool function API” box in the tool’s properties (opens from context menu). The dialog lets you to choose which inputs and outputs of the tool are actually published.

When a tool is published to the API, the Builder creates two overloaded versions of the function:

  1. An asynchronous version that will push results back using callback functions via the client’s return channel. The created API function will take three arguments at a minimum:

    • Tool input arguments, one for each published input. There needs to be at least one published input.

    • Success callback function. If the call succeeds, this function will be passed the values of the published outputs.

    • Failure callback function. If the call fails, this function will be passed a string describing the error.

    The signature of the function is (inputs..., success(outputs...), error(message)).

  2. A synchronous version that blocks until a result has been received and encodes the output values (if any) to the response body. If there is only one output, it will be written to the response body as such. If there are many outputs, they will be encoded as an array. In both cases, the response will be encoded according to the “Accept” request header.

    The signature of the function is either (inputs...), (inputs...) -> output or (inputs...) -> (outputs...), depending on the number of output parameters.

For example, if the binarize tool was published as a function, the signatures of the two function overloads would be:

binarize: (image: Image,
           level: int,
           invert: bool,
           success: (image: Image),
           failure: (message: String))

binarize: (image: Image,
           level: int,
           invert: bool) -> Image

In the first overload, the first three parameters are the inputs of the binarize tool. The fourth parameter is the success callback function that will be passed an image, which is the output of the tool. The last parameter is the failure callback that will be called with a reason phrase if the call fails. The function returns immediately and calls either success or failure later through the client’s return channel.

In JavaScript, calling such a function would happen like this:

const va = VisionAppster;
const apiCtrl = new va.RemoteObject('http://localhost:2015/apis/com.test/1');
apiCtrl.connect().then(api => {
  api.binarize.async(new va.Image(5, 5, va.Image.Type.Gray8),
                     127,
                     true,
                     image => console.log('Success', image),
                     error => console.log('Error', error));
});

The example doesn’t do anything useful with the result, it just writes either the result image or the error message to the console.

The synchronous version of the function is easier to use from programming languages that don’t yet have VisionAppster client API libraries. For example, the function could be called using curl as follows:

curl -X POST -H "Content-Type: image/jpeg" -H "Accept: image/jpeg" \
  --data-binary @input.jpeg \
  http://localhost:2015/apis/com.test/1/functions/binarize \
  > output.jpeg

Note however that this only works for a subset of possible API functions. For example, if the function requires multiple images as input or returns a tensor, you need to resort to a custom encoding scheme that is supported by the client API library only.

Note that although we used a singe tool as an example here, native tools aren’t usually published as API functions by themselves. Instead, one usually wants to create a compound that bundles many tools into one and publish that as an API function.

API editorđź”—

The properties, functions and signals of the API object are shown in the API editor that opens by clicking the small API icon at the upper right corner of the Builder’s workspace. The API editor lets you change the names of the published API entries and to see which parameters or tools are linked to which entries. Try clicking the names to see where they are linked to.

Designing an APIđź”—

To make your API consistent and usable for the widest possible range of client software you should adhere to the following guidelines. The VisionAppster client API library will work with all types of APIs, but all of the required protocols and encoding schemes aren’t widely available elsewhere.

  1. Avoid types that are not natively supported by JSON in the interface. Data types such as tables, tensors, matrices or 16-bit floats cannot be directly represented as JSON and require custom en/decoders. For data type conversions, you can use conversion tools such as Image to Tensor or Tensor to Image. In JavaScript, you can convert matrices and tensors to JS arrays, for example. Returning a JS object is also a good idea.

  2. If an API function takes an image as input, make it the first input argument. This will let client software to easily send the image as a POST request body.

  3. If an API function produces an image as output, do not provide other outputs. Multiple return values need to be always encoded in special ways, making them more difficult to parse at the receiving end.

  4. Start API entry (function, property, signal) names with a lowercase letter. Capitalize all other words. Use verbs for functions. Examples:

    • Property: inputParameter

    • Function: sendImage

    • Signal: analyzed