Client APIs

The easiest way to communicate with the VisionAppster Engine is to use one of the provided client API libraries:

VisionAppster Engine services

The services provided by the VisionAppster Engine are implemented as remote objects that can be accessed through an API that (for the applicable part) follows REST principles. The central services of the Engine are listed below. Note that you need to have the VisionAppster Engine installed and running locally (localhost:2015) to be able to access the links.

Application engine info
is mapped to /info/ on the server. It provides general information of the running AE instance, such as the version number.
Application manager
is responsible for managing installed apps and app repositories. It is also used for starting and stopping the apps. In the remote interface, it is available at /manager/.
Cameras
are controlled by the AE using device-specific camera drivers. All found cameras are published as remote objects at /cameras/.
Apps
are stored as app packets in a directory system that is mapped to /apps/. App packets are zip files that can be accessed over HTTP as if they were directories by appending a slash to the name.
App APIs
are published under /apis/. A VisionAppster app can publish any number of API objects, and they will appear under this URI. If no apps are running on the AE, the directory will be empty.

To be able to access any of these services you need to connect your client to the service's root URI. To gain deeper understanding of the underlying technology, read on.

Remote object interface

To get started with the client APIs, it is important to understand some central concepts and how they are mapped to the remote object API. To get the most out of this document you should make sure you have the VisionAppster Engine running on your computer. The embedded links can then be used to inspect and control it.

The VisionAppster Engine provides a built-in HTTP server that makes objects accessible through HTTP requests. The remote object API is modeled after REST principles as far as possible. Function calls and signals don't however naturally fit into the REST model and are handled in a different manner. Standard protocols and data formats are however used for all communication.

The remote object system supports a few different encoding schemes. In most cases, the most convenient way of encoding request and response bodies is JSON. When submitting a JSON request, the client will specify "application/json" as the "Content-Type" header. A JSON response is requested by also setting the "Accept" header to "application/json". When calling functions or setting properties, the platform automatically performs safe type conversions (such as integer to double).

Objects

VisionAppster apps are composed of objects. Each object may have an arbitrary number of properties, functions and signals. These concepts can be found in many programming languages in a form or another and are assumed to be familiar to the reader.

In the HTTP interface, each object has a base URI relative to the server's root. For example, an instance of an Application Manager object is mapped to manager/. A GET request to this URI will produce a "directory" listing that shows the structure of a remote object.

Each object instance has a globally unique ID that can be retrieved by a GET request. The ID will change if a server goes down and the object is recreated.

The ping URI is for application-level connectivity checking and for keeping a dynamically created temporary object instance alive if no other requests are made.

Properties

Properties are variables that control the behavior or appearance of an object. Generally, properties describe the object's current state, and they can be thought of as the member variables or fields of an object. Property values are specific to a single object instance.

Although properties are similar to member variables, they are technically implemented in a different way. The value of a property is set through a setter function that may validate the value before accepting it. Therefore, it is not guaranteed that a property actually assumes the value one sets to it. Properties also usually have a change notifier signal, which lets one to conveniently and efficiently detect changes.

While changing the value of a property may seemingly cause an action (such as an animation in a user interface), properties aren't used for invoking functionality. That is what functions are for.

Submitting a GET request to properties/ will list properties. Each property declaration contains optional qualifier flags such as const and volatile, a type name, a unique property name and optionally a change signal. A property may be const and thus non-settable but still change value and emit a change signal.

The current value of a property can be read by sending a GET request to properties/propertyname. For example, a GET request to properties/allApps will return a list of all installed applications.

Submitting a POST request to a property's URI will change the value of the property. The new value is sent in the request body, usually as JSON. If the final value of the property after the PUT request is different from the received value, the server will respond with the final value.

When the value of a property changes, a change notification signal will be sent to all registered clients. A client can avoid receiving an unnecessary signal when setting a property by submitting an "Client-ID" request header with a randomly generated client identifier. See signals below for further details.

Functions

Functions provide a way to invoke actions on an object. A function can take an arbitrary number of input parameters and optionally return a value.

The functions of a remote object are listed at functions/. Each declaration consists of an optional return type, a name and a (possibly empty) list of parameter types. There may be multiple overloaded versions of the same function, each taking a different set of parameters. When an overloaded function is called, the server uses passed parameter types to find a matching overload.

When a function is called, its arguments are passed as an array in a POST request body. It is also possible to pass function arguments as query parameter in a GET request, but POST requests should be preferred. A GET request is mainly useful when no arguments are passed to the function call. For example, the hasError() and clearError() functions can be called by sending a GET request.

It is possible that a function declares default values for its parameters. Such parameters can be left out of the parameter list when calling a function.

It is also possible that a function declares names for its arguments. Such functions can also be called with a map of arguments instead of an array. For example, a function setName(name: string) can be called with a JSON object as the POST request body: {"name": "John"}.

If a function returns no value, the server responds with an empty "200 OK" message or an appropriate error code. If there is a return value, it will be encoded as specified by the "Accept" header. Let's assume the server provides a remote function called "pass" that just returns a string it is given as an argument. Calling the function would be a POST request:

POST /myobject/functions/pass HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Accept: application/json
Content-Length: 15

"Hello, World!"

The server's response would be something like this:

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 15

"Hello, World!"

If a function has many parameters that cannot be encoded in the same way (e.g. JSON), the arguments can be passed as a MIME multipart message. For example, if a function takes an image and an integer as arguments, the request could be something like this:

POST /myobject/functions/threshold HTTP/1.1
Host: localhost:2015
Content-Type: multipart/mixed; boundary=boundary_marker
Accept: image/png

Content-Type: image/png
Content-Length: 12345

... image data here ...

--boundary_marker
Content-Type: application/json
Content-Length: 3

127
--boundary_marker

It is possible to call a remote function asynchronously. In this case the server won't wait for the function to complete before responding to the client. Instead, the call will be put to a queue and executed later. The results of the function will be pushed to the client's return channel once the function is done. The format of the response is specified by the Media-Type header. The client needs to give a source ID for the call so that it can recognize the return value when it appears in the return channel.

To asynchronously call the function above:

POST /myobject/functions/pass HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Media-Type: application/json
Client-ID: S3cR3t
Source-ID: 29
Content-Length: 15

"Hello, World!"

The server would return with "200 OK" and put the call into a call queue. Once done, the return value would be pushed to the client's (S3cR3t) return channel with source ID 29.

Signals

Signals are used as a way to notify listeners about changes in an object's state or to emit the results of an asynchronous function call. A signal may pass any number of (including zero) values, and it can be connected to any number of functions with a matching parameter set.

Most programming languages do not provide signals as a built-in concept. The same functionality can be achieved for example with Java-style listener interfaces. Signals are however conceptually cleaner and easier to use in practice.

Submitting a GET request to signals/ will produce a list of signals. Signal declarations are similar to functions, but there is no return value.

Property change signals are a special kind of a signal that will be emitted whenever the value of a property changes. If there is a change signal associated with a property, the signature of the signal will appear in the notifier field of the property description.

Callback functions

Callback functions provide another way for the remote object to push data to clients. They also make it possible to request data from the client. There are two types of callbacks: permanent ones and callback function arguments.

Permanent callbacks can be used to implement a "listener" design pattern. Any client can register itself as a listener to a callback function and receive the arguments of the callback function through the return channel (see below). The delivery mechanism is the same as with signals, but only one listener is allowed per function.

Unlike signals, callbacks may return a value, which will be passed back to the server through the return channel. Even if a callback function returns no value, the server receives an acknowledgement once the function call finishes. Permanent callbacks are listed at callbacks/.

In addition to permanent callbacks, a remote function may have callback arguments. Again, the delivery mechanism is the same as with signals, but the arguments of the callback function will only be delivered to the client that made the request. This makes it possible to implement for example asynchronous functions that have no return value but push their results to the calling client after a processing delay.

Pushable sources and channels

The remote object system has a concept of a pushable source that lets the programmer to define many different types of data that be sent to the client from the server side. Pushable sources are identified by their relative URIs on the server and may present many different kinds of data sources. Signals are the most commonly used type of a pushable source.

To be able to receive push notifications, the client must first open a channel. This happens by establishing a WebSocket connection to channels/new. The WebSocket URL will be, for example, ws://localhost:2015/manager/channels/new?client=3dcbff79-3f15-4a80-b7ca-b669cc0b3209. The client ID can be passed in the "Client-ID" request header if your WebSocket library allows it (browsers don't). The opened socket will then be used whenever a notification needs to be sent for the client.

The client ID should be a cryptographically strong UUID. This makes it unlikely that another client will guess it and steal or modify the channel. Once the channel has been established, it can be controlled by making requests to channels/<client-id>. In the example above, the root URI of the new channel would be /manager/channels/3dcbff79-3f15-4a80-b7ca-b669cc0b3209.

The client is in control of selecting which notifications to receive. It can add a pushable source to the channel by issuing a PUT request to channels/<channel-id>/sources/<source-id>. For example, a PUT request to /manager/channels/3dcbff79-3f15-4a80-b7ca-b669cc0b3209/sources/0 with the JSON request body {"sourceUri": "signals/runningAppsChanged"} would tell the server to notify the client whenever the runningAppsChanged signal is emitted.

Note that just like with functions, there may be multiple overloaded versions of a signal. In such a case the client must give the full signature of the signal as the sourceUri parameter. For example:

PUT /myobject/channels/<channel-id>/sources/0 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 44

{"sourceUri":"signals/booleanChanged(bool)"}

The client is responsible for picking a unique, non-negative 32-bit integer (0-2147483647) as a source ID that identifies the signal. In the example, the source ID is zero. The server will send the source ID when it pushes data to the channel so that the client can identify the signal. A DELETE request to /manager/channels/3dcbff79-3f15-4a80-b7ca-b669cc0b3209/sources/0 will tell the server to stop pushing that signal.

Connecting to multiple signals happens by giving each signal a unique ID number in the context of the channel. For example, to subscribe to the barCode and image signals, one can issue the following requests (channel ID is "d351a1612998" for illustration):

PUT /myobject/channels/d351a1612998/sources/0 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 31

{"sourceUri":"signals/barCode"}

PUT /myobject/channels/d351a1612998/sources/1 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 56

{"sourceUri":"signals/image","mediaType":["image/jpeg"]}

The mediaType parameter specifies how the parameters of the signal should be encoded by the server. See details below.

Connected sources can be listed by sending a GET request to channels/<channel-id>/sources/. A channel is killed by sending a DELETE request to channels/<channel-id>. Finally, a broken connection can be re-established by initiating a WebSocket connection to channels/<channel-id>. The server will automatically delete the channel if a broken connection isn't re-established within a "reasonable" time.

Source parameters

Each pushable source can be individually parameterized. In the examples above, the mediaType parameter was used to specify encoding for data pushed back from the server. Other configurable parameters are:

qos

Quality of service level. QoS is specified as a numeric value in the range [-1,3].

Unspecified (-1)
Use the channel's default.
At most once (0)
A.k.a fire and forget. send the message at most once and don't expect an acknowledgment.
Last at most once (1)
Same as 0, but new messages from the same source are allowed to overwrite an older message if it hasn't been sent yet. The last message will be sent once.
At least once (2)
Make sure receiver gets the message. Resend after a while if the client fails to acknowledge.
Last at least once (3)
Make sure receiver gets the last message from a source. Same as 2, but his mode allows the server to update message content until it has been acknowledged.
maxAge
The maximum number of milliseconds the message will be kept in the outgoing message queue. 0 means unlimited, -1 uses channel default.

Receiving signals and callbacks

The server supports a few different transport protocols for notifications. This section describes the WebSocket implementation.

Messages between the client and the server are sent as WebSocket binary frames. The payload of each WebSocket frame starts with an eight-byte message header that has the following format:

 0               1               2               3
 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+---------------+-+-------------+-------------------------------+
| message type  |A|    flags    |      source ID (2 MSBs)       |
|               |C|             |                               |
|               |K|             |                               |
+---------------+-------------+-+-------------------------------+
|     source ID (2 LSBs)      |R|        message index          |
|                             |E|                               |
|                             |S|                               |
+-----------------------------+-+-------------------------------+
  • The first byte contains a message type that describes how the rest of the message should be interpreted:

    1. partial: The message contains a part of a sequence. One or more parts will follow. A "sequence" can be for example an array of signal arguments. A part of a sequence is a single, individually encoded element in the array.
    2. final: The message contains the final part of a sequence. If a sequence contains only one part, the first message will be final.
    3. complete: The message contains a complete sequence of messages, all encoded in a single message. For example, a JSON encoded array of signal parameters is a complete message.
    4. error: The message contains an error. Errors are objects with at least a message field.
  • The next eight bits contain control flags. Bit 0 indicates whether the server expects the client to acknowledge the message or not. If the ACK bit is set, the client must send a reply, optionally with return data.
  • The next four bytes contain the source ID the client gave when registering a source as a 32-bit little-endian integer. The MSB (bit
  1. is reserved and must be zero.
  • The last field is an unsigned 16-bit little-endian sequence number that is specific to the source. This lets the client to detect dropped messages in case the server runs out of bandwidth.
  • The rest of the payload is data that is interpreted according to message type.

Usually, the arguments of a signal or a callback function are encoded as a JSON array in the order they appear in the signal's or callback's declaration and sent as a complete message in a single WebSocket frame (which may get fragmented in transport). When registering a pushable source to a channel, the client can however request splitting arguments to successive frames by giving a preferred encoding scheme for each argument with the mediaType parameter. For example:

PUT /myobject/channels/<channel-id>/sources/0 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 54

{"sourceUri":"signals/image","mediaType":["image/jpeg"]}

This will tell the server to connect to a signal called "image" and to encode the signal's single parameter as a JPEG in a single final message. When the signal is received, the client can decode the body of the WebSocket frame (after skipping the eight header bytes) as a JPEG image.

If a signal has many parameters, they will be sent in successive WebSocket frames in the order the parameters appear in the declaration of the signal. The messages will be partial up to the last one, which will be final.

It should be noted that "image/jpeg" requests encoding the whole parameter array as an image (which is not possible) whereas ["image/jpeg"] causes the image parameter itself to encoded and sent as a separate message.

The mediaType parameter can also be used to selectively pick signal parameters. Let's assume the object has a signal with the signature foobar: (v1: int32, v2: Image) ->> void. To receive only the image one can give null as the media type for the first parameter:

PUT /myobject/channels/<channel-id>/sources/1 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 61

{"sourceUri":"signals/foobar","mediaType":[null,"image/png"]}

Specifying a null media type for all parameters disables the signal.

Callbacks are connected to in a similar manner, but the sourceUri is "callbacks/callbackName". Unlike signals, callbacks may return a value to the server. The encoding of the return value is specified by a replyMediaType parameter when registering a connection. For example:

PUT /myobject/channels/<channel-id>/sources/2 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 65

{"sourceUri":"callbacks/getImage","replyMediaType":["image/png"]}

This would tell the server that the return value of the getImage callback will be encoded as a PNG. If replyMediaType is not specified, the channel's default will be used.

If the ACK bit of the flags field is set, the client must respond to a message. If there is no data to be sent back, an empty complete message will acknowledge the reception. If there is data to be sent back, it must be encoded as specified by replyMediaType.

The semantics of the replyMediaType parameter are the same as those of the mediaType parameter: a single value specifies the encoding for a complete sequence, and an array of values specifies the encoding for each individual element. Currently, the server does not support partial responses. Therefore, replyMediaType must be either a single string or an array containing a single string.

Callbacks as function arguments

Let us assume a server object provides a function that calculates the sum of two integer arguments and pushes the result back using a callback function. The signature of the function can be written as sum: (callback: (int) -> void, a: int, b: int) -> void, where callback is a function that will be invoked once the computation of a + b is ready. The sum function itself returns no value.

To be able to receive the asynchronous reply one needs to first set up a return channel as described above. The generated client ID must then be passed to the sum function call so that the server knows which channel to use when calling back. This happens by adding a Client-ID header to the request. Callback function arguments are replaced with the numeric source ID that will be used when pushing that callback's data back to the return channel. An example:

POST /myobject/functions/sum HTTP/1.1
Host: localhost:2015
Client-ID: R4Nd0M
Source-ID: 30
Content-Type: application/json
Content-Length: 9

[314,1,2]

This tells the server to calculate 1 + 2 and push the result back to the return channel of client R4Nd0M using 314 as the source ID. No other configuration has been given, so the server will encode the arguments of the callback function using the channel's default encoding. Since the request contains a Source-ID header, the call to sum itself is asynchronous, and the server won't wait for the function to complete. When it is done, an empty complete message with the given source ID (30) will be pushed to the client through the return channel.

If there is a need to receive the callback's arguments in a non-default format, it is possible to register and configure the source ID beforehand just like with signals:

PUT /myobject/channels/R4Nd0M/sources/314 HTTP/1.1
Host: localhost:2015
Content-Type: application/json
Content-Length: 34

{"mediaType":["application/json"]}

This would cause the result to be sent as an individual integer (3) instead of a one-element array ([3]). Unlike signals and permanent callbacks, function argument callbacks don't require a sourceUri parameter. The client identifies the callback solely based on the numeric source ID.

Permanent configuration is useful when the same parameters are used for all callback invocations. Temporary parameters that only apply during a single function call (which may cause multiple invocations of the callback) can be passed as an object type argument:

POST /myobject/functions/sum HTTP/1.1
Host: localhost:2015
Client-ID: R4Nd0M
Content-Type: application/json
Content-Length: 49

[{"id":315,"mediaType":["application/json"]},1,2]

This would cause an individual integer to be pushed back to the channel with 315 as the source ID. Since the mediaType parameter only applies to this call, further calls to sum without the parameter object would use default encoding.