Testing HTTP APIs

Web-based testing

The Engine’s web-based management interface contains an API tester that can be used to test the public HTTP interface of a running app. It is currently limited to testing tool API functions only. The tester lets one to define the input arguments of a function call and inspect the results graphically.

Once you have an app running in the Engine, you can browse to the APIS page to see its published API. Clicking the “Test” button opens an API tester. If you have the Builder running, its embedded web server provides the same functionality, by default in port 2020.

Clicking the “Test” button opens an API tester.

The tester lists the public API functions as collapsible tabs (accordion). In this case, the API consists of a single function: detectObjects. Its input parameters are shown in the first content box and output parameters in the second one. Both are initially empty.

The web page uses different display components for parameters depending on their data type. In this case, the single input parameter is an image while the output is an object, which will be displayed as JSON.

The third content box shows how the API can be used from different programming languages. The code examples are mostly self-contained; if there are prerequisites, they are explained in comments.

Initially, no input or output parameters are shown.

Once a value for each input parameter has been defined, the tester will call the API function. The system automatically interprets certain types of output parameters and is capable of providing visual feedback such as bounding boxes draw on the input image.

The API tester provides visualizations for certain output types.

Actions

If you want to run an app in VisionAppster cloud or automatically test its API for example in a continuous integration server, you need to craft a JSON document that defines actions for test and/or demonstration. The “actions.json” tab in the API example box generates one for you.

The API tester can generate action descriptions.

This document describes how to call the API function with the input parameters you have currently defined. If you copy the text and save it to a file called actions.json, you have a working action definition.

Multiple actions can be incorporated into the same document by copying and pasting them to the actions array. This is how you add demos to your cloud app in VisionAppster Store. An example:

{
  "componentId": "com.visionappster.apps.objectdetector/1",
  "actions": [
    {
      "function": "detectObjects",
      "description": "People and vehicles on a street.",
      "inputs": [
        "crowded_street.jpg"
      ]
    },
    {
      "function": "detectObjects",
      "description": "Different animals.",
      "inputs": [
        "horses_dogs_and_cats.jpg"
      ]
    }
  ]
}

The VisionAppster Store generates demos for your app using actions.json. It also automatically tests and benchmarks your app using the same data. When uploading a component to VisionAppster Store you should thus try to make sure your demo data is representative of actual data used in production. Otherwise, you may get misleading performance metrics.

In addition to actions.json, input data is needed. You need to combine these into a single file called actions.zip so that relative paths to images (and possibly other input data) are correct. A correctly formatted zip file in the example above could be created from the following directory structure:

myactions/
myactions/actions.json
myactions/crowded_street.jpg
myactions/horses_dogs_and_cats.jpg

To create an actions.zip for the Store, you would do this:

cd myactions
zip actions.zip *

That is, put everything in the root directory of the zip file.

Command-line testing

To use the command-line testing tool you need to Install Node.js first. Then, you can use the Node.js package manager to install the testing tool:

sudo npm install -g va-api-tester
# sudo is not needed on Windows

va-api-tester is a stand-alone tool that does not require an existing VisionAppster installation. You can connect it to any running VisionAppster Engine instance or let it run a self-contained Docker image for you. Once installed, a complete list of options is given by:

va-api-tester --help

As an input, the tester needs an actions.json file (see above:

va-api-tester --run actions.json --benchmark

This will use the input parameters defined in actions.json to call the listed API functions multiple times to estimate their end-to-end execution times. Note that referenced input files (images) need to be available in the same directory as actions.json. Alternatively, you can pass --data-root as a command-line argument. As an output, you’ll get a JSON document by default. XML output (xUnit) is also supported.

The --coverage option can be used to verify that your API meets VisionAppster Store requirements:

va-api-tester --run actions.json --coverage

In the third operation mode, --save, the tester goes through all actions and calls the corresponding API function once. It stores the input and output parameters of the call into .vabin files that can be used to demonstrate the API without actually running the app:

va-api-tester --run actions.json --save

If actions.json contains the two actions shown above, this will save two files: 00.vabin and 01.vabin. These files can be uploaded to the API tester web UI using the “Click to upload or drop .vabin files here.” control at the bottom of the page.

VisionAppster Store executes the exact same steps to generate demos for your cloud apps. To make sure your demos look exactly what you expect you should run tester locally first and see the result.