Web app🔗

Summary🔗

This recipe shows how to create a simple image analysis app and a web UI that communicates with it. We’ll be using a web app template that can be used with many types of API objects with minimal modifications.

Downloads🔗

Detailed description🔗

Image analysis app🔗

The app we are building consists of two tools: one that reads an image file and another that applies an edge detector to it. The fileName input is exposed as an API function (context menu: “Publish in API” → “As function”) and the image output as a signal (“Publish API signal”). The app is included in the project file, but you may want to try recreating it by yourself.

The complete edge detector app.🔗

Web app🔗

If you have the VisionAppster Engine running as a local service (Windows service, systemd service, Docker, …), you can install the provided .vapkg file (Component package) now as some of the links below point to the files installed there. It is assumed that the Engine runs in port 2015.

If you’d rather walk through the process first, you can come back and inspect the links in detail at the end. You can build and test the app without a running Engine instance.

A web app consists of at least one HTML file. The front page of the app must be called index.html. In this example, the full contents of the file are as follows:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link href="style.css" rel="stylesheet" type="text/css" media="all">
    <script src="/VisionAppster.js"></script>
    <script src="webapp.js"></script>
    <title>Web app template</title>
    <link href="/style/fonts.css" rel="stylesheet">
  </head>
  <body>
    <div class="header row">
      <ul id="sourceList" class="menuList">
        <li><img src="/visionappster.png"></li>
        <li><a id="webcamOption">Webcam</a></li>
        <li>
          <input id="imageInput" style="display:none;" type="file" accept="*.png,*.jpg,*.jpeg"/>
          <a id="imageFileOption">Image file...</a>
        </li>
        <li><a id="textOption">Text...</a></li>
      </ul>
    </div>
    <div class="row content">
      <div id="resultContainer">
        <div id="inputDisplay" class="display">
          <video id="video" autoplay playsinline></video>
          <canvas id="sourceImage" width="500" height="500"></canvas>
        </div>
        <div id="outputDisplay" class="display">
          <canvas id="resultImage" class="result" width="500" height="500"></canvas>
          <table id="resultTable">
            <thead id="resultTableHead">
            </thead>
            <tbody id="resultTableBody">
            </tbody>
          </table>
        </div>
      </div>
    </div>
    <!-- Custom style -->
    <style>
      #webcamOption, #imageFileOption, #inputDisplay {
        display: none;
      }
    </style>
  </body>
</html>

This is a generic template that works with webapp.js. The <style> block at the end is used to hide elements that are not needed in this app.

Since the web app is intended to be installed to the VisionAppster Engine, it makes references to resources available on the built-in web server instead of including everything in the app:

/VisionAppster.js contains the (minified) JavaScript client library.
/style/fonts.css and /visionappster.png are for styling the web page.

In addition, the page links to two files that are included in the app itself:

style.css contains style definitions.
webapp.js contains the actual logic of the app.

You are encouraged to look at the source code. It works with most API functions that have a single input parameter and a single output parameter. If your API has a different structure, you need to modify the code.

The only piece of code that usually needs to be modified is at the top:

const config = {
  appId: 'com.example.edgedetector/1',
  apiFunction: 'sendFileName',
  resultSignal: 'image',
  resultHandler: displayImage,
  binaryImage: true
};

The configurable parameters are as follows:

appId

The full component ID of your image analysis app. This may be different from the ID of your web app if you decide to ship them separately.

apiFunction

The name of the API function to call. This may be either a tool function or a single input function. In the latter case, resultSignal is needed to receive processing results. Note that this template only supports functions that take a single argument. If the image analysis app produces signals spontaneously, without external input, apiFunction can be left unspecified.

resultSignal

The name the signal that passes processing results. Not needed with tool functions.

resultHandler

The function that displays the processing result. You need to pick the correct function depending on the type of the result. The template provides the following display functions:

displayImage: Draws an output image on the web page.
displayTable: Creates an HTML table out of a va.Table or va.Matrix.
displayText: Converts any object to text and displays it as such.
displayBoundingBoxes: Draws bounding boxes on top of the source image.

Testing🔗

You can test a web app just like you would test any web page: copy the files in a directory that is published by a web server and open index.html in a browser.

Unzip the project file to your projects directory (by default $HOME/VisionAppster/Projects).
Open the project in VisionAppster Builder (Press Ctrl+O, select “Edge detector” and click “OK” or “Choose”, whichever your operating system’s file dialog happens to offer.)
Start the app (Press F5). Make sure that the text on the status line changes to “Running”.

Copy or link the web app to the packages/web directory under the web server’s root. On Linux:

cd ~/VisionAppster/packages
ln -s "../Projects/Edge detector/analysisapp/web" .

On Windows:

cd %USERPROFILE%\VisionAppster
xcopy "Projects\Edge detector\analysisapp\web" packages\web\ /S

Point your browser to http://localhost:2020/packages/web/index.html. This assumes that the Builder is using the default port (2020). The port can be changed in “File” → “Options…” → “Web server”.
Click “TEXT…” in the page header and type in an absolute path to a file on your local disk.

The edge detector highlights contours in the input image.🔗

If you created the app from scratch, you need to change its component ID to com.example.edgedetector in “File” → “App properties…” before starting it. This is required because the app’s API object the web app connects to is identified by the component ID.

The web app calls the sendFileName API function using the user-supplied path as a parameter. It will end up to the fileName input of the Read Image File tool. If you switch back to the Builder, you’ll see the image there if you click the image output of the tool.

Once the image has passed through the analysis pipeline, the result will appear in the image output of the Detect Edges tool. It is exposed as a signal in the API and our web app is listening to it. Therefore, the display will change whenever an image is processed.

The result of the edge detector is a binary image, which is why binaryImage was specified in the configuration. As a result, the gray levels of the output image will be scaled from [0, 1] to [0, 255].

It is important to note that the image signal is not a return value of the sendFileName function. Instead, it will be emitted whenever the output changes, and all listeners will receive the image. If another client calls the sendFileName function, you’ll be able to see the result. Try it:

image=/path/to/image.jpg
curl http://localhost:2020/apis/builder/com.example.edgedetector/1/functions/sendFileName?$image

Creating and installing a component package🔗

To build an installable web app component, you need to copy the three files (index.html, webapp.js and style.css) into any directory and build a component out of it. Here, we’ll make a single component package that contains both the analysis app and the web UI.

If you created the app from scratch, copy the whole web directory from VisionAppster/packages to VisionAppster/Projects/Edge detector/analysisapp. In VisionAppster Builder, save the project (Ctrl+S) and package the app: “File” → “Package this app…” → “Local” → “Create”.

Save the component package to the default location.🔗

The package is now ready to be installed to the VisionAppster Engine. If you did not install the ready-made package at the beginning, you can install the new one now.

Browse to the components page on the management interface of your Engine instance. If you run a local instance, the URL is http://localhost:2015/#components by default. The package can be installed on another machine as well. In the picture below, we are using a local instance that shows the web/ directory we used for testing the app in “Uploaded packages”. Install the package by selecting it and clicking the “Install” button twice.

Install the package.🔗

The apps page should now look like this:

The apps page contains two apps.🔗

Click on the com.example.edgedetector/1/web/ link to open the web app. If the server app isn’t running, the web app will start it automatically.

Troubleshooting🔗

I get a “Not Found” error.

The app is not running. If the app has not been started, its API is not available and the web page cannot connect to it. If you are testing the app in the Builder, make sure you started it (F5).
The component ID is wrong. The API object is identified by the component ID. Double-check that the component ID in webapp.js matches the one set in “File” → “App properties”. You may need to restart the app in the Builder and refresh the web page after the change.
API URL is wrong. The web app template uses a simple heuristic to automatically guess the correct URL of the analysis app’s API object: if the server’s port is 2020, the API URL will be /apis/builder/. Otherwise, it will be /apis/. You can override this by setting apiUrl in the config object in webapp.js.

Can I test the app when it is running in a system-level service?

Yes. On Linux, a system level service usually serves files from /opt/visionappster/packages. On Windows, the root directory is C:\Windows\ServiceProfiles\LocalService\VisionAppster\packages. You can copy the web app there. This requires root/administrator privileges.

Can I connect to an app running on another Engine instance?

Yes. If you host the web front-end on a dedicated web server, you need to copy the files that are linked from the Engine’s web server (VisionAppster.js, fonts.css, visionappster.png) and adjust the HTML template accordingly.

Add apiUrl to the config object in webapp.js. For example: apiUrl: 'http://hostname.lan:2015/apis/'.
Enable CORS on the server.