Image classifier demo shows you how to identify objects which belong to one of 1000 ImageNet categories. In this demo we use a deep convolutional neural network called ResNet-50. The network is stored in ONNX format. Many more classifier models are available in GitHub.
Let us walk through the schema below top-down.
- The images originate from the virtual camera configured in Image Source.
- Convert Colors takes an image in any supported format and converts it to uncompressed RGB. This tool is necessary because our intent is to analyze images sent by arbitrary users.
- Scale Image Tool scales the image to the size required by the image classifier model. In this case, the size is 224 x 224 pixels.
- Image to Tensor converts a color image into a tensor of shape 1 x 3 x
width. 1 is the number of images per tensor (N, batch size), 3 is the number of color channels (C) in an RGB image. Height (H) and width (W) are both 244 pixels. Intensity normalization is also applied to each pixel as required by the classifier model.
- Run Onnx Model runs the ResNet-50 model. The model file is selected with a file dialog which appears by clicking the
Model Fileinput parameter. Input and output sockets appear after the model file has been loaded. The output of this model is a tensor of shape 1 x 1000 which contains confidence levels for each of the 1000 ImageNet object classes. Note that in this case the model has been saved as a resource file and will thus be bundled with the app. Alternatively, you can load the model from a local disk.
- Tensor to Classification inputs the tensor of confidences, selects those that exceed the given
Confidence Thresholdand sorts them into descending order. Only up to
Max Result Countelements are taken. The
Confidenceoutputs contain the indices of the selected tensor and the corresponding confidence values.
Max Result Countmost likely results into readable strings.
The script code is given below. All 1000 classes in the
classNames array are not shown.
Here is an example of an input image and a list of 5 most likely results.
The project has an Image Source connected to the image classifier compound tool, so it can be easily tested in the Builder. Our target is however to build an API to which images are sent from clients. For this, the app doesn't need an internal image source.
- Remove the image source tool by right clicking it and selecting "Remove tool" from the menu.
- Close the classifier compound tool by right-clicking the outer light gray box containg the other tools below the image source and by selecting "Close compound" from the menu.
- Right-click the closed "Classify Image" compound tool and select "Tool properties..." from the menu.
- Check "Publish tool function API" and mark both input and output as published. Accept the dialog. The tool's illustration in the processing graph now shows an API symbol.
- Go to the application API editor by clicking the API button in the top right corner of the Builder. The tool function API should now be visible in the "Functions" section. The default name of the API is now "Compound", change it by clicking the "Compound" text in the rightmost column. Change the name to "classifyImage". This will be the name of the function in the published API. The name is case sensitive. Since the bundled web app already uses this name, it must be typed exactly right.
- Save the modified project and select "Package this app" from the file menu. In the package dialog, go to the "Remote" field and upload the package to "localhost".
- Browse to the Components tab on the Engine Front Page and the uploaded package should be visible there. Mark the package and install it.
- On the Apps tab, click "Start" to start the app and publish the API. A ready-made package with these modifications is also available here.
- Now, let us test the server application API with a web UI application. Download Web UI app and install it to the engine via the Components tab. Then click it on the Apps tab.