OpenCV Integration

The OpenCV integration API consists of a single OpenCvMat class that automatically converts the image and matrix types used by the VisionAppster platform to the corresponding OpenCV matrix and image types. It makes it possible to call OpenCV functions from your custom tool implementations without much boilerplate code.

Please see the tool C API for general instructions on how to set you your environment and build tool plugins. Since OpenCV is implemented in C++, it is important to understand compiler requirements.

The VisionAppster SDK comes with the source code of the built-in OpenCV plugin. To get started, have a look at sdk/examples/opencv under your installation. Modify the supplied Makefile to fit your setup. If you installed everything to default locations, building your first OpenCV plugin on Linux can be as simple as this:

cd ~/VisionAppster/sdk/examples/opencv
make install

Windows (MinGW-w64), assuming you copied the SDK to your desktop folder:

cd %USERPROFILE%\Desktop\sdk\examples\opencv
mingw32-make
:: This may require admin rights.
mingw32-make install

Windows (MinGW-w64 + MSYS2), assuming you copied the SDK to your desktop folder:

cd /c/Users/$USERNAME/Desktop/sdk/examples/opencv
make
# This may require admin rights.
make install

If you use OpenCV to implement your own tools, you are strongly encouraged to use the version that comes with the platform SDK. While it would be technically possible to use multiple different versions of shared libraries in a single process, it is not currently supported. If you absolutely need a different version, you may link the required parts of OpenCV statically to your tool plugin.

Example

The following example shows a complete implementation of a plugin and a tool that wraps the cv::matchTemplate function. This is how the built-in MatchTemplate tool is actually implemented. Please note that even though the interface is C, nothing prevents you from implementing the functions in C++. This however requires that you use the same C++ compiler the platform was built with.

The SDK that comes with the VisionAppster platform contains this and many other OpenCV functions as examples. Please see the sdk/examples/opencv directory in your installation.

VA_IMPLEMENT_PLUGIN("com.visionappster.opencv", "1.0.0")

VA_REGISTER_TOOLS(
( IDENTIFIER     (MatchTemplate),
  NAME           ("Template Matching"),
  TAG            ("matching"),
  PROCESS        (MatchTemplate_process),
  REQUIRED_INPUT (va_image, image),
  OPTIONAL_INPUT (va_image, templ, va_image_alloc_empty()),
  OPTIONAL_INPUT (int32_t, method, 0),
  STATIC_META    (va_array, method_choices, va_array_from_json(
                    "["
                    "{\"name\":\"TmSqdiff\",\"value\":0},"
                    "{\"name\":\"TmSqdiffNormed\",\"value\":1},"
                    "{\"name\":\"TmCcorr\",\"value\":2},"
                    "{\"name\":\"TmCcorrNormed\",\"value\":3},"
                    "{\"name\":\"TmCcoeff\",\"value\":4},"
                    "{\"name\":\"TmCcoeffNormed\",\"value\":5}"
                    "]")),
  OUTPUT         (va_fmatrix, result),
  OUTPUT         (va_image, map)
))

int opencv_match_template(const va_image* imgIn,
                          const va_image* templateIn,
                          int32_t method,
                          va_fmatrix* resultOut,
                          va_image* mapOut)
{
  try
    {
      // This is the core. The rest of the code just ensures the validity of
      // input and output arguments.
      cv::matchTemplate(OpenCvMat(imgIn),
                        OpenCvMat(templateIn),
                        OpenCvMat(resultOut),
                        method);

      double dMin = 0, dMax = 0;
      cv::minMaxLoc(OpenCvMat(resultOut), &dMin, &dMax);
      // Scale the map values to range 0...255
      if (dMax > dMin && mapOut && mapOut->type == va_gray8_image_type)
        {
          double dScaler = 255.0 / (dMax - dMin);
          for (int iRow = 0; iRow < mapOut->height; ++iRow)
            {
              uchar* pRowOut = static_cast<uchar*>(va_image_row(mapOut, iRow));
              float* pRowIn = va_fmatrix_row(resultOut, iRow);
              for (int iCol = 0; iCol < mapOut->width; ++iCol)
                pRowOut[iCol] = round((pRowIn[iCol] - dMin) * dScaler);
            }
        }
    }
  catch (cv::Exception&)
    {
      // TODO: log error
      return VA_ERR_INVALID_PARAMETER;
    }
  return VA_SUCCESS;
}

int32_t MatchTemplate_process(void* instance, void* arguments)
{
  struct MatchTemplate_args* args = (struct MatchTemplate_args*)arguments;
  (void)instance; // unused

  if (args->in.image->type != args->in.templ->type)
    return VA_ERR_INVALID_PARAMETER;

  // Calculate dimensions of the result matrix and map image.
  int iWidth = args->in.image->width - args->in.templ->width + 1;
  int iHeight = args->in.image->height - args->in.templ->height + 1;
  if (iWidth <= 0 || iHeight <= 0)
    return VA_ERR_INVALID_PARAMETER;

  args->out.result = va_fmatrix_alloc(iHeight, iWidth);
  args->out.map = va_image_alloc(va_gray8_image_type, width, height);
  // Copy calibration data from source image
  va_image_copy_fields(args->in.image, args->out.map);

  return opencv_match_template(args->in.image, args->in.templ, args->in.method,
                               args->out.result, args->out.map);
}