Positioning with QR codes🔗

Summary🔗

QR codes can be used to determine the position of the camera with respect to the plane where the QR code lies. The origin of the resulting coordinate system will be at the upper left corner of the QR code.

We assume that there is only one QR code in the image and the length of the edge of the code is known. The actual information content of the code is not relevant in the positioning process. The positioning is based solely on the relative positions of the corner patterns of the code.

Downloads🔗

Detailed description🔗

Let us walk through the schema below top-down.

Complete processing graph.

Complete processing graph.🔗

Before delving into details, let’s see where we are aiming at. The upper picture below shows the input image containing a QR code and a ruler. The picture is taken from an unknown angle. The lower picture is a perspective corrected version of the input image. The content of the QR code is also shown, as well as Euler angles of rotation. More on them later.

The target is to straighten an image taken from an unknown angle.

The target is to straighten an image taken from an unknown angle.🔗

  • The image originates from the virtual camera configured in Image Source. Position parameters are disabled by default. The focal length in terms of pixel units on the sensor must be known as accurately as possible. Focal length can be determined quickly (but not very accurately) using the method explained in the cookbook entry on Manual Calibration. The most accurate way to determine the focal length is to use a proper calibration plate and the calibration tool in te Camera Configurator. In this example, the focal length is 2000 pixels. Likewise, the most accurate method for positioning (i.e. perpective correction) is to use the positioning tool in Camera Configurator. A benefit of using a QR code for positioning is simplicity because the calibration plate is not needed. Any QR code with a known edge length will do.

  • Several tools downstream need to know the threshold between the dark and bright modules in the QR code. An estimate of such a threshold is given by Find Threshold. Input for the threshold finder is a histogram calculated over the whole image.

The focal length of the camera must be known. If unsure, use trial and error.

The focal length of the camera must be known. If unsure, use trial and error.🔗

  • The image is passed to Find Qr Key Points. The tool has to know the length of the edge of the QR code. In this case, Edge Length is 32 mm. Threshold sets the limit between dark and bright modules in the code and Invert = true means that the code has black modules on a light background. The tool detects 13 key points in the QR code and outputs the pixel coordinates and the corresponding world coordinates. This is demonstrated in the picture below by connecting the points with yellow lines.

Detected key points laid over the input image

Detected key points laid over the input image🔗

  • The two point sets are passed to Homography. The tool calculates a matrix that maps the coordinates of the detected key points to the known reference coordinates. This frame describes the position of the camera with respect to the world plane.

  • The frame is passed to Replace Coordinate Frame together with the image. The tool attaches the frame to the image without modifying the pixels.

  • The frame is also passed to Frame to Angles which calculates Euler angles in degrees. They describe 3D rotation of the camera with respect to the QR code.

The homography between detected and known key point coordinates is used as a new coordinate frame.

The homography between detected and known key point coordinates is used as a new coordinate frame.🔗

  • Unrelated to the positioning procedure, the QR code is also located in Matrix Code Detection. Based on the position information passed in Frame and Size matrices, the contents are read by Matrix Code Reading. If you are not interested in the information content of the QR code, these tools can be omitted. Notice the that information about the size of the edge (32 mm in this case) could be written to the QR code. It could be read with Matrix Code Reading tool and fed back to QR Key Point Detection tool with a simple script that converts the Code output string to a number. This way the size of the QR code would not need to be hardcoded in the application.

The contents of the QR code can optionally be readed simultaneously.

The contents of the QR code can optionally be readed simultaneously.🔗

Finally, let’s verify that the image is correctly positioned. For this end, we attach a Caliper to the corrected image coming from Set Coordinate Frame and set the start and end points of the measurement line to left and right sides of the perspective-corrected image. The width of the measurement line is set as 0.5 mm.

Caliper calculates plenty of information about the gaps. We use only the distances of the begin and end points of the gaps measured from the beginning of the measurement line. This information is comes from Gap Start Length and Gap End Length outputs. The unit of the length is now in millimeters because we told QR Key Point Detection above how long the edge of the QR code is.

The start and end length are passed to a simple script which calculates the difference between the beginning of the first gap and the end of the last gap. In this case the measurement error is about 0.1 mm, meaning 0.3% relative error. There are roughly 10 pixels per millimeter in the input image so 0.1 mm corresponds to one pixel. The caliper measures the width of the edge of the code correctly as 32mm.

The width of the QR code can now be measured with a relative accuracy of 0.3%.

The width of the QR code can now be measured with a relative accuracy of 0.3%.🔗