# Image

## Classes

struct kuvio_image
struct kuvio_image_list

## Macros

#define KUVIO_IMAGE_ROW_ALIGN 16
#define KUVIO_IMAGE_DATA_ALIGN KUVIO_IMAGE_ROW_ALIGN
#define KUVIO_IMAGE_HEADER_SIZE((sizeof(kuvio_image)+ KUVIO_IMAGE_DATA_ALIGN - 1)& -KUVIO_IMAGE_DATA_ALIGN)

## Typedefs

typedef struct kuvio_image kuvio_image

## Enumerations

enum kuvio_image_type
enum kuvio_overlapbehavior

## Functions

kuvio_image* kuvio_image_alloc_empty()
kuvio_image* kuvio_image_alloc(kuvio_image_type type, int width, int height)
kuvio_image* kuvio_image_alloc_stride(kuvio_image_type type, int width, int height, size_t stride)
kuvio_image* kuvio_image_clone(const kuvio_image* source)
void kuvio_image_copy_data(const void* source, size_t sourceStride, kuvio_image* target)
kuvio_image* kuvio_image_calloc(kuvio_image_type type, int width, int height)
kuvio_image* kuvio_image_alloc_copy(const kuvio_image* source)
kuvio_image* kuvio_image_calloc_copy(const kuvio_image* source)
kuvio_image* kuvio_image_alloc_typed_copy(const kuvio_image* source, kuvio_image_type type)
kuvio_image* kuvio_image_calloc_typed_copy(const kuvio_image* source, kuvio_image_type type)
kuvio_image* kuvio_image_alloc_more(kuvio_image* source, int rows)
void kuvio_image_clear(kuvio_image* source)
void kuvio_image_free(kuvio_image* img)
void kuvio_image_destroy(kuvio_image* img)
const char* kuvio_image_type_name(kuvio_image_type type)
int kuvio_image_set_world_frame(kuvio_image* img, const kuvio_coordinate_frame* frame)
void kuvio_image_set_default_calibration(kuvio_image* img)
size_t kuvio_image_data_size(const kuvio_image* img)
void* kuvio_image_row(const kuvio_image* img, int y)
void kuvio_image_copy_fields(const kuvio_image* source, kuvio_image* result)
int kuvio_image_virtual_view(const kuvio_image* input, const kuvio_dmatrix* transformation, double referenceDistance, int interpolation, kuvio_image* output)
int kuvio_image_align(const kuvio_image* input, const kuvio_dmatrix* transformation, double referenceDistance, int interpolation, kuvio_image* output)
int kuvio_image_virtual_view_frame(const kuvio_image* input, const kuvio_coordinate_frame* frame, double referenceDistance, int interpolation, kuvio_image* output)
int kuvio_image_align_to_frame(const kuvio_image* input, const kuvio_coordinate_frame* frame, double referenceDistance, int interpolation, kuvio_image* output)
int kuvio_image_scale(const kuvio_image* source, kuvio_image* result)
int kuvio_image_rotate(const kuvio_image* source, double angleDeg, int interpolation, int backgroundColor, kuvio_image* result)
kuvio_image* kuvio_image_create_virtual_view(const kuvio_image* img, kuvio_dmatrix* transformation, double* referenceDistance)
kuvio_image* kuvio_image_create_virtual_view_frame(const kuvio_image* img, kuvio_coordinate_frame* frame, double* referenceDistance)
int kuvio_image_project(kuvio_image_list inputImages, kuvio_overlapbehavior overlapBehavior, kuvio_image* output)
int kuvio_image_to_rgb(const kuvio_image* input, kuvio_image* output)
void kuvio_round_coordinates(const kuvio_image* img, const kuvio_real* realCoord, int* intCoord)
int kuvio_image_to_gray(const kuvio_image* input, kuvio_image* output)

## Macro Definition Documentation

#define KUVIO_IMAGE_HEADER_SIZE((sizeof(kuvio_image)+ KUVIO_IMAGE_DATA_ALIGN - 1)& -KUVIO_IMAGE_DATA_ALIGN)

The number of bytes reserved by the image header. The size of the header is rounded up to a multiple of KUVIO_IMAGE_DATA_ALIGN bytes. The image is stored contiguously in memory if img + KUVIO_IMAGE_HEADER_SIZE == img->data.

#define KUVIO_IMAGE_ROW_ALIGN 16

Image rows must be aligned to be able to process them efficiently. This is a hard requirement on FPGA and significantly increases performance when using SSE2 instructions on PC processors. This value is the minimum required alignment.

## Typedef Documentation

typedef struct kuvio_image kuvio_image

An image.

The image structure describes the format of the image and contains a pointer to the start of image data. Images are always stored in row-major order: the data pointer points to the upper left pixel, data + 1 to the pixel to the right of it and data + stride to the beginning of the row below the first one.

The image structure always contains calibration data and a coordinate frame that can be used to map the image to the physical world. This makes it possible to always make measurents in world units (e.g. millimeters) and to easily relate images to each other without using side channels to pass location information. For example, the exact location of a subimage within the original is always known because both images know their location with respect to the world.

## Enumeration Type Documentation

Supported image types.

Enumerator
kuvio_invalid_image_type

Invalid image type.

kuvio_gray8_image_type

Gray levels with 8 bits per pixel (unsigned char).

kuvio_gray16_image_type

Gray levels with up to 16 bits per pixel (signed short).

kuvio_gray32_image_type

Gray levels with up to 32 bits per pixel (signed int).

kuvio_rgb32_image_type

RGB, 8 bits per color channel. Each pixel is represented as an unsigned int in the native byte order of the machine. On little-endian hardware, kuvio_rgb32_image_type. and kuvio_bgrx32_image_type are equivalent. The fourth color channel can be used as an alpha channel.

kuvio_bgrx32_image_type

32-bit RGB. This is a byte-based format in which the blue color channel comes first. On a little-endian machine, this format corresponds to kuvio_rgb32_image_type.

+---+---+---+---+---+---+---+---+
| B | G | R | ? | B | G | R | ? | ...
+---+---+---+---+---+---+---+---+
kuvio_xrgb32_image_type

32-bit RGB. This is a byte-based format in which the red color channel comes first. On a big-endian machine, this format corresponds to kuvio_rgb32_image_type.

+---+---+---+---+---+---+---+---+
| ? | R | G | B | ? | R | G | B | ...
+---+---+---+---+---+---+---+---+
kuvio_bgr24_image_type

Packed 24-bit RGB.

+---+---+---+---+---+---+
| B | G | R | B | G | R | ...
+---+---+---+---+---+---+
kuvio_rgb24_image_type

Packed 24-bit RGB.

+---+---+---+---+---+---+
| R | G | B | R | G | B | ...
+---+---+---+---+---+---+
kuvio_rggb8_image_type

Bayer-encoded 8-bit RGB (RG). The names of the Bayer encoding schemes encode the arrangement of color filters in a 2-by-2 neighborhood. The RGGB pattern corresponds to the following arrangement:

+---+---+---+---+
| R | G | R | G | ...
+---+---+---+---+
| G | B | G | B | ...
+---+---+---+---+
kuvio_bggr8_image_type

Bayer-encoded 8-bit RGB (BG).

+---+---+---+---+
| B | G | B | G | ...
+---+---+---+---+
| G | R | G | R | ...
+---+---+---+---+
kuvio_gbrg8_image_type

Bayer-encoded 8-bit RGB (GB).

+---+---+---+---+
| G | B | G | B | ...
+---+---+---+---+
| R | G | R | G | ...
+---+---+---+---+
kuvio_grbg8_image_type

Bayer-encoded 8-bit RGB (GR).

+---+---+---+---+
| G | R | G | R | ...
+---+---+---+---+
| B | G | B | G | ...
+---+---+---+---+
kuvio_yuv444_image_type

YUV 4:4:4. All three color components are provided for each pixel in YUV order.

+---+---+---+---+---+---+
| Y | U | V | Y | U | V | ...
+---+---+---+---+---+---+
kuvio_uyv444_image_type

YUV 4:4:4. All three color components are provided for each pixel in UYV order.

+---+---+---+---+---+---+
| U | Y | V | U | Y | V | ...
+---+---+---+---+---+---+
kuvio_yvu444_image_type

YUV 4:4:4. All three color components are provided for each pixel in YVU order.

+---+---+---+---+---+---+
| Y | V | U | Y | V | U | ...
+---+---+---+---+---+---+
kuvio_uyvy422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| U | Y | V | Y | U | Y | V | Y | ...
+---+---+---+---+---+---+---+---+
kuvio_yuyv422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| Y | U | Y | V | Y | U | Y | V | ...
+---+---+---+---+---+---+---+---+
kuvio_vyuy422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| V | Y | U | Y | V | Y | U | Y | ...
+---+---+---+---+---+---+---+---+
kuvio_yvyu422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| Y | V | Y | U | Y | V | Y | U | ...
+---+---+---+---+---+---+---+---+
kuvio_uyvyyy411_image_type

YUV 4:1:1. Y is provided for each pixel, U and V shared between four neighbors.

+---+---+---+---+---+---+---+---+---+---+---+---+
| U | Y | V | Y | Y | Y | U | Y | V | Y | Y | Y | ...
+---+---+---+---+---+---+---+---+---+---+---+---+
kuvio_uyyvyy411_image_type

YUV 4:1:1. Y is provided for each pixel, U and V shared between four neighbors.

+---+---+---+---+---+---+---+---+---+---+---+---+
| U | Y | Y | V | Y | Y | U | Y | Y | V | Y | Y | ...
+---+---+---+---+---+---+---+---+---+---+---+---+
kuvio_yuv420p_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/4 consecutive U-samples, followed by N/4 consequtive V-samples. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| U1a | U2b | U3c | U4d |
+-----+-----+-----+-----+
| V1a | V2b | V3c | V4d |
+-----+-----+-----+-----+
kuvio_yvu420p_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/4 consecutive V-samples, followed by N/4 consequtive U-samples. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| V1a | V2b | V3c | V4d |
+-----+-----+-----+-----+
| U1a | U2b | U3c | U4d |
+-----+-----+-----+-----+
kuvio_yuv420sp_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/2 interlaced U+V-pairs. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| U1a | V1a | U2b | V2b |
+-----+-----+-----+-----+
| U3c | V3c | U4d | V4d |
+-----+-----+-----+-----+
kuvio_yvu420sp_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/2 interlaced V+U-pairs. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| V1a | U1a | V2b | U2b |
+-----+-----+-----+-----+
| V3c | U3c | V4d | U4d |
+-----+-----+-----+-----+
kuvio_rgbx32_image_type

32-bit RGB. This is a byte-based format in which the red color channel comes first and padding last.

+---+---+---+---+---+---+---+---+
| R | G | B | ? | R | G | B | ? | ...
+---+---+---+---+---+---+---+---+
kuvio_xbgr32_image_type

32-bit RGB. This is a byte-based format in which the red color channel comes last and padding first.

+---+---+---+---+---+---+---+---+
| ? | B | G | R | ? | B | G | R | ...
+---+---+---+---+---+---+---+---+
kuvio_compressed_image_type

Compressed data, either gray levels or colors. Compressed images are special in that their size is not constant. It is not possible to obtain a pointer to the beginning of any of the scanlines, and it is not therefore safe to access pixels unless the image is decompressed first. The stride of a compressed image tells the length of the compressed data in bytes.

Strategies for handling overlapping parts of images that are projected on each other.

Enumerator
kuvio_replace_overlap

Projected image pixels replace old data. Last value remains in effect.

kuvio_average_overlap

Average overlapping pixels. Note that this mode is much slower than 'ReplaceOverlap'.

## Function Documentation

int kuvio_image_align(const kuvio_image* input, const kuvio_dmatrix* transformation, double referenceDistance, int interpolation, kuvio_image* output )
int kuvio_image_align_to_frame(const kuvio_image* input, const kuvio_coordinate_frame* frame, double referenceDistance, int interpolation, kuvio_image* output )
kuvio_image* kuvio_image_alloc(kuvio_image_type type, int width, int height )

Allocates an uninitialized image array. If memory allocation fails or if type is kuvio_compressed_image_type, returns 0.

kuvio_image* kuvio_image_alloc_copy(const kuvio_image* source)

Allocates a new image that is the same size and type as source and has the same world frame, calibration data and channel value range. The contents won't be initialized. If memory allocation fails or if type is kuvio_compressed_image_type, returns 0.

kuvio_image* kuvio_image_alloc_empty()

Allocates space for a kuvio_image from the shared heap (shm_alloc) and initializes the structure. The returned image contains no data.

kuvio_image* kuvio_image_alloc_more(kuvio_image* source, int rows )

Appends rows uninitialized rows to source and returns either source or a pointer to a newly allocated image with the same contents as source. If memory allocation fails or if source is shared or compressed, returns 0 and leaves source untouched.

kuvio_image* kuvio_image_alloc_stride(kuvio_image_type type, int width, int height, size_t stride )

Allocates an uninitialized image array with the given stride. If stride is less than the minimum number of bytes required for each image row of the given width and type, stride will be automatically adjusted up.

This function can be used to allocate space for a compressed image, provided that you know the size of compressed data (stride) in advance.

kuvio_image* kuvio_image_alloc_typed_copy(const kuvio_image* source, kuvio_image_type type )

Allocates a new image that is the same size as source and has the same world frame, calibration data and channel value range. The type of the new image may be different from that of source. The contents will be uninitialized. If memory allocation fails or if type is kuvio_compressed_image_type, returns 0.

kuvio_image* kuvio_image_calloc(kuvio_image_type type, int width, int height )

Allocates an image array whose every pixel is initially zero. If memory allocation fails or if type is kuvio_compressed_image_type, returns 0.

kuvio_image* kuvio_image_calloc_copy(const kuvio_image* source)

Allocates a new image that is the same size and type as source and has the same world frame, calibration data and channel value range. The contents will be initialized to zeros. If memory allocation fails or if type is kuvio_compressed_image_type, returns 0.

kuvio_image* kuvio_image_calloc_typed_copy(const kuvio_image* source, kuvio_image_type type )

Allocates a new image that is the same size as source and has the same world frame, calibration data and channel value range. The type of the new image may be different from that of source. The contents will be set to zero. If memory allocation fails or if type is kuvio_compressed_image_type, returns 0.

void kuvio_image_clear(kuvio_image* source)

Sets all pixels in image to zeros.

kuvio_image* kuvio_image_clone(const kuvio_image* source)

Creates a deep copy of source. Note that the rows of the result will be packed to the default alignment, which means that the stride of the copy may be different from that of source. Returns the clone or 0 if memory allocation fails.

void kuvio_image_copy_data(const void* source, size_t sourceStride, kuvio_image* target )

Copies raw data from a buffer pointed to by source to the data buffer of target.

Parameters
 source A pointer to the beginning of the pixel buffer to copy. The buffer must be valid for at least (target->rows - 1) * sourceStride + target->columns * kuvio_image_pixel_bits(target->type) / 8 bytes. sourceStride The number of bytes between row starts in source. sourceStride must be at least target->columns * kuvio_image_pixel_bits(target->type) / 8. target Target image. Data from source will be copied to the data buffer of this image.
void kuvio_image_copy_fields(const kuvio_image* source, kuvio_image* result )

Copies channel min/max values, world and camera coordinate frames and calibration data from source to result.

kuvio_image* kuvio_image_create_virtual_view(const kuvio_image* img, kuvio_dmatrix* transformation, double* referenceDistance )

Allocates an empty image which large enough to hold the input image img transformed to the new coordinate frame. The number of pixels in the output image is identical to the number of pixels in the input image, even though the aspect ratio can be different. This function can be used to allocate an empty image which is passed to kuvio_image_virtual_view.

Parameters
 img The input image. transformation A 4-by-4 homogeneous transformation matrix that specifies the coordinate frame of the area. The translation part of the matrix is moved so that the entire transformed image will fit in the returned image when passed to kuvio_image_virtual_view. referenceDistance Pointer to the distance between the virtual camera and the center of the area, in world coordinates. The distance will be scaled such that the entire transformed image will fit in the returned image when passed to kuvio_image_virtual_view.
Returns
An empty image which large enough to hold the input image img transformed to the new coordinate frame, or null pointer if the operation failed.
kuvio_image* kuvio_image_create_virtual_view_frame(const kuvio_image* img, kuvio_coordinate_frame* frame, double* referenceDistance )

Allocates an image according to a new coordinate frame. This function works the same as kuvio_image_create_virtual_view(), but takes the target coordinate frame input parameter as a kuvio_coordinate_frame.

size_t kuvio_image_data_size(const kuvio_image* img)

Returns the number of bytes needed to store all of img's pixels. This is the difference between the address of the last pixel and that of the first one plus one. The actual value depends on pixel type and image stride, and may be a lot bigger than the number of pixels in the image, especially if the image is a sub-image reference. For compressed images, this function returns the size of the compressed data, i.e. img->stride.

void kuvio_image_destroy(kuvio_image* img)

Releases the memory allocated by img and img->data using shm_free(). This function can be used as a custom destructor if you allocated an external data buffer for image data using shm_malloc().

void kuvio_image_free(kuvio_image* img)

Releases the memory allocated by img. If there is a custom destructor function, it will be used. If not, shm_free() will be used to release img. Note that if you made img->data point to an external buffer, it will not be deallocated.

int kuvio_image_project(kuvio_image_list inputImages, kuvio_overlapbehavior overlapBehavior, kuvio_image* output )

Projects multiple images to a single output image.

Parameters
 inputImages A list of images to join. Must contain 1-8 images. overlapBehavior Specifies the strategy of handling overlapping parts. output A preallocated output image to which the inputs will be projected. The output will contain all input images as if they were seen through a camera whose location and calibration data is stored in the output image.
int kuvio_image_rotate(const kuvio_image* source, double angleDeg, int interpolation, int backgroundColor, kuvio_image* result )

Rotates the given source image angleDeg degrees clockwise to the given result image. The rotation is done about the pricipal points. The principal point of the source image is projected to the principal point of the result image regardless of its location. This function only supports non-encoded gray and color images.

void* kuvio_image_row(const kuvio_image* img, int y )

Returns a pointer to the beginning of the *y*th row. It is not safe to call this function on an empty or compressed image.

int kuvio_image_scale(const kuvio_image* source, kuvio_image* result )

Scales the given source image to the given result image, which also specifies the target size. Bilinear filtering is used for smooth scaling result. Note that if source is empty, result won't be modified. This function only supports non-encoded gray and color images.

void kuvio_image_set_default_calibration(kuvio_image* img)

Resets the calibration factors in img to default values.

int kuvio_image_set_world_frame(kuvio_image* img, const kuvio_coordinate_frame* frame )

Sets img->world_frame to frame and img->camera_frame to its inverse. Returns 1 on success and 0 on failure.

int kuvio_image_to_gray(const kuvio_image* input, kuvio_image* output )

Converts any encoded image type to 8-bit gray. The type of the output image must be kuvio_gray8_image_type and its size must match that of the input image.

If the size of the output does not match that of the input, the type of the output is not kuvio_gray8_image_type or the type of the input is kuvio_compressed_image_type, returns KUVIO_ERR_INVALID_PARAMETER. Otherwise returns KUVIO_SUCCESS.

int kuvio_image_to_rgb(const kuvio_image* input, kuvio_image* output )

Converts input image into standard 32-bit RGB image which can be used by analysis tools. The type of the output image must be kuvio_rgb32_image_type and its size must match that of the input image.

If the size of the output does not match that of the input, the type of the output is not kuvio_rgb32_image_type or the type of the input is kuvio_compressed_image_type, returns KUVIO_ERR_INVALID_PARAMETER. Otherwise returns KUVIO_SUCCESS.

const char* kuvio_image_type_name(kuvio_image_type type)

Returns a human-readable name for the given type.

int kuvio_image_virtual_view(const kuvio_image* input, const kuvio_dmatrix* transformation, double referenceDistance, int interpolation, kuvio_image* output )

Creates a view of the world out of an image and a virtual window defined by a coordinate frame and a size. This function can simultaneously perform arbitrary 3D affine transformations, perspective correction, scaling and cropping.

The function is given a transformation matrix and an output image the specifies the size of a rectangular area in the physical world (not in the input image). The resulting image is a picture of the area as if a virtual camera were placed right on top of it at referenceDistance.

The function can take any image as input and does not expect that the requested piece of the world is actually visible in it. If the input image does not contain the necessary data, portions of the output image (or even the whole image) may be left black.

To rotate an image, one only needs to create a rotated coordinate frame. Images can be scaled by simply altering referenceDistance.

Parameters
 input The input image. Must be a non-encoded gray or color image. transformation A 4-by-4 homogeneous transformation matrix that specifies the coordinate frame of the area. referenceDistance The distance between the virtual camera and the center of the area, in world coordinates. interpolation If 1, improve accuracy by interpolating point samples between pixels. If 0, round pixel coordinates to the nearest integer. output A preallocated output image. The focal lengths and the size of the output image determine the world-to-pixel scaling ratio. The function calculates and stores a new world reference frame to the output image so that the resulting image can be related to the world coordinate system.
Returns
KUVIO_SUCCESS on success, KUVIO_ERR_OUT_OF_MEMORY if there is not enough memory, KUVIO_ERR_NOT_INVERTIBLE if the transformation matrix cannot be inverted and KUVIO_ERR_INVALID_PARAMETER if input is invalid. The non-invertible transformation case is treated as if none of the requested area was visible in the input image: the whole output image will be set to zeros.
int kuvio_image_virtual_view_frame(const kuvio_image* input, const kuvio_coordinate_frame* frame, double referenceDistance, int interpolation, kuvio_image* output )

Creates a view of the world out of an image and a virtual window defined by a coordinate frame and a size. This function works the same as kuvio_image_virtual_view(), but takes the coordinate frame as a kuvio_coordinate_frame. This function only supports non-encoded gray and color images.

void kuvio_round_coordinates(const kuvio_image* img, const kuvio_real* realCoord, int* intCoord )

Rounds pixel coordinates to the nearest value which does not confuse the color coding scheme. For gray types and ordinary 32-bit RGB type this means normal rounding. For Bayer and YUV formats the rounded coordinates are divisible by either 2 or 4.

Parameters
 img input image. realCoord Pointer to real-valued coordinates (x, y). intCoord Pointer to rounded integer-valued coordinates (x, y)