# Image

## Classes

struct va_image
struct va_image_list

## Macros

#define VA_IMAGE_ROW_ALIGN 16
#define VA_IMAGE_DATA_ALIGN VA_IMAGE_ROW_ALIGN
#define VA_IMAGE_HEADER_SIZE((sizeof(va_image)+ VA_IMAGE_DATA_ALIGN - 1)& -VA_IMAGE_DATA_ALIGN)

## Typedefs

typedef struct va_image va_image

## Enumerations

enum va_image_type
enum va_overlap_behavior

## Functions

va_image* va_image_alloc_empty()
va_image* va_image_alloc(va_image_type type, int width, int height)
va_image* va_image_alloc_stride(va_image_type type, int width, int height, size_t stride)
va_image* va_image_clone(const va_image* source)
void va_image_copy_data(const void* source, size_t sourceStride, va_image* target)
va_image* va_image_calloc(va_image_type type, int width, int height)
va_image* va_image_alloc_copy(const va_image* source)
va_image* va_image_calloc_copy(const va_image* source)
va_image* va_image_alloc_typed_copy(const va_image* source, va_image_type type)
va_image* va_image_calloc_typed_copy(const va_image* source, va_image_type type)
va_image* va_image_alloc_more(va_image* source, int rows)
void va_image_clear(va_image* source)
void va_image_free(va_image* img)
void va_image_destroy(va_image* img)
const char* va_image_type_name(va_image_type type)
int va_image_set_world_frame(va_image* img, const va_coordinate_frame* frame)
void va_image_set_default_calibration(va_image* img)
size_t va_image_data_size(const va_image* img)
void* va_image_row(const va_image* img, int y)
void va_image_copy_fields(const va_image* source, va_image* result)
int va_image_virtual_view(const va_image* input, const va_dmatrix* transformation, double referenceDistance, int interpolation, va_image* output)
int va_image_align(const va_image* input, const va_dmatrix* transformation, double referenceDistance, int interpolation, va_image* output)
int va_image_virtual_view_frame(const va_image* input, const va_coordinate_frame* frame, double referenceDistance, int interpolation, va_image* output)
int va_image_align_to_frame(const va_image* input, const va_coordinate_frame* frame, double referenceDistance, int interpolation, va_image* output)
int va_image_scale(const va_image* source, va_image* result)
int va_image_rotate(const va_image* source, double angleDeg, int interpolation, int backgroundColor, va_image* result)
va_image* va_image_create_virtual_view(const va_image* img, va_dmatrix* transformation, double* referenceDistance)
va_image* va_image_create_virtual_view_frame(const va_image* img, va_coordinate_frame* frame, double* referenceDistance)
int va_image_project(va_image_list inputImages, va_overlap_behavior overlapBehavior, va_image* output)
int va_image_to_rgb(const va_image* input, va_image* output)
void va_image_round(const va_image* img, const va_real* realCoord, int* intCoord)
int va_image_to_gray(const va_image* input, va_image* output)
va_image* va_image_subimage(va_image* img, int x, int y, int width, int height)
int va_image_is_valid_subimage(const va_image* img, int x, int y, int width, int height)

## Macro Definition Documentation

#define VA_IMAGE_HEADER_SIZE((sizeof(va_image)+ VA_IMAGE_DATA_ALIGN - 1)& -VA_IMAGE_DATA_ALIGN)

The number of bytes reserved by the image header. The size of the header is rounded up to a multiple of VA_IMAGE_DATA_ALIGN bytes. The image is stored contiguously in memory if img + VA_IMAGE_HEADER_SIZE == img->data.

#define VA_IMAGE_ROW_ALIGN 16

Image rows must be aligned to be able to process them efficiently. This is a hard requirement on FPGA and significantly increases performance when using SIMD instructions on PC processors. This value is the minimum required alignment.

## Typedef Documentation

typedef struct va_image va_image

An image.

The image structure describes the format of the image and contains a pointer to the start of image data. Images are always stored in row-major order: the data pointer points to the upper left pixel, data + 1 to the pixel to the right of it and data + stride to the beginning of the row below the first one.

The image structure always contains calibration data and a coordinate frame that can be used to map the image to the physical world. This makes it possible to always make measurements in world units (e.g. millimeters) and to easily relate images to each other without using side channels to pass location information. For example, the exact location of a subimage within the original is always known because both images know their location with respect to the world.

## Enumeration Type Documentation

Supported image types.

Enumerator
va_invalid_image_type

Invalid image type.

va_gray8_image_type

Gray levels with 8 bits per pixel (unsigned char).

va_gray16_image_type

Gray levels with up to 16 bits per pixel (signed short).

va_gray32_image_type

Gray levels with up to 32 bits per pixel (signed int).

va_rgb32_image_type

RGB, 8 bits per color channel. Each pixel is represented as an unsigned int in the native byte order of the machine. On little-endian hardware, va_rgb32_image_type. and va_bgrx32_image_type are equivalent. The fourth color channel can be used as an alpha channel.

va_bgrx32_image_type

32-bit RGB. This is a byte-based format in which the blue color channel comes first. On a little-endian machine, this format corresponds to va_rgb32_image_type.

+---+---+---+---+---+---+---+---+
| B | G | R | ? | B | G | R | ? | ...
+---+---+---+---+---+---+---+---+
va_xrgb32_image_type

32-bit RGB. This is a byte-based format in which the red color channel comes first. On a big-endian machine, this format corresponds to va_rgb32_image_type.

+---+---+---+---+---+---+---+---+
| ? | R | G | B | ? | R | G | B | ...
+---+---+---+---+---+---+---+---+
va_bgr24_image_type

Packed 24-bit RGB.

+---+---+---+---+---+---+
| B | G | R | B | G | R | ...
+---+---+---+---+---+---+
va_rgb24_image_type

Packed 24-bit RGB.

+---+---+---+---+---+---+
| R | G | B | R | G | B | ...
+---+---+---+---+---+---+
va_rggb8_image_type

Bayer-encoded 8-bit RGB (RG). The names of the Bayer encoding schemes encode the arrangement of color filters in a 2-by-2 neighborhood. The RGGB pattern corresponds to the following arrangement:

+---+---+---+---+
| R | G | R | G | ...
+---+---+---+---+
| G | B | G | B | ...
+---+---+---+---+
va_bggr8_image_type

Bayer-encoded 8-bit RGB (BG).

+---+---+---+---+
| B | G | B | G | ...
+---+---+---+---+
| G | R | G | R | ...
+---+---+---+---+
va_gbrg8_image_type

Bayer-encoded 8-bit RGB (GB).

+---+---+---+---+
| G | B | G | B | ...
+---+---+---+---+
| R | G | R | G | ...
+---+---+---+---+
va_grbg8_image_type

Bayer-encoded 8-bit RGB (GR).

+---+---+---+---+
| G | R | G | R | ...
+---+---+---+---+
| B | G | B | G | ...
+---+---+---+---+
va_yuv444_image_type

YUV 4:4:4. All three color components are provided for each pixel in YUV order.

+---+---+---+---+---+---+
| Y | U | V | Y | U | V | ...
+---+---+---+---+---+---+
va_uyv444_image_type

YUV 4:4:4. All three color components are provided for each pixel in UYV order.

+---+---+---+---+---+---+
| U | Y | V | U | Y | V | ...
+---+---+---+---+---+---+
va_yvu444_image_type

YUV 4:4:4. All three color components are provided for each pixel in YVU order.

+---+---+---+---+---+---+
| Y | V | U | Y | V | U | ...
+---+---+---+---+---+---+
va_uyvy422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| U | Y | V | Y | U | Y | V | Y | ...
+---+---+---+---+---+---+---+---+
va_yuyv422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| Y | U | Y | V | Y | U | Y | V | ...
+---+---+---+---+---+---+---+---+
va_vyuy422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| V | Y | U | Y | V | Y | U | Y | ...
+---+---+---+---+---+---+---+---+
va_yvyu422_image_type

YUV 4:2:2. Y is provided for each pixel, U and V shared between two neighbors.

+---+---+---+---+---+---+---+---+
| Y | V | Y | U | Y | V | Y | U | ...
+---+---+---+---+---+---+---+---+
va_uyvyyy411_image_type

YUV 4:1:1. Y is provided for each pixel, U and V shared between four neighbors.

+---+---+---+---+---+---+---+---+---+---+---+---+
| U | Y | V | Y | Y | Y | U | Y | V | Y | Y | Y | ...
+---+---+---+---+---+---+---+---+---+---+---+---+
va_uyyvyy411_image_type

YUV 4:1:1. Y is provided for each pixel, U and V shared between four neighbors.

+---+---+---+---+---+---+---+---+---+---+---+---+
| U | Y | Y | V | Y | Y | U | Y | Y | V | Y | Y | ...
+---+---+---+---+---+---+---+---+---+---+---+---+
va_yuv420p_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/4 consecutive U-samples, followed by N/4 consequtive V-samples. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| U1a | U2b | U3c | U4d |
+-----+-----+-----+-----+
| V1a | V2b | V3c | V4d |
+-----+-----+-----+-----+
va_yvu420p_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/4 consecutive V-samples, followed by N/4 consequtive U-samples. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| V1a | V2b | V3c | V4d |
+-----+-----+-----+-----+
| U1a | U2b | U3c | U4d |
+-----+-----+-----+-----+
va_yuv420sp_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/2 interlaced U+V-pairs. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| U1a | V1a | U2b | V2b |
+-----+-----+-----+-----+
| U3c | V3c | U4d | V4d |
+-----+-----+-----+-----+
va_yvu420sp_image_type

YUV 4:2:0. First there are as many Y-samples as there are pixels (N), followed by N/2 interlaced V+U-pairs. The total number of samples is hence 3*N/2.

+-----+-----+-----+-----+
| Y1a | Y2a | Y3b | Y4b |
+-----+-----+-----+-----+
| Y5a | Y6a | Y7b | Y8b |
+-----+-----+-----+-----+
| Y9c | Y10c| Y11d| Y12d|
+-----+-----+-----+-----+
| Y13c| Y14c| Y15d| Y16d|
+-----+-----+-----+-----+
| V1a | U1a | V2b | U2b |
+-----+-----+-----+-----+
| V3c | U3c | V4d | U4d |
+-----+-----+-----+-----+
va_rgbx32_image_type

32-bit RGB. This is a byte-based format in which the red color channel comes first and padding last.

+---+---+---+---+---+---+---+---+
| R | G | B | ? | R | G | B | ? | ...
+---+---+---+---+---+---+---+---+
va_xbgr32_image_type

32-bit RGB. This is a byte-based format in which the red color channel comes last and padding first.

+---+---+---+---+---+---+---+---+
| ? | B | G | R | ? | B | G | R | ...
+---+---+---+---+---+---+---+---+
va_compressed_image_type

Compressed data, either gray levels or colors. Compressed images are special in that their size is not constant. It is not possible to obtain a pointer to the beginning of any of the scanlines, and it is not therefore safe to access pixels unless the image is decompressed first. The stride of a compressed image tells the length of the compressed data in bytes.

Strategies for handling overlapping parts of images that are projected on each other.

Enumerator
va_replace_overlap

Projected image pixels replace old data. Last value remains in effect.

va_average_overlap

Average overlapping pixels. Note that this mode is much slower than 'ReplaceOverlap'.

## Function Documentation

int va_image_align(const va_image* input, const va_dmatrix* transformation, double referenceDistance, int interpolation, va_image* output )
int va_image_align_to_frame(const va_image* input, const va_coordinate_frame* frame, double referenceDistance, int interpolation, va_image* output )
va_image* va_image_alloc(va_image_type type, int width, int height )

Allocates an uninitialized image array. If memory allocation fails or if type is va_compressed_image_type, returns NULL.

va_image* va_image_alloc_copy(const va_image* source)

Allocates a new image that is the same size and type as source and has the same world frame, calibration data and channel value range. The contents won't be initialized. If memory allocation fails or if type is va_compressed_image_type, returns NULL.

va_image* va_image_alloc_empty()

Allocates space for a va_image from the shared heap (shm_alloc) and initializes the structure. The returned image contains no data.

va_image* va_image_alloc_more(va_image* source, int rows )

Appends rows uninitialized rows to source and returns either source or a pointer to a newly allocated image with the same contents as source. If memory allocation fails or if source is compressed, returns NULL and leaves source untouched.

va_image* va_image_alloc_stride(va_image_type type, int width, int height, size_t stride )

Allocates an uninitialized image array with the given stride. If stride is less than the minimum number of bytes required for each image row of the given width and type, stride will be automatically adjusted up.

This function can be used to allocate space for a compressed image, provided that you know the size of compressed data (stride) in advance.

va_image* va_image_alloc_typed_copy(const va_image* source, va_image_type type )

Allocates a new image that is the same size as source and has the same world frame, calibration data and channel value range. The type of the new image may be different from that of source. The contents will be uninitialized. If memory allocation fails or if type is va_compressed_image_type, returns NULL.

va_image* va_image_calloc(va_image_type type, int width, int height )

Allocates an image array whose every pixel is initially zero. If memory allocation fails or if type is va_compressed_image_type, returns NULL.

va_image* va_image_calloc_copy(const va_image* source)

Allocates a new image that is the same size and type as source and has the same world frame, calibration data and channel value range. The contents will be initialized to zeros. If memory allocation fails or if type is va_compressed_image_type, returns NULL.

va_image* va_image_calloc_typed_copy(const va_image* source, va_image_type type )

Allocates a new image that is the same size as source and has the same world frame, calibration data and channel value range. The type of the new image may be different from that of source. The contents will be set to zero. If memory allocation fails or if type is va_compressed_image_type, returns NULL.

void va_image_clear(va_image* source)

Sets all pixels in image to zeros.

va_image* va_image_clone(const va_image* source)

Creates a deep copy of source. Note that the rows of the result will be packed to the default alignment, which means that the stride of the copy may be different from that of source. Returns the clone or NULL if memory allocation fails.

Returns a copy of va_image structure pointed to by source. The returned image will point to the same data as source. If memory allocation fails, returns NULL.

void va_image_copy_data(const void* source, size_t sourceStride, va_image* target )

Copies raw data from a buffer pointed to by source to the data buffer of target.

Parameters
 source A pointer to the beginning of the pixel buffer to copy. The buffer must be valid for at least (target->rows - 1) * sourceStride + target->columns * va_image_pixel_bits(target->type) / 8 bytes. sourceStride The number of bytes between row starts in source. sourceStride must be at least target->columns * va_image_pixel_bits(target->type) / 8. target Target image. Data from source will be copied to the data buffer of this image.
void va_image_copy_fields(const va_image* source, va_image* result )

Copies channel min/max values, world and camera coordinate frames and calibration data from source to result.

va_image* va_image_create_virtual_view(const va_image* img, va_dmatrix* transformation, double* referenceDistance )

Allocates an empty image which large enough to hold the input image img transformed to the new coordinate frame. The number of pixels in the output image is identical to the number of pixels in the input image, even though the aspect ratio can be different. This function can be used to allocate an empty image which is passed to va_image_virtual_view.

Parameters
 img The input image. transformation A 4-by-4 homogeneous transformation matrix that specifies the coordinate frame of the area. The translation part of the matrix is moved so that the entire transformed image will fit in the returned image when passed to va_image_virtual_view. referenceDistance Pointer to the distance between the virtual camera and the center of the area, in world coordinates. The distance will be scaled such that the entire transformed image will fit in the returned image when passed to va_image_virtual_view.
Returns
An empty image which large enough to hold the input image img transformed to the new coordinate frame, or null pointer if the operation failed.
va_image* va_image_create_virtual_view_frame(const va_image* img, va_coordinate_frame* frame, double* referenceDistance )

Allocates an image according to a new coordinate frame. This function works the same as va_image_create_virtual_view(), but takes the target coordinate frame input parameter as a va_coordinate_frame.

size_t va_image_data_size(const va_image* img)

Returns the number of bytes needed to store all of img's pixels. This is the difference between the address of the last pixel and that of the first one plus one. The actual value depends on pixel type and image stride, and may be a lot bigger than the number of pixels in the image, especially if the image is a sub-image reference. For compressed images, this function returns the size of the compressed data, i.e. img->stride.

void va_image_destroy(va_image* img)

Releases the memory allocated by img and img->data using shm_free(). This function can be used as a custom destructor if you allocated an external data buffer for image data using shm_malloc().

void va_image_free(va_image* img)

Releases the memory allocated by img. If there is a custom destructor function, it will be used. If not, shm_free() will be used to release img. Note that if you made img->data point to an external buffer, it will not be deallocated.

int va_image_is_valid_subimage(const va_image* img, int x, int y, int width, int height )

Returns a non-zero value if it is possible to take a shared subimage reference from img at the given location and zero otherwise. A subimage reference can be taken if the source image is non-planar and non-compressed, and the subimage completely fits into it.

int va_image_project(va_image_list inputImages, va_overlap_behavior overlapBehavior, va_image* output )

Projects multiple images to a single output image.

Parameters
 inputImages A list of images to join. Must contain 1-8 images. overlapBehavior Specifies the strategy of handling overlapping parts. output A preallocated output image to which the inputs will be projected. The output will contain all input images as if they were seen through a camera whose location and calibration data is stored in the output image.
int va_image_rotate(const va_image* source, double angleDeg, int interpolation, int backgroundColor, va_image* result )

Rotates the given source image angleDeg degrees clockwise to the given result image. The rotation is done about the pricipal points. The principal point of the source image is projected to the principal point of the result image regardless of its location. This function only supports non-encoded gray and color images.

void va_image_round(const va_image* img, const va_real* realCoord, int* intCoord )

Rounds real-valued pixel coordinates to the nearest value that does not confuse the color coding scheme. For gray types and ordinary 32-bit RGB type this means normal rounding. For Bayer and YUV formats the rounded coordinates are divisible by either 2 or 4.

Parameters
 img input image. realCoord Pointer to real-valued coordinates (x, y). intCoord Pointer to rounded integer-valued coordinates (x, y)
void* va_image_row(const va_image* img, int y )

Returns a pointer to the beginning of the *y*th row. It is not safe to call this function on an empty or compressed image.

int va_image_scale(const va_image* source, va_image* result )

Scales the given source image to the given result image, which also specifies the target size. Bilinear filtering is used for smooth scaling result. Note that if source is empty, result won't be modified. This function only supports non-encoded gray and color images.

void va_image_set_default_calibration(va_image* img)

Resets the calibration factors in img to default values.

int va_image_set_world_frame(va_image* img, const va_coordinate_frame* frame )

Sets img->world_frame to frame and img->camera_frame to its inverse. Returns 1 on success and 0 on failure.

va_image* va_image_subimage(va_image* img, int x, int y, int width, int height )

Creates a new image that references a window within another. Modifications to the returned image will affect img, which must remain valid during the lifetime of the returned image.

The source image must be non-planar and non-compressed, and the subimage must completely fit into it. This function does not check the preconditions, use va_image_is_valid_subimage() for that. If memory allocation fails, this function returns NULL.

int va_image_to_gray(const va_image* input, va_image* output )

Converts any encoded image type to 8-bit gray. The type of the output image must be va_gray8_image_type and its size must match that of the input image.

If the size of the output does not match that of the input, the type of the output is not va_gray8_image_type or the type of the input is va_compressed_image_type, returns VA_ERR_INVALID_PARAMETER. Otherwise returns VA_SUCCESS.

int va_image_to_rgb(const va_image* input, va_image* output )

Converts input image into standard 32-bit RGB image which can be used by analysis tools. The type of the output image must be va_rgb32_image_type and its size must match that of the input image.

If the size of the output does not match that of the input, the type of the output is not va_rgb32_image_type or the type of the input is va_compressed_image_type, returns VA_ERR_INVALID_PARAMETER. Otherwise returns VA_SUCCESS.

const char* va_image_type_name(va_image_type type)

Returns a human-readable name for the given type.

int va_image_virtual_view(const va_image* input, const va_dmatrix* transformation, double referenceDistance, int interpolation, va_image* output )

Creates a view of the world out of an image and a virtual window defined by a coordinate frame and a size. This function can simultaneously perform arbitrary 3D affine transformations, perspective correction, scaling and cropping.

The function is given a transformation matrix and an output image the specifies the size of a rectangular area in the physical world (not in the input image). The resulting image is a picture of the area as if a virtual camera were placed right on top of it at referenceDistance.

The function can take any image as input and does not expect that the requested piece of the world is actually visible in it. If the input image does not contain the necessary data, portions of the output image (or even the whole image) may be left black.

To rotate an image, one only needs to create a rotated coordinate frame. Images can be scaled by simply altering referenceDistance.

Parameters
 input The input image. Must be a non-encoded gray or color image. transformation A 4-by-4 homogeneous transformation matrix that specifies the coordinate frame of the area. referenceDistance The distance between the virtual camera and the center of the area, in world coordinates. interpolation If 1, improve accuracy by interpolating point samples between pixels. If 0, round pixel coordinates to the nearest integer. output A preallocated output image. The focal lengths and the size of the output image determine the world-to-pixel scaling ratio. The function calculates and stores a new world reference frame to the output image so that the resulting image can be related to the world coordinate system.
Returns
VA_SUCCESS on success, VA_ERR_OUT_OF_MEMORY if there is not enough memory, VA_ERR_NOT_INVERTIBLE if the transformation matrix cannot be inverted and VA_ERR_INVALID_PARAMETER if input is invalid. The non-invertible transformation case is treated as if none of the requested area was visible in the input image: the whole output image will be set to zeros.
int va_image_virtual_view_frame(const va_image* input, const va_coordinate_frame* frame, double referenceDistance, int interpolation, va_image* output )

Creates a view of the world out of an image and a virtual window defined by a coordinate frame and a size. This function works the same as va_image_virtual_view(), but takes the coordinate frame as a va_coordinate_frame. This function only supports non-encoded gray and color images.