Run ONNX model

Runs a machine learning model stored in the ONNX format.

Inputs

• modelFile: Path to an ONNX model file.

• executor: The preferred execution back-end for the machine learning model. This parameter can be used to force the tool to stick to one of the available execution back-ends. If the selected executor is not available, a generic CPU implementation will be used. Note that all supported executors are selectable even though the machine on which you use the tool doesn’t have the required hardware and/or libraries.

• deviceIndex: The index of the computation device used by the selected executor. Usually, this is the index of a CUDA device as listed by nvidia-smi. If the chosen device is not available at run time, the first device will be used instead.

Input tensors are defined by the model. They appear in the tool when a model has been loaded.

Outputs

Output tensors are defined by the model. They appear in the tool when a model has been loaded.

enum Executor

Execution back-ends for running the ONNX model.

Values:

enumerator AutoExecutor

Execute the model on the “best” hardware possible.

The preference order is system-dependent, but the fastest GPU will be generally tried first. If GPU acceleration is not available, an optimized CPU implementation will be used. If that fails, a generic CPU back-end will be used.

enumerator CpuExecutor

Run the model on the CPU.

enumerator CudaExecutor

Use CUDA (cuDNN, to be specific), if it is installed.

enumerator MkldnnExecutor

Use MKL-DNN, if it is installed.

enumerator TensorrtExecutor

Use TensorRT, if it is installed.