Run ONNX model

Run ONNX model

Runs a machine learning model stored in the ONNX format.

Inputs

modelFile
Path to an ONNX model file.
executor
The preferred execution back-end for the machine learning model. This parameter can be used to force the tool to stick to one of the available execution back-ends. If the selected executor is not available, a generic CPU implementation will be used. Note that all supported executors are selectable even though the machine on which you use the tool doesn't have the required hardware and/or libraries.
deviceIndex
The index of the computation device used by the selected executor. Usually, this is the index of a CUDA device as listed by nvidia-smi. If the chosen device is not available at run time, the first device will be used instead.

Input tensors are defined by the model. They appear in the tool when a model has been loaded.

Outputs

Output tensors are defined by the model. They appear in the tool when a model has been loaded.

Execution back-ends for running the ONNX model.

Enumerator
AutoExecutor 

Execute the model on the "best" hardware possible. The preference order is system-dependent, but the fastest GPU will be generally tried first. If GPU acceleration is not available, an optimized CPU implementation will be used. If that fails, a generic CPU back-end will be used.

CpuExecutor 

Run the model on the CPU.

CudaExecutor 

Use CUDA (cuDNN, to be specific), if it is installed.

MkldnnExecutor 

Use MKL-DNN, if it is installed.

TensorrtExecutor 

Use TensorRT, if it is installed.