Run ONNX model🔗
Runs a machine learning model stored in the ONNX format.
modelFile: Path to an ONNX model file.
executor: The preferred execution back-end for the machine learning model. This parameter can be used to force the tool to stick to one of the available execution back-ends. If the selected executor is not available, a generic CPU implementation will be used. Note that all supported executors are selectable even though the machine on which you use the tool doesn’t have the required hardware and/or libraries.
deviceIndex: The index of the computation device used by the selected executor. Usually, this is the index of a CUDA device as listed by
nvidia-smi. If the chosen device is not available at run time, the first device will be used instead.
Input tensors are defined by the model. They appear in the tool when a model has been loaded.
Output tensors are defined by the model. They appear in the tool when a model has been loaded.
Execution back-ends for running the ONNX model.
Execute the model on the “best” hardware possible.
The preference order is system-dependent, but the fastest GPU will be generally tried first. If GPU acceleration is not available, an optimized CPU implementation will be used. If that fails, a generic CPU back-end will be used.
Run the model on the CPU.
Use CUDA (cuDNN, to be specific), if it is installed.
Use MKL-DNN, if it is installed.
Use TensorRT, if it is installed.