Note
Go to the end to download the full example code.
Common errors with onnxruntime#
This example looks into several common situations in which onnxruntime does not return the model prediction but raises an exception instead. It starts by loading the model trained in example Step 1: Train a model using your favorite framework which produced a logistic regression trained on Iris datasets. The model takes a vector of dimension 2 and returns a class among three.
import numpy
import onnxruntime as rt
from onnxruntime.capi.onnxruntime_pybind11_state import InvalidArgument
from onnxruntime.datasets import get_example
example2 = get_example("logreg_iris.onnx")
sess = rt.InferenceSession(example2, providers=rt.get_available_providers())
input_name = sess.get_inputs()[0].name
output_name = sess.get_outputs()[0].name
The first example fails due to bad types. onnxruntime only expects single floats (4 bytes) and cannot handle any other kind of floats.
try:
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float64)
sess.run([output_name], {input_name: x})
except Exception as e:
print("Unexpected type")
print(f"{type(e)}: {e}")
Unexpected type
<class 'onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument'>: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Unexpected input data type. Actual: (tensor(double)) , expected: (tensor(float))
The model fails to return an output if the name is misspelled.
try:
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
sess.run(["misspelled"], {input_name: x})
except Exception as e:
print("Misspelled output name")
print(f"{type(e)}: {e}")
Misspelled output name
<class 'onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument'>: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid output name:misspelled
The output name is optional, it can be replaced by None and onnxruntime will then return all the outputs.
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
try:
res = sess.run(None, {input_name: x})
print("All outputs")
print(res)
except (RuntimeError, InvalidArgument) as e:
print(e)
All outputs
[array([0, 0, 0], dtype=int64), [{0: 0.9505997896194458, 1: 0.027834143489599228, 2: 0.021566055715084076}, {0: 0.9974970817565918, 1: 5.6270167988259345e-05, 2: 0.0024466365575790405}, {0: 0.9997311234474182, 1: 1.787709464906584e-07, 2: 0.0002686927327886224}]]
The same goes if the input name is misspelled.
try:
x = numpy.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=numpy.float32)
sess.run([output_name], {"misspelled": x})
except Exception as e:
print("Misspelled input name")
print(f"{type(e)}: {e}")
Misspelled input name
<class 'ValueError'>: Required inputs (['float_input']) are missing from input feed (['misspelled']).
onnxruntime does not necessarily fail if the input dimension is a multiple of the expected input dimension.
for x in [
numpy.array([1.0, 2.0, 3.0, 4.0], dtype=numpy.float32),
numpy.array([[1.0, 2.0, 3.0, 4.0]], dtype=numpy.float32),
numpy.array([[1.0, 2.0], [3.0, 4.0]], dtype=numpy.float32),
numpy.array([1.0, 2.0, 3.0], dtype=numpy.float32),
numpy.array([[1.0, 2.0, 3.0]], dtype=numpy.float32),
]:
try:
r = sess.run([output_name], {input_name: x})
print(f"Shape={x.shape} and predicted labels={r}")
except (RuntimeError, InvalidArgument) as e:
print(f"ERROR with Shape={x.shape} - {e}")
for x in [
numpy.array([1.0, 2.0, 3.0, 4.0], dtype=numpy.float32),
numpy.array([[1.0, 2.0, 3.0, 4.0]], dtype=numpy.float32),
numpy.array([[1.0, 2.0], [3.0, 4.0]], dtype=numpy.float32),
numpy.array([1.0, 2.0, 3.0], dtype=numpy.float32),
numpy.array([[1.0, 2.0, 3.0]], dtype=numpy.float32),
]:
try:
r = sess.run(None, {input_name: x})
print(f"Shape={x.shape} and predicted probabilities={r[1]}")
except (RuntimeError, InvalidArgument) as e:
print(f"ERROR with Shape={x.shape} - {e}")
ERROR with Shape=(4,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs/outputs or the model.
ERROR with Shape=(1, 4) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
index: 0 Got: 1 Expected: 3
index: 1 Got: 4 Expected: 2
Please fix either the inputs/outputs or the model.
ERROR with Shape=(2, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
index: 0 Got: 2 Expected: 3
Please fix either the inputs/outputs or the model.
ERROR with Shape=(3,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs/outputs or the model.
ERROR with Shape=(1, 3) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
index: 0 Got: 1 Expected: 3
index: 1 Got: 3 Expected: 2
Please fix either the inputs/outputs or the model.
ERROR with Shape=(4,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs/outputs or the model.
ERROR with Shape=(1, 4) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
index: 0 Got: 1 Expected: 3
index: 1 Got: 4 Expected: 2
Please fix either the inputs/outputs or the model.
ERROR with Shape=(2, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
index: 0 Got: 2 Expected: 3
Please fix either the inputs/outputs or the model.
ERROR with Shape=(3,) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 1 Expected: 2 Please fix either the inputs/outputs or the model.
ERROR with Shape=(1, 3) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: float_input for the following indices
index: 0 Got: 1 Expected: 3
index: 1 Got: 3 Expected: 2
Please fix either the inputs/outputs or the model.
It does not fail either if the number of dimension is higher than expects but produces a warning.
for x in [
numpy.array([[[1.0, 2.0], [3.0, 4.0]]], dtype=numpy.float32),
numpy.array([[[1.0, 2.0, 3.0]]], dtype=numpy.float32),
numpy.array([[[1.0, 2.0]], [[3.0, 4.0]]], dtype=numpy.float32),
]:
try:
r = sess.run([output_name], {input_name: x})
print(f"Shape={x.shape} and predicted labels={r}")
except (RuntimeError, InvalidArgument) as e:
print(f"ERROR with Shape={x.shape} - {e}")
ERROR with Shape=(1, 2, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 3 Expected: 2 Please fix either the inputs/outputs or the model.
ERROR with Shape=(1, 1, 3) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 3 Expected: 2 Please fix either the inputs/outputs or the model.
ERROR with Shape=(2, 1, 2) - [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: float_input Got: 3 Expected: 2 Please fix either the inputs/outputs or the model.
Total running time of the script: (0 minutes 0.010 seconds)