Package ai.onnxruntime

A Java interface to the ONNX Runtime.

Provides access to the same execution backends as the C library. Non-representable types in Java (such as fp16) are converted into the nearest Java primitive type when accessed through this API.

There are two shared libraries required: onnxruntime and onnxruntime4j_jni . The loader is in OnnxRuntime and the logic is in this order:

1. The user may signal to skip loading of a shared library using a property in the form onnxruntime.native.LIB_NAME.skip with a value of true. This means the user has decided to load the library by some other means.
2. The user may specify an explicit location of all native library files using a property in the form onnxruntime.native.path. This uses System.load(java.lang.String).
3. The user may specify an explicit location of the shared library file using a property in the form onnxruntime.native.LIB_NAME.path. This uses System.load(java.lang.String).
4. The shared library is autodiscovered:
   1. If the shared library is present in the classpath resources, load using System.load(java.lang.String) via a temporary file. Ideally, this should be the default use case when adding JAR's/dependencies containing the shared libraries to your classpath.
   2. If the shared library is not present in the classpath resources, then load using System.loadLibrary(java.lang.String), which usually looks elsewhere on the filesystem for the library. The semantics and behavior of that method are system/JVM dependent. Typically, the java.library.path property is used to specify the location of native libraries.

For troubleshooting, all shared library loading events are reported to Java logging at the level FINE.

Note that CUDA, ROCM, DNNL, OpenVINO and TensorRT are all "shared library execution providers" and must be stored either in the directory containing the ONNX Runtime core native library, or as a classpath resource. This is because these providers are loaded by the ONNX Runtime native library itself and the Java API cannot control the loading location.

Interface Summary

Interface	Description
OnnxValue	Top interface for input and output values from ONNX models.
OrtProviderOptions.OrtProviderSupplier	Functional interface mirroring a Java supplier, but can throw OrtException.
ValueInfo	Interface for info objects describing an OnnxValue.

Class Summary

Class	Description
MapInfo	Describes an OnnxMap object or output node.
NodeInfo	The info for an input or output node from an ONNX model.
OnnxMap	A container for a map returned by OrtSession.run(Map).
OnnxModelMetadata	Contains the metadata associated with an ONNX model.
OnnxSequence	A sequence of OnnxValues all of the same type.
OnnxSparseTensor	A Java object wrapping an OnnxSparseTensor.
OnnxSparseTensor.BlockSparseTensor	The Java side representation of a block sparse tensor.
OnnxSparseTensor.COOTensor	The Java side representation of a COO sparse tensor.
OnnxSparseTensor.CSRCTensor	The Java side representation of a CSRC sparse tensor.
OnnxSparseTensor.SparseTensor<T extends java.nio.Buffer>	Abstract base class for Java sparse tensors
OnnxTensor	A Java object wrapping an OnnxTensor.
OnnxTensorLike	Currently implemented by OnnxTensor, OnnxSparseTensor.
OrtEnvironment	The host object for the ONNX Runtime system.
OrtEnvironment.ThreadingOptions	Controls the global thread pools in the environment.
OrtLoraAdapter	A container for an adapter which can be supplied to OrtSession.RunOptions.addActiveLoraAdapter(OrtLoraAdapter) to apply the adapter to a specific execution of a model.
OrtProviderOptions	An abstract base class for execution provider options classes.
OrtSession	Wraps an ONNX model and allows inference calls.
OrtSession.Result	An AutoCloseable wrapper around a Map containing OnnxValues.
OrtSession.RunOptions	Used to control logging and termination of a call to OrtSession.run(java.util.Map<java.lang.String, ? extends ai.onnxruntime.OnnxTensorLike>).
OrtSession.SessionOptions	Represents the options used to construct this session.
OrtTrainingSession	Wraps an ONNX training model and allows training and inference calls.
OrtUtil	Util code for interacting with Java arrays.
SequenceInfo	Describes an OnnxSequence, including it's element type if known.
TensorInfo	Describes an OnnxTensor, including it's size, shape and element type.

Enum Summary

Enum	Description
OnnxJavaType	An enum representing ONNX Runtime supported Java primitive types (and String).
OnnxMap.OnnxMapValueType	An enum representing the Java type of the values stored in an OnnxMap.
OnnxSparseTensor.SparseTensorType	The type of the sparse tensor.
OnnxValue.OnnxValueType	The type of the OnnxValue, mirroring the id in the C API.
OrtException.OrtErrorCode	Maps the OrtErrorCode struct in onnxruntime_c_api.h with an additional entry for Java side errors.
OrtLoggingLevel	The logging severity level.
OrtProvider	The execution providers available through the Java API.
OrtSession.SessionOptions.ExecutionMode	The execution mode to use.
OrtSession.SessionOptions.OptLevel	The optimisation level to use.
TensorInfo.OnnxTensorType	The native element types supported by the ONNX runtime.

Exception Summary

Exception	Description
OrtException	An exception which contains the error message and code produced by the native onnxruntime.