Package ai.onnxruntime

Class OrtEnvironment

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    public final class OrtEnvironment
extends java.lang.Object
implements java.lang.AutoCloseable
    The host object for the ONNX Runtime system. Can create OrtSessions which encapsulate specific models. This object should be instantiated before any other ONNX Runtime classes are created.

    There can be at most one OrtEnvironment object created in a JVM lifetime. This class implements AutoCloseable as before for backwards compatibility with 1.10 and earlier, but the close method is a no-op. The environment is closed by a JVM shutdown hook registered on construction.

    • Field Detail

      • DEFAULT_NAME

        public static final java.lang.String DEFAULT_NAME
        The default name for ORT environments constructed from Java.
        See Also:
        Constant Field Values

    • Method Detail

      • getEnvironment

        public static OrtEnvironment getEnvironment​(java.lang.String name)
        Gets the OrtEnvironment. If there is not an environment currently created, it creates one using the supplied name and OrtLoggingLevel.ORT_LOGGING_LEVEL_WARNING.

        If the environment already exists then it returns the existing one and logs a warning if the name or log level is different from the requested one.

        Parameters:
        name - The logging id of the environment.
        Returns:
        The OrtEnvironment singleton.

      • getEnvironment

        public static OrtEnvironment getEnvironment​(OrtLoggingLevel logLevel)
        Gets the OrtEnvironment. If there is not an environment currently created, it creates one using the DEFAULT_NAME and the supplied logging level.

        If the environment already exists then it returns the existing one and logs a warning if the name or log level is different from the requested one.

        Parameters:
        logLevel - The logging level to use.
        Returns:
        The OrtEnvironment singleton.

      • getEnvironment

        public static OrtEnvironment getEnvironment​(OrtLoggingLevel loggingLevel,
                                            java.lang.String name)
        Gets the OrtEnvironment. If there is not an environment currently created, it creates one using the supplied name and logging level. If an environment already exists with a different name, that environment is returned and a warning is logged.
        Parameters:
        loggingLevel - The logging level to use.
        name - The log id.
        Returns:
        The OrtEnvironment singleton.

      • getEnvironment

        public static OrtEnvironment getEnvironment​(OrtLoggingLevel loggingLevel,
                                            java.lang.String name,
                                            OrtEnvironment.ThreadingOptions threadOptions)
        Creates an OrtEnvironment using the specified global thread pool options. Note unlike the other getEnvironment methods if there already is an existing OrtEnvironment this call throws IllegalStateException as we cannot guarantee that the environment has the appropriate thread pool configuration.
        Parameters:
        loggingLevel - The logging level to use.
        name - The log id.
        threadOptions - The global thread pool options.
        Returns:
        The OrtEnvironment singleton.

      • createSession

        public OrtSession createSession​(java.lang.String modelPath)
                         throws OrtException
        Create a session using the default OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        modelPath - Path on disk to load the model from.
        Returns:
        An OrtSession with the specified model.
        Throws:
        OrtException - If the model failed to load, wasn't compatible or caused an error.

      • createSession

        public OrtSession createSession​(java.lang.String modelPath,
                                OrtSession.SessionOptions options)
                         throws OrtException
        Create a session using the specified OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        modelPath - Path on disk to load the model from.
        options - The session options.
        Returns:
        An OrtSession with the specified model.
        Throws:
        OrtException - If the model failed to load, wasn't compatible or caused an error.

      • createSession

        public OrtSession createSession​(java.nio.ByteBuffer modelBuffer,
                                OrtSession.SessionOptions options)
                         throws OrtException
        Create a session using the specified OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        modelBuffer - Byte buffer representing an ONNX model. Must be a direct byte buffer.
        options - The session options.
        Returns:
        An OrtSession with the specified model.
        Throws:
        OrtException - If the model failed to parse, wasn't compatible or caused an error.

      • createSession

        public OrtSession createSession​(java.nio.ByteBuffer modelBuffer)
                         throws OrtException
        Create a session using the default OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        modelBuffer - Byte buffer representing an ONNX model. Must be a direct byte buffer.
        Returns:
        An OrtSession with the specified model.
        Throws:
        OrtException - If the model failed to parse, wasn't compatible or caused an error.

      • createSession

        public OrtSession createSession​(byte[] modelArray)
                         throws OrtException
        Create a session using the default OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        modelArray - Byte array representing an ONNX model.
        Returns:
        An OrtSession with the specified model.
        Throws:
        OrtException - If the model failed to parse, wasn't compatible or caused an error.

      • createTrainingSession

        public OrtTrainingSession createTrainingSession​(java.lang.String checkpointPath,
                                                java.lang.String trainPath,
                                                java.lang.String evalPath,
                                                java.lang.String optimizerPath)
                                         throws OrtException
        Create a training session using the default OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        checkpointPath - Path to the checkpoint folder.
        trainPath - Path to the training model.
        evalPath - Path to the evaluation model. Null signifies there is no eval model.
        optimizerPath - Path to the optimizer model. Null signifies there is no optimizer model.
        Returns:
        An OrtTrainingSession with the specified model loaded.
        Throws:
        OrtException - If the model failed to load, wasn't compatible or caused an error.

      • createTrainingSession

        public OrtTrainingSession createTrainingSession​(java.lang.String checkpointPath,
                                                java.lang.String trainPath,
                                                java.lang.String evalPath,
                                                java.lang.String optimizerPath,
                                                OrtSession.SessionOptions options)
                                         throws OrtException
        Create a training session using the specified OrtSession.SessionOptions, model and the default memory allocator.
        Parameters:
        checkpointPath - Path to the checkpoint folder.
        trainPath - Path to the training model.
        evalPath - Path to the evaluation model. Null signifies there is no eval model.
        optimizerPath - Path to the optimizer model. Null signifies there is no optimizer model.
        options - The session options.
        Returns:
        An OrtTrainingSession with the specified model.
        Throws:
        OrtException - If the model failed to load, wasn't compatible or caused an error.

      • isTrainingEnabled

        public boolean isTrainingEnabled()
        Is training enabled in this build of ONNX Runtime?
        Returns:
        True if training is enabled.

      • setTelemetry

        public void setTelemetry​(boolean sendTelemetry)
                  throws OrtException
        Turns on or off the telemetry.
        Parameters:
        sendTelemetry - If true then send telemetry on ONNX Runtime usage.
        Throws:
        OrtException - If the call failed.

      • getVersion

        public java.lang.String getVersion()
        Gets the native library version string.
        Returns:
        The version string.

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • getAvailableProviders

        public static java.util.EnumSet<OrtProvider> getAvailableProviders()
        Gets the providers available in this environment.
        Returns:
        An enum set of the available execution providers.

      • registerExecutionProviderLibrary

        public void registerExecutionProviderLibrary​(java.lang.String registrationName,
                                             java.lang.String libraryPath)
                                      throws OrtException
        Registers an execution provider library with this OrtEnvironment.
        Parameters:
        registrationName - The name to register the library with (used to remove it later with unregisterExecutionProviderLibrary(String)).
        libraryPath - The path to the library binary on disk.
        Throws:
        OrtException - If the library could not be registered.

      • unregisterExecutionProviderLibrary

        public void unregisterExecutionProviderLibrary​(java.lang.String registrationName)
                                        throws OrtException
        Unregisters an execution provider library from this OrtEnvironment.
        Parameters:
        registrationName - The name the library was registered under.
        Throws:
        OrtException - If the library could not be removed.

      • getModelCompatibilityForEpDevices

        public OrtEnvironment.OrtCompiledModelCompatibility getModelCompatibilityForEpDevices​(java.util.List<OrtEpDevice> epDevices,
                                                                                      java.lang.String modelInfo)
                                                                               throws OrtException
        Checks the supplied model info string against the list of OrtEpDevices to see if the model is compatible.
        Parameters:
        epDevices - The EP-Device tuples to use.
        modelInfo - The model info string to check.
        Returns:
        The model compatibility.
        Throws:
        OrtException - If the call failed.

      • close

        public void close()
        Close is a no-op on OrtEnvironment since ORT 1.11.
        Specified by:
        close in interface java.lang.AutoCloseable