Package ai.onnxruntime

Class OrtModelCompilationOptions

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    public final class OrtModelCompilationOptions
extends java.lang.Object
implements java.lang.AutoCloseable
    Configuration options for compiling ONNX models.

    • Method Detail

      • createFromSessionOptions

        public static OrtModelCompilationOptions createFromSessionOptions​(OrtEnvironment env,
                                                                  OrtSession.SessionOptions sessionOptions)
                                                           throws OrtException
        Creates a model compilation options from an existing SessionOptions.

        An OrtModelCompilationOptions object contains the settings used to generate a compiled ONNX model. The OrtSessionOptions object has the execution providers with which the model will be compiled.

        Parameters:
        env - The OrtEnvironment.
        sessionOptions - The session options to use.
        Returns:
        A constructed model compilation options instance.
        Throws:
        OrtException - If the construction failed.

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable

      • setInputModelPath

        public void setInputModelPath​(java.lang.String inputModelPath)
                       throws OrtException
        Sets the file path to the input ONNX model.

        The input model's location must be set either to a path on disk with this method, or by supplying an in-memory reference with setInputModelFromBuffer(java.nio.ByteBuffer).

        Parameters:
        inputModelPath - The path to the model on disk.
        Throws:
        OrtException - If the set failed.

      • setInputModelFromBuffer

        public void setInputModelFromBuffer​(java.nio.ByteBuffer inputModelBuffer)
                             throws OrtException
        Uses the supplied buffer as the input ONNX model.

        The input model's location must be set either to an in-memory reference with this method, or by supplying a path on disk with setInputModelPath(String).

        If the ByteBuffer is not direct it is copied into a direct buffer. In either case this object holds a reference to the buffer to prevent it from being GC'd.

        Parameters:
        inputModelBuffer - The buffer.
        Throws:
        OrtException - If the buffer could not be set.

      • setOutputModelPath

        public void setOutputModelPath​(java.lang.String outputModelPath)
                        throws OrtException
        Sets the file path for the output compiled ONNX model.

        If this is unset it will append `_ctx` to the file name, e.g., my_model.onnx becomes my_model_ctx.onnx.

        Parameters:
        outputModelPath - The output model path.
        Throws:
        OrtException - If the path could not be set.

      • setOutputExternalInitializersPath

        public void setOutputExternalInitializersPath​(java.lang.String outputExternalInitializersPath,
                                              long sizeThreshold)
                                       throws OrtException
        Optionally sets the file that stores initializers for the compiled ONNX model. If unset then initializers are stored inside the model.

        Only initializers for nodes that were not compiled are stored in the external initializers file. Compiled nodes contain their initializer data within the `ep_cache_context` attribute of EPContext nodes.

        Parameters:
        outputExternalInitializersPath - Path to the file.
        sizeThreshold - Initializers larger than this threshold are stored in the file.
        Throws:
        OrtException - If the path could not be set.
        See Also:
        setEpContextEmbedMode(boolean)

      • setEpContextEmbedMode

        public void setEpContextEmbedMode​(boolean embedEpContext)
                           throws OrtException
        Enables or disables the embedding of EPContext binary data into the ep_cache_context attribute of EPContext nodes.

        Defaults to false. When enabled, the `ep_cache_context` attribute of EPContext nodes will store the context binary data, which may include weights for compiled subgraphs. When disabled, the `ep_cache_context` attribute of EPContext nodes will contain the path to the file containing the context binary data. The path is set by the execution provider creating the EPContext node.

        For more details see the EPContext design document.

        Parameters:
        embedEpContext - True to embed EPContext binary data into the EPContext node's ep_cache_context attribute.
        Throws:
        OrtException - If the set operation failed.

      • compileModel

        public void compileModel()
                  throws OrtException
        Compiles the ONNX model with the configuration described by this instance of OrtModelCompilationOptions.
        Throws:
        OrtException - If the compilation failed.