Class OrtSession

  • All Implemented Interfaces:
    java.lang.AutoCloseable

    public class OrtSession
    extends java.lang.Object
    implements java.lang.AutoCloseable
    Wraps an ONNX model and allows inference calls.

    Allows the inspection of the model's input and output nodes. Produced by an OrtEnvironment.

    Most instance methods throw IllegalStateException if the session is closed and the methods are called.

    • Method Detail

      • getNumInputs

        public long getNumInputs()
        Returns the number of inputs this model expects.
        Returns:
        The number of inputs.
      • getNumOutputs

        public long getNumOutputs()
        Returns the number of outputs this model expects.
        Returns:
        The number of outputs.
      • getInputNames

        public java.util.Set<java.lang.String> getInputNames()
        Returns the input names. The underlying collection is sorted based on the input id number.
        Returns:
        The input names.
      • getOutputNames

        public java.util.Set<java.lang.String> getOutputNames()
        Returns the output names. The underlying collection is sorted based on the output id number.
        Returns:
        The output names.
      • getInputInfo

        public java.util.Map<java.lang.String,​NodeInfo> getInputInfo()
                                                                    throws OrtException
        Returns the info objects for the inputs, including their names and types. The underlying collection is sorted based on the input id number.
        Returns:
        The input information.
        Throws:
        OrtException - If there was an error in native code.
      • getOutputInfo

        public java.util.Map<java.lang.String,​NodeInfo> getOutputInfo()
                                                                     throws OrtException
        Returns the info objects for the outputs, including their names and types. The underlying collection is sorted based on the output id number.
        Returns:
        The output information.
        Throws:
        OrtException - If there was an error in native code.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs)
                              throws OrtException
        Scores an input feed dict, returning the map of all inferred outputs.

        The outputs are sorted based on their id number.

        Parameters:
        inputs - The inputs to score.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input names are invalid, or if there are zero or too many inputs.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs,
                                     OrtSession.RunOptions runOptions)
                              throws OrtException
        Scores an input feed dict, returning the map of all inferred outputs.

        The outputs are sorted based on their id number.

        Parameters:
        inputs - The inputs to score.
        runOptions - The RunOptions to control this run.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input names are invalid, or if there are zero or too many inputs.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs,
                                     java.util.Set<java.lang.String> requestedOutputs)
                              throws OrtException
        Scores an input feed dict, returning the map of requested inferred outputs.

        The outputs are sorted based on the supplied set traversal order.

        Parameters:
        inputs - The inputs to score.
        requestedOutputs - The requested outputs.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs,
                                     java.util.Set<java.lang.String> requestedOutputs,
                                     OrtSession.RunOptions runOptions)
                              throws OrtException
        Scores an input feed dict, returning the map of requested inferred outputs.

        The outputs are sorted based on the supplied set traversal order.

        Parameters:
        inputs - The inputs to score.
        requestedOutputs - The requested outputs.
        runOptions - The RunOptions to control this run.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs,
                                     java.util.Map<java.lang.String,​? extends OnnxValue> pinnedOutputs)
                              throws OrtException
        Scores an input feed dict, returning the map of pinned outputs.

        The outputs are sorted based on the supplied map traversal order.

        Note: pinned outputs are not owned by the OrtSession.Result object, and are not closed when the result object is closed.

        Parameters:
        inputs - The inputs to score.
        pinnedOutputs - The requested outputs which the user has allocated.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs,
                                     java.util.Set<java.lang.String> requestedOutputs,
                                     java.util.Map<java.lang.String,​? extends OnnxValue> pinnedOutputs)
                              throws OrtException
        Scores an input feed dict, returning the map of requested and pinned outputs.

        The outputs are sorted based on the supplied set traversal order with pinned outputs first, then requested outputs. An IllegalArgumentException is thrown if the same output name appears in both the requested outputs and the pinned outputs.

        Note: pinned outputs are not owned by the OrtSession.Result object, and are not closed when the result object is closed.

        Parameters:
        inputs - The inputs to score.
        requestedOutputs - The requested outputs which ORT will allocate.
        pinnedOutputs - The requested outputs which the user has allocated.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.
      • run

        public OrtSession.Result run​(java.util.Map<java.lang.String,​? extends OnnxTensorLike> inputs,
                                     java.util.Set<java.lang.String> requestedOutputs,
                                     java.util.Map<java.lang.String,​? extends OnnxValue> pinnedOutputs,
                                     OrtSession.RunOptions runOptions)
                              throws OrtException
        Scores an input feed dict, returning the map of requested and pinned outputs.

        The outputs are sorted based on the supplied set traversal order with pinned outputs first, then requested outputs. An IllegalArgumentException is thrown if the same output name appears in both the requested outputs and the pinned outputs.

        Note: pinned outputs are not owned by the OrtSession.Result object, and are not closed when the result object is closed.

        Parameters:
        inputs - The inputs to score.
        requestedOutputs - The requested outputs which ORT will allocate.
        pinnedOutputs - The requested outputs which the user has allocated.
        runOptions - The RunOptions to control this run.
        Returns:
        The inferred outputs.
        Throws:
        OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.
      • getProfilingStartTimeInNs

        public long getProfilingStartTimeInNs()
                                       throws OrtException
        Returns the timestamp that profiling started in nanoseconds.
        Returns:
        the profiling start time in ns.
        Throws:
        OrtException - If the native call failed.
      • endProfiling

        public java.lang.String endProfiling()
                                      throws OrtException
        Ends the profiling session and returns the output of the profiler.

        Profiling should be enabled in the OrtSession.SessionOptions used to construct this Session.

        Returns:
        The profiling output.
        Throws:
        OrtException - If the native call failed.
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • close

        public void close()
                   throws OrtException
        Closes the session, releasing it's resources.
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        OrtException - If it failed to close.