Class OnnxTensor

  • All Implemented Interfaces:
    OnnxValue, java.lang.AutoCloseable

    public class OnnxTensor
    extends OnnxTensorLike
    A Java object wrapping an OnnxTensor. Tensors are the main input to the library, and can also be returned as outputs.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Closes the tensor, releasing its underlying memory (if it's not backed by an NIO buffer).
      static OnnxTensor createTensor​(OrtEnvironment env, java.lang.Object data)
      Create a Tensor from a Java primitive, primitive multidimensional array or String multidimensional array.
      static OnnxTensor createTensor​(OrtEnvironment env, java.lang.String[] data, long[] shape)
      Create a tensor from a flattened string array.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.ByteBuffer data, long[] shape)
      Create an OnnxTensor backed by a direct ByteBuffer.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.ByteBuffer data, long[] shape, OnnxJavaType type)
      Create an OnnxTensor backed by a direct ByteBuffer.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.DoubleBuffer data, long[] shape)
      Create an OnnxTensor backed by a direct DoubleBuffer.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.FloatBuffer data, long[] shape)
      Create an OnnxTensor backed by a direct FloatBuffer.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.IntBuffer data, long[] shape)
      Create an OnnxTensor backed by a direct IntBuffer.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.LongBuffer data, long[] shape)
      Create an OnnxTensor backed by a direct LongBuffer.
      static OnnxTensor createTensor​(OrtEnvironment env, java.nio.ShortBuffer data, long[] shape)
      Create an OnnxTensor backed by a direct ShortBuffer.
      java.util.Optional<java.nio.Buffer> getBufferRef()
      Returns a reference to the buffer which backs this OnnxTensor.
      java.nio.ByteBuffer getByteBuffer()
      Returns a copy of the underlying OnnxTensor as a ByteBuffer.
      java.nio.DoubleBuffer getDoubleBuffer()
      Returns a copy of the underlying OnnxTensor as a DoubleBuffer if the underlying type is a double, otherwise it returns null.
      java.nio.FloatBuffer getFloatBuffer()
      Returns a copy of the underlying OnnxTensor as a FloatBuffer if it can be losslessly converted into a float (i.e.
      java.nio.IntBuffer getIntBuffer()
      Returns a copy of the underlying OnnxTensor as an IntBuffer if the underlying type is int32 or uint32, otherwise it returns null.
      java.nio.LongBuffer getLongBuffer()
      Returns a copy of the underlying OnnxTensor as a LongBuffer if the underlying type is int64 or uint64, otherwise it returns null.
      java.nio.ShortBuffer getShortBuffer()
      Returns a copy of the underlying OnnxTensor as a ShortBuffer if the underlying type is int16, uint16, fp16 or bf16, otherwise it returns null.
      OnnxValue.OnnxValueType getType()
      Gets the type of this OnnxValue.
      java.lang.Object getValue()
      Either returns a boxed primitive if the Tensor is a scalar, or a multidimensional array of primitives if it has multiple dimensions.
      boolean ownsBuffer()
      Returns true if the buffer in this OnnxTensor was created on construction of this tensor, i.e., it is a copy of a user supplied buffer or array and may hold the only reference to that buffer.
      java.lang.String toString()  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Method Detail

      • ownsBuffer

        public boolean ownsBuffer()
        Returns true if the buffer in this OnnxTensor was created on construction of this tensor, i.e., it is a copy of a user supplied buffer or array and may hold the only reference to that buffer.

        When this is true the backing buffer was copied from the user input, so users cannot mutate the state of this buffer without first getting the reference via getBufferRef().

        Returns:
        True if the buffer in this OnnxTensor was allocated by it on construction (i.e., it is a copy of a user buffer.)
      • getBufferRef

        public java.util.Optional<java.nio.Buffer> getBufferRef()
        Returns a reference to the buffer which backs this OnnxTensor. If the tensor is not backed by a buffer (i.e., it was created from a Java array, or is backed by memory allocated by ORT) this method returns an empty Optional.

        Changes to the buffer elements will be reflected in the native OrtValue, this can be used to repeatedly update a single tensor for multiple different inferences without allocating new tensors, though the inputs must remain the same size and shape.

        Note: the tensor could refer to a contiguous range of elements in this buffer, not the whole buffer. It is up to the user to manage this information by respecting the position and limit. As a consequence, accessing this reference should be considered problematic when multiple threads hold references to the buffer.

        Returns:
        A reference to the buffer.
      • getValue

        public java.lang.Object getValue()
                                  throws OrtException
        Either returns a boxed primitive if the Tensor is a scalar, or a multidimensional array of primitives if it has multiple dimensions.

        Java multidimensional arrays are quite slow for more than 2 dimensions, in that case it is recommended you use the Buffer extractors below (e.g., getFloatBuffer()).

        Returns:
        A Java value.
        Throws:
        OrtException - If the value could not be extracted as the Tensor is invalid, or if the native code encountered an error.
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • close

        public void close()
        Closes the tensor, releasing its underlying memory (if it's not backed by an NIO buffer). If it is backed by a buffer then the memory is released when the buffer is GC'd.
      • getByteBuffer

        public java.nio.ByteBuffer getByteBuffer()
        Returns a copy of the underlying OnnxTensor as a ByteBuffer.

        This method returns null if the OnnxTensor contains Strings as they are stored externally to the OnnxTensor.

        Returns:
        A ByteBuffer copy of the OnnxTensor.
      • getFloatBuffer

        public java.nio.FloatBuffer getFloatBuffer()
        Returns a copy of the underlying OnnxTensor as a FloatBuffer if it can be losslessly converted into a float (i.e. it's a float, fp16 or bf16), otherwise it returns null.
        Returns:
        A FloatBuffer copy of the OnnxTensor.
      • getDoubleBuffer

        public java.nio.DoubleBuffer getDoubleBuffer()
        Returns a copy of the underlying OnnxTensor as a DoubleBuffer if the underlying type is a double, otherwise it returns null.
        Returns:
        A DoubleBuffer copy of the OnnxTensor.
      • getShortBuffer

        public java.nio.ShortBuffer getShortBuffer()
        Returns a copy of the underlying OnnxTensor as a ShortBuffer if the underlying type is int16, uint16, fp16 or bf16, otherwise it returns null.
        Returns:
        A ShortBuffer copy of the OnnxTensor.
      • getIntBuffer

        public java.nio.IntBuffer getIntBuffer()
        Returns a copy of the underlying OnnxTensor as an IntBuffer if the underlying type is int32 or uint32, otherwise it returns null.
        Returns:
        An IntBuffer copy of the OnnxTensor.
      • getLongBuffer

        public java.nio.LongBuffer getLongBuffer()
        Returns a copy of the underlying OnnxTensor as a LongBuffer if the underlying type is int64 or uint64, otherwise it returns null.
        Returns:
        A LongBuffer copy of the OnnxTensor.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.lang.Object data)
                                       throws OrtException
        Create a Tensor from a Java primitive, primitive multidimensional array or String multidimensional array. The shape is inferred from the object using reflection. The default allocator is used.

        Note: Java multidimensional arrays are not dense and this method requires traversing a large number of pointers for high dimensional arrays. For types other than Strings it is recommended to use one of the createTensor methods which accepts a Buffer, e.g. createTensor(OrtEnvironment, FloatBuffer, long[]) as those methods are zero copy to transfer data into ORT when using direct buffers.

        Parameters:
        env - The current OrtEnvironment.
        data - The data to store in a tensor.
        Returns:
        An OnnxTensor storing the data.
        Throws:
        OrtException - If the onnx runtime threw an error.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.lang.String[] data,
                                              long[] shape)
                                       throws OrtException
        Create a tensor from a flattened string array.

        Requires the array to be flattened in row-major order. Uses the default allocator.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data
        shape - the shape of the tensor
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.FloatBuffer data,
                                              long[] shape)
                                       throws OrtException
        Create an OnnxTensor backed by a direct FloatBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.DoubleBuffer data,
                                              long[] shape)
                                       throws OrtException
        Create an OnnxTensor backed by a direct DoubleBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.ByteBuffer data,
                                              long[] shape)
                                       throws OrtException
        Create an OnnxTensor backed by a direct ByteBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator. Tells the runtime it's OnnxJavaType.INT8.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.ByteBuffer data,
                                              long[] shape,
                                              OnnxJavaType type)
                                       throws OrtException
        Create an OnnxTensor backed by a direct ByteBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator. Tells the runtime it's the specified type.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        type - The type to use for the byte buffer elements.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.ShortBuffer data,
                                              long[] shape)
                                       throws OrtException
        Create an OnnxTensor backed by a direct ShortBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.IntBuffer data,
                                              long[] shape)
                                       throws OrtException
        Create an OnnxTensor backed by a direct IntBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.
      • createTensor

        public static OnnxTensor createTensor​(OrtEnvironment env,
                                              java.nio.LongBuffer data,
                                              long[] shape)
                                       throws OrtException
        Create an OnnxTensor backed by a direct LongBuffer. The buffer should be in nativeOrder.

        If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

        Parameters:
        env - The current OrtEnvironment.
        data - The tensor data.
        shape - The shape of tensor.
        Returns:
        An OnnxTensor of the required shape.
        Throws:
        OrtException - Thrown if there is an onnx error or if the data and shape don't match.