Class OnnxTensor
- java.lang.Object
-
- ai.onnxruntime.OnnxTensorLike
-
- ai.onnxruntime.OnnxTensor
-
- All Implemented Interfaces:
OnnxValue
,java.lang.AutoCloseable
public class OnnxTensor extends OnnxTensorLike
A Java object wrapping an OnnxTensor. Tensors are the main input to the library, and can also be returned as outputs.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface ai.onnxruntime.OnnxValue
OnnxValue.OnnxValueType
-
-
Field Summary
-
Fields inherited from class ai.onnxruntime.OnnxTensorLike
allocatorHandle, closed, info, nativeHandle
-
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
Closes the tensor, releasing its underlying memory (if it's not backed by an NIO buffer).static OnnxTensor
createTensor(OrtEnvironment env, java.lang.Object data)
Create a Tensor from a Java primitive, primitive multidimensional array or String multidimensional array.static OnnxTensor
createTensor(OrtEnvironment env, java.lang.String[] data, long[] shape)
Create a tensor from a flattened string array.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.ByteBuffer data, long[] shape)
Create an OnnxTensor backed by a direct ByteBuffer.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.ByteBuffer data, long[] shape, OnnxJavaType type)
Create an OnnxTensor backed by a direct ByteBuffer.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.DoubleBuffer data, long[] shape)
Create an OnnxTensor backed by a direct DoubleBuffer.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.FloatBuffer data, long[] shape)
Create an OnnxTensor backed by a direct FloatBuffer.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.IntBuffer data, long[] shape)
Create an OnnxTensor backed by a direct IntBuffer.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.LongBuffer data, long[] shape)
Create an OnnxTensor backed by a direct LongBuffer.static OnnxTensor
createTensor(OrtEnvironment env, java.nio.ShortBuffer data, long[] shape)
Create an OnnxTensor backed by a direct ShortBuffer.java.util.Optional<java.nio.Buffer>
getBufferRef()
Returns a reference to the buffer which backs thisOnnxTensor
.java.nio.ByteBuffer
getByteBuffer()
Returns a copy of the underlying OnnxTensor as a ByteBuffer.java.nio.DoubleBuffer
getDoubleBuffer()
Returns a copy of the underlying OnnxTensor as a DoubleBuffer if the underlying type is a double, otherwise it returns null.java.nio.FloatBuffer
getFloatBuffer()
Returns a copy of the underlying OnnxTensor as a FloatBuffer if it can be losslessly converted into a float (i.e.java.nio.IntBuffer
getIntBuffer()
Returns a copy of the underlying OnnxTensor as an IntBuffer if the underlying type is int32 or uint32, otherwise it returns null.java.nio.LongBuffer
getLongBuffer()
Returns a copy of the underlying OnnxTensor as a LongBuffer if the underlying type is int64 or uint64, otherwise it returns null.java.nio.ShortBuffer
getShortBuffer()
Returns a copy of the underlying OnnxTensor as a ShortBuffer if the underlying type is int16, uint16, fp16 or bf16, otherwise it returns null.OnnxValue.OnnxValueType
getType()
Gets the type of this OnnxValue.java.lang.Object
getValue()
Either returns a boxed primitive if the Tensor is a scalar, or a multidimensional array of primitives if it has multiple dimensions.boolean
ownsBuffer()
Returns true if the buffer in this OnnxTensor was created on construction of this tensor, i.e., it is a copy of a user supplied buffer or array and may hold the only reference to that buffer.java.lang.String
toString()
-
Methods inherited from class ai.onnxruntime.OnnxTensorLike
checkClosed, getInfo, isClosed
-
-
-
-
Method Detail
-
ownsBuffer
public boolean ownsBuffer()
Returns true if the buffer in this OnnxTensor was created on construction of this tensor, i.e., it is a copy of a user supplied buffer or array and may hold the only reference to that buffer.When this is true the backing buffer was copied from the user input, so users cannot mutate the state of this buffer without first getting the reference via
getBufferRef()
.- Returns:
- True if the buffer in this OnnxTensor was allocated by it on construction (i.e., it is a copy of a user buffer.)
-
getBufferRef
public java.util.Optional<java.nio.Buffer> getBufferRef()
Returns a reference to the buffer which backs thisOnnxTensor
. If the tensor is not backed by a buffer (i.e., it was created from a Java array, or is backed by memory allocated by ORT) this method returns an emptyOptional
.Changes to the buffer elements will be reflected in the native
OrtValue
, this can be used to repeatedly update a single tensor for multiple different inferences without allocating new tensors, though the inputs must remain the same size and shape.Note: the tensor could refer to a contiguous range of elements in this buffer, not the whole buffer. It is up to the user to manage this information by respecting the position and limit. As a consequence, accessing this reference should be considered problematic when multiple threads hold references to the buffer.
- Returns:
- A reference to the buffer.
-
getType
public OnnxValue.OnnxValueType getType()
Description copied from interface:OnnxValue
Gets the type of this OnnxValue.- Returns:
- The value type.
-
getValue
public java.lang.Object getValue() throws OrtException
Either returns a boxed primitive if the Tensor is a scalar, or a multidimensional array of primitives if it has multiple dimensions.Java multidimensional arrays are quite slow for more than 2 dimensions, in that case it is recommended you use the
Buffer
extractors below (e.g.,getFloatBuffer()
).- Returns:
- A Java value.
- Throws:
OrtException
- If the value could not be extracted as the Tensor is invalid, or if the native code encountered an error.
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
close
public void close()
Closes the tensor, releasing its underlying memory (if it's not backed by an NIO buffer). If it is backed by a buffer then the memory is released when the buffer is GC'd.
-
getByteBuffer
public java.nio.ByteBuffer getByteBuffer()
Returns a copy of the underlying OnnxTensor as a ByteBuffer.This method returns null if the OnnxTensor contains Strings as they are stored externally to the OnnxTensor.
- Returns:
- A ByteBuffer copy of the OnnxTensor.
-
getFloatBuffer
public java.nio.FloatBuffer getFloatBuffer()
Returns a copy of the underlying OnnxTensor as a FloatBuffer if it can be losslessly converted into a float (i.e. it's a float, fp16 or bf16), otherwise it returns null.- Returns:
- A FloatBuffer copy of the OnnxTensor.
-
getDoubleBuffer
public java.nio.DoubleBuffer getDoubleBuffer()
Returns a copy of the underlying OnnxTensor as a DoubleBuffer if the underlying type is a double, otherwise it returns null.- Returns:
- A DoubleBuffer copy of the OnnxTensor.
-
getShortBuffer
public java.nio.ShortBuffer getShortBuffer()
Returns a copy of the underlying OnnxTensor as a ShortBuffer if the underlying type is int16, uint16, fp16 or bf16, otherwise it returns null.- Returns:
- A ShortBuffer copy of the OnnxTensor.
-
getIntBuffer
public java.nio.IntBuffer getIntBuffer()
Returns a copy of the underlying OnnxTensor as an IntBuffer if the underlying type is int32 or uint32, otherwise it returns null.- Returns:
- An IntBuffer copy of the OnnxTensor.
-
getLongBuffer
public java.nio.LongBuffer getLongBuffer()
Returns a copy of the underlying OnnxTensor as a LongBuffer if the underlying type is int64 or uint64, otherwise it returns null.- Returns:
- A LongBuffer copy of the OnnxTensor.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.lang.Object data) throws OrtException
Create a Tensor from a Java primitive, primitive multidimensional array or String multidimensional array. The shape is inferred from the object using reflection. The default allocator is used.Note: Java multidimensional arrays are not dense and this method requires traversing a large number of pointers for high dimensional arrays. For types other than Strings it is recommended to use one of the
createTensor
methods which accepts aBuffer
, e.g.createTensor(OrtEnvironment, FloatBuffer, long[])
as those methods are zero copy to transfer data into ORT when using direct buffers.- Parameters:
env
- The current OrtEnvironment.data
- The data to store in a tensor.- Returns:
- An OnnxTensor storing the data.
- Throws:
OrtException
- If the onnx runtime threw an error.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.lang.String[] data, long[] shape) throws OrtException
Create a tensor from a flattened string array.Requires the array to be flattened in row-major order. Uses the default allocator.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor datashape
- the shape of the tensor- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.FloatBuffer data, long[] shape) throws OrtException
Create an OnnxTensor backed by a direct FloatBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.DoubleBuffer data, long[] shape) throws OrtException
Create an OnnxTensor backed by a direct DoubleBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.ByteBuffer data, long[] shape) throws OrtException
Create an OnnxTensor backed by a direct ByteBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator. Tells the runtime it's
OnnxJavaType.INT8
.- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.ByteBuffer data, long[] shape, OnnxJavaType type) throws OrtException
Create an OnnxTensor backed by a direct ByteBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator. Tells the runtime it's the specified type.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.type
- The type to use for the byte buffer elements.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.ShortBuffer data, long[] shape) throws OrtException
Create an OnnxTensor backed by a direct ShortBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.IntBuffer data, long[] shape) throws OrtException
Create an OnnxTensor backed by a direct IntBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
createTensor
public static OnnxTensor createTensor(OrtEnvironment env, java.nio.LongBuffer data, long[] shape) throws OrtException
Create an OnnxTensor backed by a direct LongBuffer. The buffer should be in nativeOrder.If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.
- Parameters:
env
- The current OrtEnvironment.data
- The tensor data.shape
- The shape of tensor.- Returns:
- An OnnxTensor of the required shape.
- Throws:
OrtException
- Thrown if there is an onnx error or if the data and shape don't match.
-
-