Class FixedBufferOnnxValue

This is a legacy class that is kept for backward compatibility. Use OrtValue based API.

Represents an OrtValue with its underlying buffer pinned

Inheritance

Object

FixedBufferOnnxValue

Implements

IDisposable

Inherited Members

Object.Equals(Object)

Object.Equals(Object, Object)

Object.GetHashCode()

Object.GetType()

Object.MemberwiseClone()

Object.ReferenceEquals(Object, Object)

Object.ToString()

Namespace: Microsoft.ML.OnnxRuntime

Assembly: Microsoft.ML.OnnxRuntime.dll

Syntax

public class FixedBufferOnnxValue : IDisposable

Methods

CreateFromMemory<T>(OrtMemoryInfo, Memory<T>, TensorElementType, Int64[], Int64)

This is a factory method that creates a disposable instance of FixedBufferOnnxValue on top of a buffer. Internally, it will pin managed buffer and will create an OrtValue containing a tensor that will not own the memory. Such instance of FixedBufferOnnxValue can be used both as input and output in InferenceSession.Run() overload. As compared to CreateFromTensor(), this allows you to pass in buffers with custom data types that are blittable as defined in https://docs.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types I.e. those that have the same binary representation as the original type. This includes all existing types but may also allow using custom types for Float16 and BFloat16 providing they have the same layout and size. The resulting instance must be disposed of to release pinned memory and deallocate native OrtValue See example below.

Declaration

public static FixedBufferOnnxValue CreateFromMemory<T>(OrtMemoryInfo memoryInfo, Memory<T> memory, TensorElementType elementType, long[] shape, long bytesSize)
    where T : struct

Parameters

Type	Name	Description
OrtMemoryInfo	memoryInfo	memoryInfo. For managed buffers simply use OrtMemoryInfo.DefaultInstance
Memory<T>	memory
TensorElementType	elementType	TensorElementType
Int64[]	shape	shape of the tensor to be created
Int64	bytesSize	size of the allocation in bytes

Returns

Type	Description
FixedBufferOnnxValue	a disposable instance of FixedBufferOnnxValue

Type Parameters

Name	Description
T	Blittable data type, compatible with supported types

Examples

Here is an example of using a 3rd party library class for processing float16/bfloat16. Currently, to pass tensor data and create a tensor one must copy data to Float16/BFloat16 structures so DenseTensor can recognize it.

If you are using a library that has a class Half and it is blittable, that is its managed in memory representation matches native one and its size is 16-bits, you can use the following conceptual example to feed/fetch data for inference using Half array. This allows you to avoid copying data from your Half[] to Float16[]

\code{.cs} unsafe { Debug.Assert(sizeof(ushort) == sizeof(Half)); } Half[] input = new Half[] { 5646, 12345 }; var input_shape = new long[] {input.Length}; Half[] output = new Half[40]; // Whatever the expected len/shape is must match var output_shape = new long[] {output.Length};

var memInfo = OrtMemoryInfo.DefaultInstance; // CPU

using(var fixedBufferInput = FixedBufferOnnxvalue.CreateFromMemory{Half}(memInfo, input, TensorElementType.Float16, input_shape, input.Length * sizeof(ushort)) using(var fixedBufferOutput = FixedBufferOnnxvalue.CreateFromMemory{Half}(memInfo, output, TensorElementType.Float16, output_shape, output.Length * sizeof(ushort)) { FixedBufferOnnxvalue[] inputValues = new FixedBufferOnnxvalue[]{fixedBufferInput}; FixedBufferOnnxvalue[] outputValues = new FixedBufferOnnxvalue[]{fixedBufferOutput}; session.Run(inputNames, inputValues, outputNames, outputValues); // Output is now in output[] } \endcode

CreateFromTensor<T>(Tensor<T>)

Creates a FixedBufferOnnxValue object from the tensor and pins its underlying buffer.

Declaration

public static FixedBufferOnnxValue CreateFromTensor<T>(Tensor<T> value)

Parameters

Type	Name	Description
Tensor<T>	value

Returns

Type	Description
FixedBufferOnnxValue	a disposable instance of FixedBufferOnnxValue

Type Parameters

Name	Description
T

Dispose()

IDisposable implementation

Declaration

public void Dispose()

Dispose(Boolean)

IDisposable implementation

Declaration

protected virtual void Dispose(bool disposing)

Parameters

Type	Name	Description
Boolean	disposing	true if invoked from Dispose()

Implements

System.IDisposable