Class InferenceSession

Represents an Inference Session on an ONNX Model. This is a IDisposable class and it must be disposed of using either a explicit call to Dispose() method or a pattern of using() block. If this is a member of another class that class must also become IDisposable and it must dispose of InferenceSession in its Dispose() method.

Inheritance

Object

InferenceSession

Implements

IDisposable

Inherited Members

Object.Equals(Object)

Object.Equals(Object, Object)

Object.GetHashCode()

Object.GetType()

Object.MemberwiseClone()

Object.ReferenceEquals(Object, Object)

Object.ToString()

Namespace: Microsoft.ML.OnnxRuntime

Assembly: Microsoft.ML.OnnxRuntime.dll

Syntax

public class InferenceSession : IDisposable

Constructors

InferenceSession(Byte[])

Constructs an InferenceSession from a model data in byte array

Declaration

public InferenceSession(byte[] model)

Parameters

Type	Name	Description
Byte[]	model

InferenceSession(Byte[], PrePackedWeightsContainer)

Constructs an InferenceSession from a model data (in byte array) and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

Declaration

public InferenceSession(byte[] model, PrePackedWeightsContainer prepackedWeightsContainer)

Parameters

Type	Name	Description
Byte[]	model	Model as byte array
PrePackedWeightsContainer	prepackedWeightsContainer	Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

InferenceSession(Byte[], SessionOptions)

Constructs an InferenceSession from a model data in byte array, with some additional session options

Declaration

public InferenceSession(byte[] model, SessionOptions options)

Parameters

Type	Name	Description
Byte[]	model
SessionOptions	options

InferenceSession(Byte[], SessionOptions, PrePackedWeightsContainer)

Constructs an InferenceSession from a model data (in byte array) with some additional session options and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

Declaration

public InferenceSession(byte[] model, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)

Parameters

Type	Name	Description
Byte[]	model	Model as byte array
SessionOptions	options	Session Options
PrePackedWeightsContainer	prepackedWeightsContainer	Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

InferenceSession(String)

Constructs an InferenceSession from a model file

Declaration

public InferenceSession(string modelPath)

Parameters

Type	Name	Description
String	modelPath

InferenceSession(String, PrePackedWeightsContainer)

Constructs an InferenceSession from a model file and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

Declaration

public InferenceSession(string modelPath, PrePackedWeightsContainer prepackedWeightsContainer)

Parameters

Type	Name	Description
String	modelPath	Model path
PrePackedWeightsContainer	prepackedWeightsContainer	Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

InferenceSession(String, SessionOptions)

Constructs an InferenceSession from a model file, with some additional session options

Declaration

public InferenceSession(string modelPath, SessionOptions options)

Parameters

Type	Name	Description
String	modelPath
SessionOptions	options

InferenceSession(String, SessionOptions, PrePackedWeightsContainer)

Constructs an InferenceSession from a model file, with some additional session options and it will use the provided pre-packed weights container to store and share pre-packed buffers of shared initializers across sessions if any.

Declaration

public InferenceSession(string modelPath, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)

Parameters

Type	Name	Description
String	modelPath	Model path
SessionOptions	options	Session options
PrePackedWeightsContainer	prepackedWeightsContainer	Instance of PrepackedWeightsContainer. Lifetime of 'prepackedWeightsContainer' must be managed by the user and it must outlive any sessions reliant on it

Properties

InputMetadata

Meta data regarding the input nodes, keyed by input names

Declaration

public IReadOnlyDictionary<string, NodeMetadata> InputMetadata { get; }

Property Value

Type	Description
IReadOnlyDictionary<String, NodeMetadata>

InputNames

Ordered list of input names that can be accessed by index;

Declaration

public IReadOnlyList<string> InputNames { get; }

Property Value

Type	Description
IReadOnlyList<String>

ModelMetadata

This property queries model metadata, constructs an instance of ModelMetadata and caches it

Declaration

public ModelMetadata ModelMetadata { get; }

Property Value

Type	Description
ModelMetadata	Instance of ModelMetdata

OutputMetadata

Metadata regarding the output nodes, keyed by output names

Declaration

public IReadOnlyDictionary<string, NodeMetadata> OutputMetadata { get; }

Property Value

Type	Description
IReadOnlyDictionary<String, NodeMetadata>

OutputNames

Ordered list of output names that can be accessed by index.

Declaration

public IReadOnlyList<string> OutputNames { get; }

Property Value

Type	Description
IReadOnlyList<String>

OverridableInitializerMetadata

Metadata regarding the overridable initializers, keyed by node names

Declaration

public IReadOnlyDictionary<string, NodeMetadata> OverridableInitializerMetadata { get; }

Property Value

Type	Description
IReadOnlyDictionary<String, NodeMetadata>

ProfilingStartTimeNs

Return the nanoseconds of profiling's start time On some platforms, this timer may not be as precise as nanoseconds For instance, on Windows and MacOS, the precision will be ~100ns

Declaration

public ulong ProfilingStartTimeNs { get; }

Property Value

Type	Description
UInt64

Methods

CreateIoBinding()

Create OrtIoBinding instance to bind pre-allocated buffers to input/output

Declaration

public OrtIoBinding CreateIoBinding()

Returns

Type	Description
OrtIoBinding	A new instance of OrtIoBinding

Dispose()

IDisposable implementation

Declaration

public void Dispose()

Dispose(Boolean)

IDisposable implementation

Declaration

protected virtual void Dispose(bool disposing)

Parameters

Type	Name	Description
Boolean	disposing	true if invoked from Dispose() method

EndProfiling()

Ends profiling for the session.

Declaration

public string EndProfiling()

Returns

Type	Description
String	Returns the profile file name.

Finalize()

Finalizer. to cleanup session in case it runs and the user forgets to Dispose() of the session

Declaration

protected void Finalize()

Run(RunOptions, IReadOnlyCollection<String>, IReadOnlyCollection<OrtValue>, IReadOnlyCollection<String>)

The API runs the inference taking a collection of OrtValues as input and returning a collection of output OrtValues.

Declaration

public IDisposableReadOnlyCollection<OrtValue> Run(RunOptions runOptions, IReadOnlyCollection<string> inputNames, IReadOnlyCollection<OrtValue> inputValues, IReadOnlyCollection<string> outputNames)

Parameters

Type	Name	Description
RunOptions	runOptions	runOptions
IReadOnlyCollection<String>	inputNames	A collection of input names. To supply all names, use InputNames property
IReadOnlyCollection<OrtValue>	inputValues	Input OrtValues. The size of the collection must match the size and the order of the inputNames
IReadOnlyCollection<String>	outputNames	Output names requested. To supply all names, use OutputNames property.

Returns

Type	Description
IDisposableReadOnlyCollection<OrtValue>	A disposable collection of disposable OrtValues

Exceptions

Type	Condition
System.ArgumentException

Run(RunOptions, IReadOnlyCollection<String>, IReadOnlyCollection<OrtValue>, IReadOnlyCollection<String>, IReadOnlyCollection<OrtValue>)

The API takes collections of inputNames/inputValues and collections of outputNames/outputValues. The sizes of the corresponding collections must match.

The output OrtValues are pre-allocated and the API will fill the data into the OrtValues. These MUST be tensors. The API does not support non-tensor types for output values.

The API is useful when the output values are tensors and their shapes are known, and you prefer the output to go to the pre-allocated memory. In such a case, you create output OrtValues over those pre-allocated buffers and pass them to the API.

Declaration

public void Run(RunOptions runOptions, IReadOnlyCollection<string> inputNames, IReadOnlyCollection<OrtValue> inputValues, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<OrtValue> outputValues)

Parameters

Type	Name	Description
RunOptions	runOptions	runOptions, if null the defaults are used
IReadOnlyCollection<String>	inputNames	collection of input names.
IReadOnlyCollection<OrtValue>	inputValues	collection of input OrtValues. Must match the order and the number of input names.
IReadOnlyCollection<String>	outputNames	Requested output names.
IReadOnlyCollection<OrtValue>	outputValues	Pre-allocated output values. The order and the number must match the specified output names. Shapes must match actual output values.

Exceptions

Type	Condition
System.ArgumentException

Run(RunOptions, IReadOnlyDictionary<String, OrtValue>, IReadOnlyCollection<String>)

This API takes inputs as a dictionary of input names paired with input OrtValues

It returns a disposable collection of OrtValues for outputs that were designated by outputNames

Declaration

public IDisposableReadOnlyCollection<OrtValue> Run(RunOptions runOptions, IReadOnlyDictionary<string, OrtValue> inputs, IReadOnlyCollection<string> outputNames)

Parameters

Type	Name	Description
RunOptions	runOptions
IReadOnlyDictionary<String, OrtValue>	inputs	Dictionary of name/value pairs
IReadOnlyCollection<String>	outputNames	requested outputs. To request all outputs, use OutputNames property of this sessions

Returns

Type	Description
IDisposableReadOnlyCollection<OrtValue>	A disposable collection of outputs

Run(IReadOnlyCollection<NamedOnnxValue>)

Runs the loaded model for the given inputs, and fetches all the outputs.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<NamedOnnxValue> inputs)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	specify a collection of NamedOnnxValue that indicates the input values.

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<NamedOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to receive the fetched data.

Declaration

public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<NamedOnnxValue> outputs)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue>	outputs	Specify a collection of NamedOnnxValue that indicates the output values.

Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<NamedOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to receive the fetched data.

Declaration

public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<NamedOnnxValue> outputs, RunOptions options)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue>	outputs	Specify a collection of NamedOnnxValue that indicates the output values.
RunOptions	options

Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>)

Runs the loaded model for the given inputs, and fetches the outputs specified in outputNames.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names to fetch.

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>, RunOptions)

Runs the loaded model for the given inputs, and fetches the specified outputs in outputNames. Uses the given RunOptions for this run.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames, RunOptions options)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names to fetch.
RunOptions	options

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to receive the fetched data.

Declaration

public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names. Should match outputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	outputValues	Specify a collection of FixedBufferOnnxValue that indicates the output values.

Run(IReadOnlyCollection<NamedOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to receive the fetched data.

Declaration

public void Run(IReadOnlyCollection<NamedOnnxValue> inputs, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues, RunOptions options)

Parameters

Type	Name	Description
IReadOnlyCollection<NamedOnnxValue>	inputs	Specify a collection of NamedOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names. Should match outputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	outputValues	Specify a collection of FixedBufferOnnxValue that indicates the output values.
RunOptions	options

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>)

Runs the loaded model for the given inputs, and fetches all the outputs.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<NamedOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to receive the fetched data.

Declaration

public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<NamedOnnxValue> outputs)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue>	outputs	Specify a collection of NamedOnnxValue that indicates the output values.

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<NamedOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to receive the fetched data.

Declaration

public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<NamedOnnxValue> outputs, RunOptions options)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<NamedOnnxValue>	outputs	Specify a collection of NamedOnnxValue that indicates the output values.
RunOptions	options

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>)

Runs the loaded model for the given inputs, and fetches the outputs specified in outputNames.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names to fetch.

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>, RunOptions)

Runs the loaded model for the given inputs, and fetches the specified outputs in outputNames. Uses the given RunOptions for this run.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames, RunOptions options)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names to fetch.
RunOptions	options

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	Output Tensors in a Collection of NamedOnnxValue. User must dispose the output.

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>)

Runs the loaded model for the given inputs and outputs.

Outputs need to be created with correct type and dimension to accept the fetched data.

Declaration

public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names. Should match outputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	outputValues	Specify a collection of FixedBufferOnnxValue that indicates the output values.

Run(IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, IReadOnlyCollection<String>, IReadOnlyCollection<FixedBufferOnnxValue>, RunOptions)

Runs the loaded model for the given inputs and outputs. Uses the given RunOptions for this run.

Outputs need to be created with correct type and dimension to accept the fetched data.

Declaration

public void Run(IReadOnlyCollection<string> inputNames, IReadOnlyCollection<FixedBufferOnnxValue> inputValues, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<FixedBufferOnnxValue> outputValues, RunOptions options)

Parameters

Type	Name	Description
IReadOnlyCollection<String>	inputNames	Specify a collection of string that indicates the input names. Should match inputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	inputValues	Specify a collection of FixedBufferOnnxValue that indicates the input values.
IReadOnlyCollection<String>	outputNames	Specify a collection of string that indicates the output names. Should match outputValues.
IReadOnlyCollection<FixedBufferOnnxValue>	outputValues	Specify a collection of FixedBufferOnnxValue that indicates the output values.
RunOptions	options

RunAsync(RunOptions, IReadOnlyCollection<String>, IReadOnlyCollection<OrtValue>, IReadOnlyCollection<String>, IReadOnlyCollection<OrtValue>)

Run inference asynchronous in a thread of intra-op thread pool

Declaration

public async Task<IReadOnlyCollection<OrtValue>> RunAsync(RunOptions options, IReadOnlyCollection<string> inputNames, IReadOnlyCollection<OrtValue> inputValues, IReadOnlyCollection<string> outputNames, IReadOnlyCollection<OrtValue> outputValues)

Parameters

Type	Name	Description
RunOptions	options	run option, can be null
IReadOnlyCollection<String>	inputNames	name of inputs
IReadOnlyCollection<OrtValue>	inputValues	input ort values
IReadOnlyCollection<String>	outputNames	name of outputs
IReadOnlyCollection<OrtValue>	outputValues	output of ort values

Returns

Type	Description
Task<IReadOnlyCollection<OrtValue>>	task to be awaited

Exceptions

Type	Condition
OnnxRuntimeException

RunWithBinding(RunOptions, OrtIoBinding)

This method runs inference on the OrtIoBinding instance The method does not return anything. This is a lightweight version of RunWithBindingAndNames(). When you bind pre-allocated buffers to the output values you may not want to fetch the outputs since you already have access to them so you can spare the expense of fetching them and pairing with names. You can still fetch the outputs by calling OrtIOBinding.GetOutputValues()

Declaration

public void RunWithBinding(RunOptions runOptions, OrtIoBinding ioBinding)

Parameters

Type	Name	Description
RunOptions	runOptions	runOptions
OrtIoBinding	ioBinding	ioBinding instance to use

RunWithBindingAndNames(RunOptions, OrtIoBinding, String[])

This method return a collection of DisposableNamedOnnxValue as in other interfaces Query names from OrtIoBinding object and pair then with the array of OrtValues returned from OrtIoBinding.GetOutputValues().

This API will be deprecated in favor of the API that returns a collection of OrtValues.

Declaration

public IDisposableReadOnlyCollection<DisposableNamedOnnxValue> RunWithBindingAndNames(RunOptions runOptions, OrtIoBinding ioBinding, string[] names = null)

Parameters

Type	Name	Description
RunOptions	runOptions	RunOptions
OrtIoBinding	ioBinding	OrtIoBinding instance with bindings
String[]	names	optional parameter. If you already know the names of the outputs you can save a native call to retrieve output names. They will be paired with the returned OrtValues and combined into DisposbleNamedOnnxValues. Otherwise, the method will retrieve output names from the OrtIoBinding instance. It is an error if you supply a different number of names than the returned outputs

Returns

Type	Description
IDisposableReadOnlyCollection<DisposableNamedOnnxValue>	A disposable collection of DisposableNamedOnnxValue that encapsulate output OrtValues

RunWithBoundResults(RunOptions, OrtIoBinding)

This method runs inference on the OrtIoBinding instance. It returns a collection of OrtValues. This method is useful when it is impossible to bind outputs to pre-allocated buffers, because the output shape is not known in advance. In this case, the OrtValues returned by this method are allocated and owned by ORT. The caller is responsible for disposing the collection.

Declaration

public IDisposableReadOnlyCollection<OrtValue> RunWithBoundResults(RunOptions runOptions, OrtIoBinding ioBinding)

Parameters

Type	Name	Description
RunOptions	runOptions	RunOptions
OrtIoBinding	ioBinding	Binding instance

Returns

Type	Description
IDisposableReadOnlyCollection<OrtValue>	A disposable collection of OrtValues

Implements

System.IDisposable