Class SessionOptions

Holds the options for creating an InferenceSession It forces the instantiation of the OrtEnv singleton.

Inheritance
Object
CriticalFinalizerObject
SafeHandle
SessionOptions
Implements
IDisposable
Inherited Members
SafeHandle.handle
SafeHandle.Close()
SafeHandle.DangerousAddRef(Boolean)
SafeHandle.DangerousGetHandle()
SafeHandle.DangerousRelease()
SafeHandle.Dispose()
SafeHandle.Dispose(Boolean)
SafeHandle.SetHandle(IntPtr)
SafeHandle.SetHandleAsInvalid()
SafeHandle.IsClosed
Object.Equals(Object)
Object.Equals(Object, Object)
Object.GetHashCode()
Object.GetType()
Object.MemberwiseClone()
Object.ReferenceEquals(Object, Object)
Object.ToString()
Namespace: Microsoft.ML.OnnxRuntime
Assembly: Microsoft.ML.OnnxRuntime.dll
Syntax
public class SessionOptions : SafeHandle, IDisposable

Constructors

| Improve this Doc View Source

SessionOptions()

Constructs an empty SessionOptions

Declaration
public SessionOptions()

Properties

| Improve this Doc View Source

EnableCpuMemArena

Enables Arena allocator for the CPU memory allocations. Default is true.

Declaration
public bool EnableCpuMemArena { get; set; }
Property Value
Type Description
Boolean

returns _enableCpuMemArena flag value
| Improve this Doc View Source

EnableMemoryPattern

Enables the use of the memory allocation patterns in the first Run() call for subsequent runs. Default = true.

Declaration
public bool EnableMemoryPattern { get; set; }
Property Value
Type Description
Boolean

returns enableMemoryPattern flag value
| Improve this Doc View Source

EnableProfiling

Enables profiling of InferenceSession.Run() calls. Default is false

Declaration
public bool EnableProfiling { get; set; }
Property Value
Type Description
Boolean

returns _enableProfiling flag value
| Improve this Doc View Source

ExecutionMode

Sets the execution mode for the session. Default is set to ORT_SEQUENTIAL. See [ONNX_Runtime_Perf_Tuning.md] for more details.

Declaration
public ExecutionMode ExecutionMode { get; set; }
Property Value
Type Description
ExecutionMode

returns _executionMode value
| Improve this Doc View Source

GraphOptimizationLevel

Sets the graph optimization level for the session. Default is set to ORT_ENABLE_ALL.

Declaration
public GraphOptimizationLevel GraphOptimizationLevel { get; set; }
Property Value
Type Description
GraphOptimizationLevel

returns _graphOptimizationLevel value
| Improve this Doc View Source

InterOpNumThreads

Sets the number of threads used to parallelize the execution of the graph (across nodes) If sequential execution is enabled this value is ignored A value of 0 means ORT will pick a default

Declaration
public int InterOpNumThreads { get; set; }
Property Value
Type Description
Int32

returns _interOpNumThreads value
| Improve this Doc View Source

IntraOpNumThreads

Sets the number of threads used to parallelize the execution within nodes A value of 0 means ORT will pick a default

Declaration
public int IntraOpNumThreads { get; set; }
Property Value
Type Description
Int32

returns _intraOpNumThreads value
| Improve this Doc View Source

IsInvalid

Overrides SafeHandle.IsInvalid

Declaration
public override bool IsInvalid { get; }
Property Value
Type Description
Boolean

returns true if handle is equal to Zero
Overrides
System.Runtime.InteropServices.SafeHandle.IsInvalid
| Improve this Doc View Source

LogId

Log Id to be used for the session. Default is empty string.

Declaration
public string LogId { get; set; }
Property Value
Type Description
String

returns _logId value
| Improve this Doc View Source

LogSeverityLevel

Log Severity Level for the session logs. Default = ORT_LOGGING_LEVEL_WARNING

Declaration
public OrtLoggingLevel LogSeverityLevel { get; set; }
Property Value
Type Description
OrtLoggingLevel

returns _logSeverityLevel value
| Improve this Doc View Source

LogVerbosityLevel

Log Verbosity Level for the session logs. Default = 0. Valid values are >=0. This takes into effect only when the LogSeverityLevel is set to ORT_LOGGING_LEVEL_VERBOSE.

Declaration
public int LogVerbosityLevel { get; set; }
Property Value
Type Description
Int32

returns _logVerbosityLevel value
| Improve this Doc View Source

OptimizedModelFilePath

Set filepath to save optimized model after graph level transformations. Default is empty, which implies saving is disabled.

Declaration
public string OptimizedModelFilePath { get; set; }
Property Value
Type Description
String

returns _optimizedModelFilePath flag value
| Improve this Doc View Source

ProfileOutputPathPrefix

Path prefix to use for output of profiling data

Declaration
public string ProfileOutputPathPrefix { get; set; }
Property Value
Type Description
String

Methods

| Improve this Doc View Source

AddFreeDimensionOverride(String, Int64)

Override symbolic dimensions (by specific denotation strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)

Declaration
public void AddFreeDimensionOverride(string dimDenotation, long dimValue)
Parameters
Type Name Description
String dimDenotation

denotation name
Int64 dimValue

denotation value
| Improve this Doc View Source

AddFreeDimensionOverrideByName(String, Int64)

Override symbolic dimensions (by specific name strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)

Declaration
public void AddFreeDimensionOverrideByName(string dimName, long dimValue)
Parameters
Type Name Description
String dimName

dimension name
Int64 dimValue

dimension value
| Improve this Doc View Source

AddInitializer(String, OrtValue)

Add a pre-allocated initializer to a session. If a model contains an initializer with a name that is same as the name passed to this API call, ORT will use this initializer instance instead of deserializing one from the model file. This is useful when you want to share the same initializer across sessions.

Declaration
public void AddInitializer(string name, OrtValue ortValue)
Parameters
Type Name Description
String name

name of the initializer
OrtValue ortValue

OrtValue containing the initializer. Lifetime of 'val' and the underlying initializer buffer must be managed by the user (created using the CreateTensorWithDataAsOrtValue API) and it must outlive the session object
| Improve this Doc View Source

AddSessionConfigEntry(String, String)

Set a single session configuration entry as a pair of strings If a configuration with same key exists, this will overwrite the configuration with the given configValue

Declaration
public void AddSessionConfigEntry(string configKey, string configValue)
Parameters
Type Name Description
String configKey

config key name
String configValue

config key value
| Improve this Doc View Source

AppendExecutionProvider(String, Dictionary<String, String>)

Append QNN, SNPE or XNNPACK execution provider

Declaration
public void AppendExecutionProvider(string providerName, Dictionary<string, string> providerOptions = null)
Parameters
Type Name Description
String providerName

Execution provider to add. 'QNN', 'SNPE' 'XNNPACK', 'CoreML and 'AZURE are currently supported.
Dictionary<String, String> providerOptions

Optional key/value pairs to specify execution provider options.
| Improve this Doc View Source

AppendExecutionProvider_CoreML(CoreMLFlags)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_CoreML(CoreMLFlags coremlFlags = CoreMLFlags.COREML_FLAG_USE_NONE)
Parameters
Type Name Description
CoreMLFlags coremlFlags

CoreML specific flags
| Improve this Doc View Source

AppendExecutionProvider_CPU(Int32)

Appends CPU EP to a list of available execution providers for the session.

Declaration
public void AppendExecutionProvider_CPU(int useArena = 1)
Parameters
Type Name Description
Int32 useArena

1 - use arena, 0 - do not use arena
| Improve this Doc View Source

AppendExecutionProvider_CUDA(OrtCUDAProviderOptions)

Append a CUDA EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_CUDA(OrtCUDAProviderOptions cudaProviderOptions)
Parameters
Type Name Description
OrtCUDAProviderOptions cudaProviderOptions

CUDA EP provider options
| Improve this Doc View Source

AppendExecutionProvider_CUDA(Int32)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_CUDA(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId

integer device ID
| Improve this Doc View Source

AppendExecutionProvider_DML(Int32)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_DML(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId

device identification
| Improve this Doc View Source

AppendExecutionProvider_Dnnl(Int32)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_Dnnl(int useArena = 1)
Parameters
Type Name Description
Int32 useArena

1 - use allocation arena, 0 - otherwise
| Improve this Doc View Source

AppendExecutionProvider_MIGraphX(Int32)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_MIGraphX(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId

device identification
| Improve this Doc View Source

AppendExecutionProvider_Nnapi(NnapiFlags)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_Nnapi(NnapiFlags nnapiFlags = NnapiFlags.NNAPI_FLAG_USE_NONE)
Parameters
Type Name Description
NnapiFlags nnapiFlags

NNAPI specific flag mask
| Improve this Doc View Source

AppendExecutionProvider_OpenVINO(String)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_OpenVINO(string deviceId = "")
Parameters
Type Name Description
String deviceId

device identification, default empty string
| Improve this Doc View Source

AppendExecutionProvider_ROCm(OrtROCMProviderOptions)

Append a ROCm EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_ROCm(OrtROCMProviderOptions rocmProviderOptions)
Parameters
Type Name Description
OrtROCMProviderOptions rocmProviderOptions

ROCm EP provider options
| Improve this Doc View Source

AppendExecutionProvider_ROCm(Int32)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_ROCm(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId

Device Id
| Improve this Doc View Source

AppendExecutionProvider_Tensorrt(OrtTensorRTProviderOptions)

Append a TensorRT EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_Tensorrt(OrtTensorRTProviderOptions trtProviderOptions)
Parameters
Type Name Description
OrtTensorRTProviderOptions trtProviderOptions

TensorRT EP provider options
| Improve this Doc View Source

AppendExecutionProvider_Tensorrt(Int32)

Use only if you have the onnxruntime package specific to this Execution Provider.

Declaration
public void AppendExecutionProvider_Tensorrt(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId

device identification
| Improve this Doc View Source

DisablePerSessionThreads()

Disables the per session threads. Default is true. This makes all sessions in the process use a global TP.

Declaration
public void DisablePerSessionThreads()
| Improve this Doc View Source

MakeSessionOptionWithCudaProvider(OrtCUDAProviderOptions)

A helper method to construct a SessionOptions object for CUDA execution provider. Use only if CUDA is installed and you have the onnxruntime package specific to this Execution Provider.

Declaration
public static SessionOptions MakeSessionOptionWithCudaProvider(OrtCUDAProviderOptions cudaProviderOptions)
Parameters
Type Name Description
OrtCUDAProviderOptions cudaProviderOptions

CUDA EP provider options
Returns
Type Description
SessionOptions

A SessionsOptions() object configured for execution on provider options
| Improve this Doc View Source

MakeSessionOptionWithCudaProvider(Int32)

A helper method to construct a SessionOptions object for CUDA execution. Use only if CUDA is installed and you have the onnxruntime package specific to this Execution Provider.

Declaration
public static SessionOptions MakeSessionOptionWithCudaProvider(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId
Returns
Type Description
SessionOptions

A SessionsOptions() object configured for execution on deviceId
| Improve this Doc View Source

MakeSessionOptionWithRocmProvider(OrtROCMProviderOptions)

A helper method to construct a SessionOptions object for ROCm execution provider. Use only if ROCm is installed and you have the onnxruntime package specific to this Execution Provider.

Declaration
public static SessionOptions MakeSessionOptionWithRocmProvider(OrtROCMProviderOptions rocmProviderOptions)
Parameters
Type Name Description
OrtROCMProviderOptions rocmProviderOptions

ROCm EP provider options
Returns
Type Description
SessionOptions

A SessionsOptions() object configured for execution on provider options
| Improve this Doc View Source

MakeSessionOptionWithRocmProvider(Int32)

A helper method to construct a SessionOptions object for ROCM execution. Use only if ROCM is installed and you have the onnxruntime package specific to this Execution Provider.

Declaration
public static SessionOptions MakeSessionOptionWithRocmProvider(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId

Device Id
Returns
Type Description
SessionOptions

A SessionsOptions() object configured for execution on deviceId
| Improve this Doc View Source

MakeSessionOptionWithTensorrtProvider(OrtTensorRTProviderOptions)

A helper method to construct a SessionOptions object for TensorRT execution provider. Use only if CUDA/TensorRT are installed and you have the onnxruntime package specific to this Execution Provider.

Declaration
public static SessionOptions MakeSessionOptionWithTensorrtProvider(OrtTensorRTProviderOptions trtProviderOptions)
Parameters
Type Name Description
OrtTensorRTProviderOptions trtProviderOptions

TensorRT EP provider options
Returns
Type Description
SessionOptions

A SessionsOptions() object configured for execution on provider options
| Improve this Doc View Source

MakeSessionOptionWithTensorrtProvider(Int32)

A helper method to construct a SessionOptions object for TensorRT execution. Use only if CUDA/TensorRT are installed and you have the onnxruntime package specific to this Execution Provider.

Declaration
public static SessionOptions MakeSessionOptionWithTensorrtProvider(int deviceId = 0)
Parameters
Type Name Description
Int32 deviceId
Returns
Type Description
SessionOptions

A SessionsOptions() object configured for execution on deviceId
| Improve this Doc View Source

RegisterCustomOpLibrary(String)

Loads a DLL named 'libraryPath' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions* options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base.

Prior to v1.15 this leaked the library handle and RegisterCustomOpLibraryV2 was added to resolve that.

From v1.15 on ONNX Runtime will manage the lifetime of the handle.

Declaration
public void RegisterCustomOpLibrary(string libraryPath)
Parameters
Type Name Description
String libraryPath

path to the custom op library
| Improve this Doc View Source

RegisterCustomOpLibraryV2(String, out IntPtr)

Loads a DLL named 'libraryPath' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions* options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base. The handle to the loaded library is returned in 'libraryHandle'. It can be unloaded by the caller after all sessions using the passed in session options are destroyed, or if an error occurs and it is non null. Hint: .NET Core 3.1 has a 'NativeLibrary' class that can be used to free the library handle

Declaration
public void RegisterCustomOpLibraryV2(string libraryPath, out IntPtr libraryHandle)
Parameters
Type Name Description
String libraryPath

Custom op library path
IntPtr libraryHandle

out parameter, library handle
| Improve this Doc View Source

RegisterOrtExtensions()

Register the custom operators from the Microsoft.ML.OnnxRuntime.Extensions NuGet package. A reference to Microsoft.ML.OnnxRuntime.Extensions must be manually added to your project.

Declaration
public void RegisterOrtExtensions()
Exceptions
Type Condition
OnnxRuntimeException

Throws if the extensions library is not found.
| Improve this Doc View Source

ReleaseHandle()

Overrides SafeHandle.ReleaseHandle() to properly dispose of the native instance of SessionOptions

Declaration
protected override bool ReleaseHandle()
Returns
Type Description
Boolean

always returns true
Overrides
System.Runtime.InteropServices.SafeHandle.ReleaseHandle()
| Improve this Doc View Source

SetLoadCancellationFlag(Boolean)

Sets the load cancellation flag for the session. Default is set to false. Provides an opportunity for the user to cancel model loading.

Declaration
public void SetLoadCancellationFlag(bool value)
Parameters
Type Name Description
Boolean value

true to request cancellation, false to proceed

Implements

System.IDisposable

Extension Methods