Class SessionOptions
Holds the options for creating an InferenceSession It forces the instantiation of the OrtEnv singleton.
Inheritance
Implements
Inherited Members
Namespace: Microsoft.ML.OnnxRuntime
Assembly: Microsoft.ML.OnnxRuntime.dll
Syntax
public class SessionOptions : SafeHandle, IDisposable
Constructors
| Improve this Doc View SourceSessionOptions()
Constructs an empty SessionOptions
Declaration
public SessionOptions()
Properties
| Improve this Doc View SourceEnableCpuMemArena
Enables Arena allocator for the CPU memory allocations. Default is true.
Declaration
public bool EnableCpuMemArena { get; set; }
Property Value
Type | Description |
---|---|
Boolean | returns _enableCpuMemArena flag value |
EnableMemoryPattern
Enables the use of the memory allocation patterns in the first Run() call for subsequent runs. Default = true.
Declaration
public bool EnableMemoryPattern { get; set; }
Property Value
Type | Description |
---|---|
Boolean | returns enableMemoryPattern flag value |
EnableProfiling
Enables profiling of InferenceSession.Run() calls. Default is false
Declaration
public bool EnableProfiling { get; set; }
Property Value
Type | Description |
---|---|
Boolean | returns _enableProfiling flag value |
ExecutionMode
Sets the execution mode for the session. Default is set to ORT_SEQUENTIAL. See [ONNX_Runtime_Perf_Tuning.md] for more details.
Declaration
public ExecutionMode ExecutionMode { get; set; }
Property Value
Type | Description |
---|---|
ExecutionMode | returns _executionMode value |
GraphOptimizationLevel
Sets the graph optimization level for the session. Default is set to ORT_ENABLE_ALL.
Declaration
public GraphOptimizationLevel GraphOptimizationLevel { get; set; }
Property Value
Type | Description |
---|---|
GraphOptimizationLevel | returns _graphOptimizationLevel value |
InterOpNumThreads
Sets the number of threads used to parallelize the execution of the graph (across nodes) If sequential execution is enabled this value is ignored A value of 0 means ORT will pick a default
Declaration
public int InterOpNumThreads { get; set; }
Property Value
Type | Description |
---|---|
Int32 | returns _interOpNumThreads value |
IntraOpNumThreads
Sets the number of threads used to parallelize the execution within nodes A value of 0 means ORT will pick a default
Declaration
public int IntraOpNumThreads { get; set; }
Property Value
Type | Description |
---|---|
Int32 | returns _intraOpNumThreads value |
IsInvalid
Overrides SafeHandle.IsInvalid
Declaration
public override bool IsInvalid { get; }
Property Value
Type | Description |
---|---|
Boolean | returns true if handle is equal to Zero |
Overrides
LogId
Log Id to be used for the session. Default is empty string.
Declaration
public string LogId { get; set; }
Property Value
Type | Description |
---|---|
String | returns _logId value |
LogSeverityLevel
Log Severity Level for the session logs. Default = ORT_LOGGING_LEVEL_WARNING
Declaration
public OrtLoggingLevel LogSeverityLevel { get; set; }
Property Value
Type | Description |
---|---|
OrtLoggingLevel | returns _logSeverityLevel value |
LogVerbosityLevel
Log Verbosity Level for the session logs. Default = 0. Valid values are >=0. This takes into effect only when the LogSeverityLevel is set to ORT_LOGGING_LEVEL_VERBOSE.
Declaration
public int LogVerbosityLevel { get; set; }
Property Value
Type | Description |
---|---|
Int32 | returns _logVerbosityLevel value |
OptimizedModelFilePath
Set filepath to save optimized model after graph level transformations. Default is empty, which implies saving is disabled.
Declaration
public string OptimizedModelFilePath { get; set; }
Property Value
Type | Description |
---|---|
String | returns _optimizedModelFilePath flag value |
ProfileOutputPathPrefix
Path prefix to use for output of profiling data
Declaration
public string ProfileOutputPathPrefix { get; set; }
Property Value
Type | Description |
---|---|
String |
Methods
| Improve this Doc View SourceAddFreeDimensionOverride(String, Int64)
Override symbolic dimensions (by specific denotation strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)
Declaration
public void AddFreeDimensionOverride(string dimDenotation, long dimValue)
Parameters
Type | Name | Description |
---|---|---|
String | dimDenotation | denotation name |
Int64 | dimValue | denotation value |
AddFreeDimensionOverrideByName(String, Int64)
Override symbolic dimensions (by specific name strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)
Declaration
public void AddFreeDimensionOverrideByName(string dimName, long dimValue)
Parameters
Type | Name | Description |
---|---|---|
String | dimName | dimension name |
Int64 | dimValue | dimension value |
AddInitializer(String, OrtValue)
Add a pre-allocated initializer to a session. If a model contains an initializer with a name that is same as the name passed to this API call, ORT will use this initializer instance instead of deserializing one from the model file. This is useful when you want to share the same initializer across sessions.
Declaration
public void AddInitializer(string name, OrtValue ortValue)
Parameters
Type | Name | Description |
---|---|---|
String | name | name of the initializer |
OrtValue | ortValue | OrtValue containing the initializer. Lifetime of 'val' and the underlying initializer buffer must be managed by the user (created using the CreateTensorWithDataAsOrtValue API) and it must outlive the session object |
AddSessionConfigEntry(String, String)
Set a single session configuration entry as a pair of strings If a configuration with same key exists, this will overwrite the configuration with the given configValue
Declaration
public void AddSessionConfigEntry(string configKey, string configValue)
Parameters
Type | Name | Description |
---|---|---|
String | configKey | config key name |
String | configValue | config key value |
AppendExecutionProvider(String, Dictionary<String, String>)
Append QNN, SNPE or XNNPACK execution provider
Declaration
public void AppendExecutionProvider(string providerName, Dictionary<string, string> providerOptions = null)
Parameters
Type | Name | Description |
---|---|---|
String | providerName | Execution provider to add. 'QNN', 'SNPE' 'XNNPACK', 'CoreML and 'AZURE are currently supported. |
Dictionary<String, String> | providerOptions | Optional key/value pairs to specify execution provider options. |
AppendExecutionProvider_CoreML(CoreMLFlags)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_CoreML(CoreMLFlags coremlFlags = CoreMLFlags.COREML_FLAG_USE_NONE)
Parameters
Type | Name | Description |
---|---|---|
CoreMLFlags | coremlFlags | CoreML specific flags |
AppendExecutionProvider_CPU(Int32)
Appends CPU EP to a list of available execution providers for the session.
Declaration
public void AppendExecutionProvider_CPU(int useArena = 1)
Parameters
Type | Name | Description |
---|---|---|
Int32 | useArena | 1 - use arena, 0 - do not use arena |
AppendExecutionProvider_CUDA(OrtCUDAProviderOptions)
Append a CUDA EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_CUDA(OrtCUDAProviderOptions cudaProviderOptions)
Parameters
Type | Name | Description |
---|---|---|
OrtCUDAProviderOptions | cudaProviderOptions | CUDA EP provider options |
AppendExecutionProvider_CUDA(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_CUDA(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId | integer device ID |
AppendExecutionProvider_DML(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_DML(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId | device identification |
AppendExecutionProvider_Dnnl(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Dnnl(int useArena = 1)
Parameters
Type | Name | Description |
---|---|---|
Int32 | useArena | 1 - use allocation arena, 0 - otherwise |
AppendExecutionProvider_MIGraphX(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_MIGraphX(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId | device identification |
AppendExecutionProvider_Nnapi(NnapiFlags)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Nnapi(NnapiFlags nnapiFlags = NnapiFlags.NNAPI_FLAG_USE_NONE)
Parameters
Type | Name | Description |
---|---|---|
NnapiFlags | nnapiFlags | NNAPI specific flag mask |
AppendExecutionProvider_OpenVINO(String)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_OpenVINO(string deviceId = "")
Parameters
Type | Name | Description |
---|---|---|
String | deviceId | device identification, default empty string |
AppendExecutionProvider_ROCm(OrtROCMProviderOptions)
Append a ROCm EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_ROCm(OrtROCMProviderOptions rocmProviderOptions)
Parameters
Type | Name | Description |
---|---|---|
OrtROCMProviderOptions | rocmProviderOptions | ROCm EP provider options |
AppendExecutionProvider_ROCm(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_ROCm(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId | Device Id |
AppendExecutionProvider_Tensorrt(OrtTensorRTProviderOptions)
Append a TensorRT EP instance (based on specified configuration) to the SessionOptions instance. Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Tensorrt(OrtTensorRTProviderOptions trtProviderOptions)
Parameters
Type | Name | Description |
---|---|---|
OrtTensorRTProviderOptions | trtProviderOptions | TensorRT EP provider options |
AppendExecutionProvider_Tensorrt(Int32)
Use only if you have the onnxruntime package specific to this Execution Provider.
Declaration
public void AppendExecutionProvider_Tensorrt(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId | device identification |
DisablePerSessionThreads()
Disables the per session threads. Default is true. This makes all sessions in the process use a global TP.
Declaration
public void DisablePerSessionThreads()
MakeSessionOptionWithCudaProvider(OrtCUDAProviderOptions)
A helper method to construct a SessionOptions object for CUDA execution provider. Use only if CUDA is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithCudaProvider(OrtCUDAProviderOptions cudaProviderOptions)
Parameters
Type | Name | Description |
---|---|---|
OrtCUDAProviderOptions | cudaProviderOptions | CUDA EP provider options |
Returns
Type | Description |
---|---|
SessionOptions | A SessionsOptions() object configured for execution on provider options |
MakeSessionOptionWithCudaProvider(Int32)
A helper method to construct a SessionOptions object for CUDA execution. Use only if CUDA is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithCudaProvider(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId |
Returns
Type | Description |
---|---|
SessionOptions | A SessionsOptions() object configured for execution on deviceId |
MakeSessionOptionWithRocmProvider(OrtROCMProviderOptions)
A helper method to construct a SessionOptions object for ROCm execution provider. Use only if ROCm is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithRocmProvider(OrtROCMProviderOptions rocmProviderOptions)
Parameters
Type | Name | Description |
---|---|---|
OrtROCMProviderOptions | rocmProviderOptions | ROCm EP provider options |
Returns
Type | Description |
---|---|
SessionOptions | A SessionsOptions() object configured for execution on provider options |
MakeSessionOptionWithRocmProvider(Int32)
A helper method to construct a SessionOptions object for ROCM execution. Use only if ROCM is installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithRocmProvider(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId | Device Id |
Returns
Type | Description |
---|---|
SessionOptions | A SessionsOptions() object configured for execution on deviceId |
MakeSessionOptionWithTensorrtProvider(OrtTensorRTProviderOptions)
A helper method to construct a SessionOptions object for TensorRT execution provider. Use only if CUDA/TensorRT are installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithTensorrtProvider(OrtTensorRTProviderOptions trtProviderOptions)
Parameters
Type | Name | Description |
---|---|---|
OrtTensorRTProviderOptions | trtProviderOptions | TensorRT EP provider options |
Returns
Type | Description |
---|---|
SessionOptions | A SessionsOptions() object configured for execution on provider options |
MakeSessionOptionWithTensorrtProvider(Int32)
A helper method to construct a SessionOptions object for TensorRT execution. Use only if CUDA/TensorRT are installed and you have the onnxruntime package specific to this Execution Provider.
Declaration
public static SessionOptions MakeSessionOptionWithTensorrtProvider(int deviceId = 0)
Parameters
Type | Name | Description |
---|---|---|
Int32 | deviceId |
Returns
Type | Description |
---|---|
SessionOptions | A SessionsOptions() object configured for execution on deviceId |
RegisterCustomOpLibrary(String)
Loads a DLL named 'libraryPath' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions* options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base.
Prior to v1.15 this leaked the library handle and RegisterCustomOpLibraryV2 was added to resolve that.
From v1.15 on ONNX Runtime will manage the lifetime of the handle.
Declaration
public void RegisterCustomOpLibrary(string libraryPath)
Parameters
Type | Name | Description |
---|---|---|
String | libraryPath | path to the custom op library |
RegisterCustomOpLibraryV2(String, out IntPtr)
Loads a DLL named 'libraryPath' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions* options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base. The handle to the loaded library is returned in 'libraryHandle'. It can be unloaded by the caller after all sessions using the passed in session options are destroyed, or if an error occurs and it is non null. Hint: .NET Core 3.1 has a 'NativeLibrary' class that can be used to free the library handle
Declaration
public void RegisterCustomOpLibraryV2(string libraryPath, out IntPtr libraryHandle)
Parameters
Type | Name | Description |
---|---|---|
String | libraryPath | Custom op library path |
IntPtr | libraryHandle | out parameter, library handle |
RegisterOrtExtensions()
Register the custom operators from the Microsoft.ML.OnnxRuntime.Extensions NuGet package. A reference to Microsoft.ML.OnnxRuntime.Extensions must be manually added to your project.
Declaration
public void RegisterOrtExtensions()
Exceptions
Type | Condition |
---|---|
OnnxRuntimeException | Throws if the extensions library is not found. |
ReleaseHandle()
Overrides SafeHandle.ReleaseHandle() to properly dispose of the native instance of SessionOptions
Declaration
protected override bool ReleaseHandle()
Returns
Type | Description |
---|---|
Boolean | always returns true |