Class OrtMemoryInfo

This class encapsulates and most of the time owns the underlying native OrtMemoryInfo instance. Instance returned from OrtAllocator will not own OrtMemoryInfo, the class must be disposed regardless.

Use this class to query and create OrtAllocator instances so you can pre-allocate memory for model inputs/outputs and use it for binding. Instances of the class can also used to created OrtValues bound to pre-allocated memory. In that case, the instance of OrtMemoryInfo contains the information about the allocator used to allocate the underlying memory.

Inheritance

object

CriticalFinalizerObject

SafeHandle

OrtMemoryInfo

Implements

IDisposable

Inherited Members

SafeHandle.handle

SafeHandle.Close()

SafeHandle.DangerousAddRef(ref bool)

SafeHandle.DangerousGetHandle()

SafeHandle.DangerousRelease()

SafeHandle.Dispose()

SafeHandle.Dispose(bool)

SafeHandle.SetHandle(nint)

SafeHandle.SetHandleAsInvalid()

SafeHandle.IsClosed

object.Equals(object, object)

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Microsoft.ML.OnnxRuntime

Assembly: Microsoft.ML.OnnxRuntime.dll

Syntax

public class OrtMemoryInfo : SafeHandle, IDisposable

Constructors

| Improve this Doc View Source

OrtMemoryInfo(byte[], OrtAllocatorType, int, OrtMemType)

Create an instance of OrtMemoryInfo according to the specification Memory info instances are usually used to get a handle of a native allocator that is present within the current inference session object. That, in turn, depends of what execution providers are available within the binary that you are using and are registered with Add methods.

Declaration

public OrtMemoryInfo(byte[] utf8AllocatorName, OrtAllocatorType allocatorType, int deviceId, OrtMemType memoryType)

Parameters

Type	Name	Description
byte[]	utf8AllocatorName	Allocator name. Use of the predefined above.
OrtAllocatorType	allocatorType	Allocator type
int	deviceId	Device id
OrtMemType	memoryType	Memory type

| Improve this Doc View Source

OrtMemoryInfo(string, OrtAllocatorType, int, OrtMemType)

Create an instance of OrtMemoryInfo according to the specification.

Declaration

public OrtMemoryInfo(string allocatorName, OrtAllocatorType allocatorType, int deviceId, OrtMemType memoryType)

Parameters

Type	Name	Description
string	allocatorName	Allocator name
OrtAllocatorType	allocatorType	Allocator type
int	deviceId	Device id
OrtMemType	memoryType	Memory type

| Improve this Doc View Source

OrtMemoryInfo(string, OrtMemoryInfoDeviceType, uint, int, OrtDeviceMemoryType, ulong, OrtAllocatorType)

Creates an instance of OrtMemoryInfo using OrtCreateMemoryInfoV2

Declaration

public OrtMemoryInfo(string allocatorName, OrtMemoryInfoDeviceType deviceType, uint vendorId, int deviceId, OrtDeviceMemoryType deviceMemoryType, ulong alignment, OrtAllocatorType allocatorType)

Parameters

Type	Name	Description
string	allocatorName	In this overload this is an arbitrary name
OrtMemoryInfoDeviceType	deviceType	Device Type
uint	vendorId	Vendor Id
int	deviceId	Device Id
OrtDeviceMemoryType	deviceMemoryType	Device Memory Type
ulong	alignment	Alignment is required or 0
OrtAllocatorType	allocatorType	Allocator Type

Fields

| Improve this Doc View Source

allocatorCPU

Predefined utf8 encoded allocator names. Use them to construct an instance of OrtMemoryInfo to avoid UTF-16 to UTF-8 conversion costs.

Declaration

public static readonly byte[] allocatorCPU

Field Value

Type	Description
byte[]

| Improve this Doc View Source

allocatorCUDA

Predefined utf8 encoded allocator names. Use them to construct an instance of OrtMemoryInfo to avoid UTF-16 to UTF-8 conversion costs.

Declaration

public static readonly byte[] allocatorCUDA

Field Value

Type	Description
byte[]

| Improve this Doc View Source

allocatorCUDA_PINNED

Predefined utf8 encoded allocator names. Use them to construct an instance of OrtMemoryInfo to avoid UTF-16 to UTF-8 conversion costs.

Declaration

public static readonly byte[] allocatorCUDA_PINNED

Field Value

Type	Description
byte[]

| Improve this Doc View Source

allocatorHIP

Predefined utf8 encoded allocator names. Use them to construct an instance of OrtMemoryInfo to avoid UTF-16 to UTF-8 conversion costs.

Declaration

public static readonly byte[] allocatorHIP

Field Value

Type	Description
byte[]

| Improve this Doc View Source

allocatorHIP_PINNED

Predefined utf8 encoded allocator names. Use them to construct an instance of OrtMemoryInfo to avoid UTF-16 to UTF-8 conversion costs.

Declaration

public static readonly byte[] allocatorHIP_PINNED

Field Value

Type	Description
byte[]

Properties

| Improve this Doc View Source

DefaultInstance

Default CPU based instance

Declaration

public static OrtMemoryInfo DefaultInstance { get; }

Property Value

Type	Description
OrtMemoryInfo	Singleton instance of a CpuMemoryInfo

| Improve this Doc View Source

Id

Returns device ID

Declaration

public int Id { get; }

Property Value

Type	Description
int	returns integer Id value

| Improve this Doc View Source

IsInvalid

Overrides SafeHandle.IsInvalid

Declaration

public override bool IsInvalid { get; }

Property Value

Type	Description
bool	returns true if handle is equal to Zero

Overrides

SafeHandle.IsInvalid

| Improve this Doc View Source

Name

Name of the allocator associated with the OrtMemoryInfo instance

Declaration

public string Name { get; }

Property Value

Type	Description
string

Methods

| Improve this Doc View Source

Equals(OrtMemoryInfo)

Compares this instance with another

Declaration

public bool Equals(OrtMemoryInfo other)

Parameters

Type	Name	Description
OrtMemoryInfo	other	OrtMemoryInfo to compare to

Returns

Type	Description
bool	true if instances are equal according to OrtCompareMemoryInfo.

| Improve this Doc View Source

Equals(object)

Overrides System.Object.Equals(object)

Declaration

public override bool Equals(object obj)

Parameters

Type	Name	Description
object	obj	object to compare to

Returns

Type	Description
bool	true if obj is an instance of OrtMemoryInfo and is equal to this

Overrides

object.Equals(object)

| Improve this Doc View Source

GetAllocatorType()

Fetches allocator type from the underlying OrtAllocator

Declaration

public OrtAllocatorType GetAllocatorType()

Returns

Type	Description
OrtAllocatorType	Returns allocator type

| Improve this Doc View Source

GetDeviceMemoryType()

Return the device memory type associated with this memory info

Declaration

public OrtDeviceMemoryType GetDeviceMemoryType()

Returns

Type	Description
OrtDeviceMemoryType	OrtDeviceMemoryType for the device

| Improve this Doc View Source

GetHashCode()

Overrides System.Object.GetHashCode()

Declaration

public override int GetHashCode()

Returns

Type	Description
int	integer hash value

Overrides

object.GetHashCode()

| Improve this Doc View Source

GetMemoryType()

The below 2 are really properties but naming them is a challenge as names would conflict with the returned type. Also, there are native calls behind them so exposing them as Get() would be appropriate.

Declaration

public OrtMemType GetMemoryType()

Returns

Type	Description
OrtMemType	OrtMemoryType for the instance

| Improve this Doc View Source

GetVendorId()

Fetches vendor ID

Declaration

public uint GetVendorId()

Returns

Type	Description
uint	uint32_t

| Improve this Doc View Source

ReleaseHandle()

Overrides SafeHandle.ReleaseHandle() to properly dispose of the native instance of OrtMmeoryInfo

Declaration

protected override bool ReleaseHandle()

Returns

Type	Description
bool	always returns true

Overrides

SafeHandle.ReleaseHandle()

Implements

IDisposable