OrtAllocator Struct Reference

Memory allocation interface. More...

#include <onnxruntime_c_api.h>

Public Member Functions
OrtStatus *	GetStats (const struct OrtAllocator *this_, OrtKeyValuePairs **out)
	Function used to get the statistics of the allocator.

Public Attributes
uint32_t	version
	Must be initialized to ORT_API_VERSION.
void *(*	Alloc )(struct OrtAllocator *this_, size_t size)
	Returns a pointer to an allocated block of size bytes.
void(*	Free )(struct OrtAllocator *this_, void *p)
	Free a block of memory previously allocated with OrtAllocator::Alloc.
const struct OrtMemoryInfo *(*	Info )(const struct OrtAllocator *this_)
	Return a pointer to an OrtMemoryInfo that describes this allocator.
void *(*	Reserve )(struct OrtAllocator *this_, size_t size)
	Optional allocation function to use for memory allocations made during session initialization. Use this function if you want to separate allocations made by ORT during Run() calls from those made during session initialization. This allows for separate memory management strategies for these allocations.
void *(*	AllocOnStream )(struct OrtAllocator *this_, size_t size, OrtSyncStream *stream)
	Allocate using a stream.

Detailed Description

Memory allocation interface.

Structure of function pointers that defines a memory allocator. This can be created and filled in by the user for custom allocators.

When an allocator is passed to any function, be sure that the allocator object is not destroyed until the last allocated object using it is freed.

Member Function Documentation

GetStats()

OrtStatus * OrtAllocator::GetStats	(	const struct OrtAllocator *	this_,
		OrtKeyValuePairs **	out
	)

Function used to get the statistics of the allocator.

Return a pointer to the OrtKeyValuePairs structure that contains the statistics of the allocator. The user should call OrtApi::ReleaseKeyValuePairs when done.

Current known keys are:

• Limit: Bytes limit of the allocator. -1 if no limit is set.
• InUse: Number of bytes in use.
• TotalAllocated: The total number of allocated bytes by the allocator.
• MaxInUse: The maximum bytes in use.
• NumAllocs: Number of allocations.
• NumReserves: Number of reserves. (Number of calls to Reserve() in arena-based allocators)
• NumArenaExtensions: Number of arena extensions (Relevant only for arena based allocators)
• NumArenaShrinkages: Number of arena shrinkages (Relevant only for arena based allocators)
• MaxAllocSize: The max single allocation seen.

The allocator is free to add other entries as appropriate.

Note

Implementation of this function is optional and GetStats may be set to a nullptr. If the OrtAllocator is wrapping an internal ORT allocator that does not implement GetStats the returned OrtKeyValuePairs instance will be empty.

Since

1.23

Member Data Documentation

Alloc

void *( * OrtAllocator::Alloc) (struct OrtAllocator *this_, size_t size)

Returns a pointer to an allocated block of size bytes.

AllocOnStream

void *( * OrtAllocator::AllocOnStream) (struct OrtAllocator *this_, size_t size, OrtSyncStream *stream)

Allocate using a stream.

If the allocator is stream aware this performs allocation using a stream.

Alloc will be used if this is nullptr.

Parameters

[in]	this_	OrtAllocator instance
[in]	size	Size of the allocation in bytes. nullptr if size was 0 or allocation failed.
[in]	stream	The stream to allocate on.

Returns

pointer to an allocated block of size bytes

Note

Implementation of this function is optional and AllocOnStream may be set to a nullptr.

Since

1.23

Free

void( * OrtAllocator::Free) (struct OrtAllocator *this_, void *p)

Free a block of memory previously allocated with OrtAllocator::Alloc.

Info

const struct OrtMemoryInfo *( * OrtAllocator::Info) (const struct OrtAllocator *this_)

Return a pointer to an OrtMemoryInfo that describes this allocator.

Reserve

void *( * OrtAllocator::Reserve) (struct OrtAllocator *this_, size_t size)

Optional allocation function to use for memory allocations made during session initialization. Use this function if you want to separate allocations made by ORT during Run() calls from those made during session initialization. This allows for separate memory management strategies for these allocations.

Returns

pointer to an allocated block of size bytes. nullptr if size was 0 or allocation failed.

Since

1.18

version

uint32_t OrtAllocator::version

Must be initialized to ORT_API_VERSION.