Struct that an EP implements for external resource import (memory + semaphore import). More...

#include <onnxruntime_ep_c_api.h>

Public Member Functions
OrtStatus *	ImportMemory (OrtExternalResourceImporterImpl this_ptr, const OrtExternalMemoryDescriptor desc, OrtExternalMemoryHandle **out_handle)
	Import external memory.

OrtStatus *	CreateTensorFromMemory (OrtExternalResourceImporterImpl this_ptr, const OrtExternalMemoryHandle mem_handle, const OrtExternalTensorDescriptor tensor_desc, OrtValue *out_tensor)
	Create a tensor backed by imported external memory.

OrtStatus *	ImportSemaphore (OrtExternalResourceImporterImpl this_ptr, const OrtExternalSemaphoreDescriptor desc, OrtExternalSemaphoreHandle **out_handle)
	Import an external semaphore.

OrtStatus *	WaitSemaphore (OrtExternalResourceImporterImpl this_ptr, OrtExternalSemaphoreHandle handle, OrtSyncStream *stream, uint64_t value)
	Wait on an external semaphore on the EP's stream.

OrtStatus *	SignalSemaphore (OrtExternalResourceImporterImpl this_ptr, OrtExternalSemaphoreHandle handle, OrtSyncStream *stream, uint64_t value)
	Signal an external semaphore from the EP's stream.

Public Attributes
uint32_t	ort_version_supported
	Must be initialized to ORT_API_VERSION.

bool(*	CanImportMemory )(const OrtExternalResourceImporterImpl *this_ptr, OrtExternalMemoryHandleType handle_type)
	Check if the implementation can import external memory of the given handle type.

void(*	ReleaseMemory )(OrtExternalResourceImporterImpl this_ptr, OrtExternalMemoryHandle handle)
	Release an imported external memory handle.

bool(*	CanImportSemaphore )(const OrtExternalResourceImporterImpl *this_ptr, OrtExternalSemaphoreType type)
	Check if the implementation can import external semaphores of the given type.

void(*	ReleaseSemaphore )(OrtExternalResourceImporterImpl this_ptr, OrtExternalSemaphoreHandle handle)
	Release an imported external semaphore handle.

void(*	Release )(OrtExternalResourceImporterImpl *this_ptr)
	Release the OrtExternalResourceImporterImpl instance.

Detailed Description

Struct that an EP implements for external resource import (memory + semaphore import).

This capability object provides methods for importing external GPU memory and semaphores for zero-copy import. EPs that support D3D12, CUDA, HIP, or Vulkan external resource APIs can implement this interface.

Since: Version 1.24.

Member Function Documentation

◆ CreateTensorFromMemory()

OrtStatus * OrtExternalResourceImporterImpl::CreateTensorFromMemory	(	OrtExternalResourceImporterImpl *	this_ptr,
		const OrtExternalMemoryHandle *	mem_handle,
		const OrtExternalTensorDescriptor *	tensor_desc,
		OrtValue **	out_tensor
	)

Create a tensor backed by imported external memory.

The created tensor is a view over the imported memory and does not copy data.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	mem_handle	The imported external memory handle (EP casts to its derived type).
[in]	tensor_desc	Descriptor specifying tensor element type, shape, and optional offset.
[out]	out_tensor	Output parameter set to the created OrtValue containing the tensor.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ ImportMemory()

OrtStatus * OrtExternalResourceImporterImpl::ImportMemory	(	OrtExternalResourceImporterImpl *	this_ptr,
		const OrtExternalMemoryDescriptor *	desc,
		OrtExternalMemoryHandle **	out_handle
	)

Import external memory.

The EP creates a derived type of OrtExternalMemoryHandle and returns a pointer to the base. EP is responsible for the lifetime of the handle (release via ReleaseMemory).

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	desc	Descriptor containing the external memory handle and properties.
[out]	out_handle	Output parameter set to the created OrtExternalMemoryHandle (EP's derived type).

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ ImportSemaphore()

OrtStatus * OrtExternalResourceImporterImpl::ImportSemaphore	(	OrtExternalResourceImporterImpl *	this_ptr,
		const OrtExternalSemaphoreDescriptor *	desc,
		OrtExternalSemaphoreHandle **	out_handle
	)

Import an external semaphore.

The EP creates a derived type of OrtExternalSemaphoreHandle and returns a pointer to the base. EP is responsible for the lifetime of the handle (release via ReleaseSemaphore).

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	desc	Descriptor containing the external semaphore handle and type.
[out]	out_handle	Output parameter set to the created OrtExternalSemaphoreHandle (EP's derived type).

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ SignalSemaphore()

OrtStatus * OrtExternalResourceImporterImpl::SignalSemaphore	(	OrtExternalResourceImporterImpl *	this_ptr,
		OrtExternalSemaphoreHandle *	handle,
		OrtSyncStream *	stream,
		uint64_t	value
	)

Signal an external semaphore from the EP's stream.

Inserts a signal operation into the EP's stream that sets the semaphore to the specified value when reached.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	handle	The imported external semaphore (EP casts to its derived type).
[in]	stream	The OrtSyncStream to signal from.
[in]	value	The fence/semaphore value to signal.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ WaitSemaphore()

OrtStatus * OrtExternalResourceImporterImpl::WaitSemaphore	(	OrtExternalResourceImporterImpl *	this_ptr,
		OrtExternalSemaphoreHandle *	handle,
		OrtSyncStream *	stream,
		uint64_t	value
	)

Wait on an external semaphore on the EP's stream.

Inserts a wait operation into the EP's stream that blocks until the semaphore reaches the specified value.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	handle	The imported external semaphore (EP casts to its derived type).
[in]	stream	The OrtSyncStream to wait on.
[in]	value	The fence/semaphore value to wait for.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

Member Data Documentation

◆ CanImportMemory

bool( * OrtExternalResourceImporterImpl::CanImportMemory) (const OrtExternalResourceImporterImpl *this_ptr, OrtExternalMemoryHandleType handle_type)

Check if the implementation can import external memory of the given handle type.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	handle_type	The type of external memory handle to check.

Returns: True if the handle type is supported.

Since: Version 1.24.

◆ CanImportSemaphore

bool( * OrtExternalResourceImporterImpl::CanImportSemaphore) (const OrtExternalResourceImporterImpl *this_ptr, OrtExternalSemaphoreType type)

Check if the implementation can import external semaphores of the given type.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	type	The type of external semaphore to check.

Returns: True if the semaphore type is supported.

Since: Version 1.24.

◆ ort_version_supported

uint32_t OrtExternalResourceImporterImpl::ort_version_supported

Must be initialized to ORT_API_VERSION.

◆ Release

void( * OrtExternalResourceImporterImpl::Release) (OrtExternalResourceImporterImpl *this_ptr)

Release the OrtExternalResourceImporterImpl instance.

This is called by ORT when the OrtExternalResourceImporterImpl instance is no longer needed. The implementation should release any resources held by the instance.

Parameters

[in] this_ptr Pointer to the OrtExternalResourceImporterImpl instance.

Since: Version 1.24.

◆ ReleaseMemory

void( * OrtExternalResourceImporterImpl::ReleaseMemory) (OrtExternalResourceImporterImpl *this_ptr, OrtExternalMemoryHandle *handle)

Release an imported external memory handle.

The EP deletes its derived type instance.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	handle	The OrtExternalMemoryHandle to release (EP casts to its derived type).

Since: Version 1.24.

◆ ReleaseSemaphore

void( * OrtExternalResourceImporterImpl::ReleaseSemaphore) (OrtExternalResourceImporterImpl *this_ptr, OrtExternalSemaphoreHandle *handle)

Release an imported external semaphore handle.

The EP deletes its derived type instance.

Parameters

[in]	this_ptr	Pointer to the OrtExternalResourceImporterImpl instance.
[in]	handle	The OrtExternalSemaphoreHandle to release (EP casts to its derived type).

Since: Version 1.24.

Public Member Functions

Public Attributes

Detailed Description

Member Function Documentation

◆ CreateTensorFromMemory()

◆ ImportMemory()

◆ ImportSemaphore()

◆ SignalSemaphore()

◆ WaitSemaphore()

Member Data Documentation

◆ CanImportMemory

◆ CanImportSemaphore

◆ ort_version_supported

◆ Release

◆ ReleaseMemory

◆ ReleaseSemaphore