The OrtInteropApi struct provides functions for external resource interop with execution providers. More...

#include <onnxruntime_c_api.h>

Public Member Functions
OrtExternalResourceImporter
OrtStatus *	CreateExternalResourceImporterForDevice (const OrtEpDevice ep_device, OrtExternalResourceImporter *out_importer)
	Create an external resource importer for a specific EP device.

void	ReleaseExternalResourceImporter (OrtExternalResourceImporter *input)
	Release an OrtExternalResourceImporter instance.

Memory Import
OrtStatus *	CanImportMemory (const OrtExternalResourceImporter importer, OrtExternalMemoryHandleType handle_type, bool out_supported)
	Check if the external resource importer can import a specific memory handle type.

OrtStatus *	ImportMemory (OrtExternalResourceImporter importer, const OrtExternalMemoryDescriptor desc, OrtExternalMemoryHandle **out_handle)
	Import external memory into the execution provider.

void	ReleaseExternalMemoryHandle (OrtExternalMemoryHandle *input)
	Release an OrtExternalMemoryHandle instance.

OrtStatus *	CreateTensorFromMemory (OrtExternalResourceImporter importer, const OrtExternalMemoryHandle mem_handle, const OrtExternalTensorDescriptor tensor_desc, OrtValue *out_tensor)
	Create a tensor backed by imported external memory.

Semaphore Import
OrtStatus *	CanImportSemaphore (const OrtExternalResourceImporter importer, OrtExternalSemaphoreType type, bool out_supported)
	Check if the external resource importer can import a specific semaphore type.

OrtStatus *	ImportSemaphore (OrtExternalResourceImporter importer, const OrtExternalSemaphoreDescriptor desc, OrtExternalSemaphoreHandle **out_handle)
	Import an external semaphore into the execution provider.

void	ReleaseExternalSemaphoreHandle (OrtExternalSemaphoreHandle *input)
	Release an OrtExternalSemaphoreHandle instance.

OrtStatus *	WaitSemaphore (OrtExternalResourceImporter importer, OrtExternalSemaphoreHandle semaphore_handle, OrtSyncStream *stream, uint64_t value)
	Wait on an external semaphore on the EP's stream.

OrtStatus *	SignalSemaphore (OrtExternalResourceImporter importer, OrtExternalSemaphoreHandle semaphore_handle, OrtSyncStream *stream, uint64_t value)
	Signal an external semaphore from the EP's stream.

Detailed Description

The OrtInteropApi struct provides functions for external resource interop with execution providers.

This API enables importing external GPU resources (memory and semaphores) for zero-copy sharing between ORT inference and other GPU workloads (e.g., D3D12 applications, media pipelines).

The API is designed to be EP-agnostic and can be extended to support various GPU interop mechanisms (D3D12 shared handles, CUDA external memory, Vulkan, etc.).

Example usage (error handling not shown): const OrtInteropApi* interop_api = ort_api->GetInteropApi(); OrtExternalResourceImporter* importer = NULL;

status = interop_api->CreateExternalResourceImporterForDevice(ep_device, &importer); if (importer == nullptr) { // External resource import is optional for EPs to implement return; } bool can_import = false; status = interop_api->CanImportMemory(importer, ORT_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE, &can_import); if (can_import) { OrtExternalMemoryHandle* mem_handle = NULL; status = interop_api->ImportMemory(importer, &mem_desc, &mem_handle); // ... use mem_handle to create tensors ... interop_api->ReleaseExternalMemoryHandle(mem_handle); } interop_api->ReleaseExternalResourceImporter(importer);

Since: Version 1.24.

Member Function Documentation

◆ CanImportMemory()

OrtStatus * OrtInteropApi::CanImportMemory	(	const OrtExternalResourceImporter *	importer,
		OrtExternalMemoryHandleType	handle_type,
		bool *	out_supported
	)

Check if the external resource importer can import a specific memory handle type.

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	handle_type	The type of external memory handle to check.
[out]	out_supported	Set to true if the handle type is supported.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ CanImportSemaphore()

OrtStatus * OrtInteropApi::CanImportSemaphore	(	const OrtExternalResourceImporter *	importer,
		OrtExternalSemaphoreType	type,
		bool *	out_supported
	)

Check if the external resource importer can import a specific semaphore type.

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	type	The type of external semaphore to check.
[out]	out_supported	Set to true if the semaphore type is supported.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ CreateExternalResourceImporterForDevice()

OrtStatus * OrtInteropApi::CreateExternalResourceImporterForDevice	(	const OrtEpDevice *	ep_device,
		OrtExternalResourceImporter **	out_importer
	)

Create an external resource importer for a specific EP device.

The external resource importer is a capability object that provides methods for importing external GPU memory and semaphores for zero-copy import with an execution provider.

This is an optional EP capability. If the EP does not support external resource import, out_importer is set to nullptr and the function returns success (nullptr status). This allows callers to use the simple "if (status != nullptr) handle_error()" pattern and check out_importer separately for capability detection.

Parameters

[in]	ep_device	The OrtEpDevice instance to create the importer for.
[out]	out_importer	Output parameter set to the created OrtExternalResourceImporter instance, or nullptr if the EP does not support external resource import.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ CreateTensorFromMemory()

OrtStatus * OrtInteropApi::CreateTensorFromMemory	(	OrtExternalResourceImporter *	importer,
		const OrtExternalMemoryHandle *	mem_handle,
		const OrtExternalTensorDescriptor *	tensor_desc,
		OrtValue **	out_tensor
	)

Create a tensor backed by imported external memory.

The created tensor is a view over the imported memory and does not copy data. The OrtExternalMemoryHandle must remain valid for the lifetime of the tensor.

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	mem_handle	The imported external memory handle.
[in]	tensor_desc	Descriptor specifying tensor element type, shape, and optional offset.
[out]	out_tensor	Output parameter set to the created OrtValue containing the tensor. The caller owns the returned tensor and must call ReleaseValue to free it.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ ImportMemory()

OrtStatus * OrtInteropApi::ImportMemory	(	OrtExternalResourceImporter *	importer,
		const OrtExternalMemoryDescriptor *	desc,
		OrtExternalMemoryHandle **	out_handle
	)

Import external memory into the execution provider.

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	desc	Descriptor containing the external memory handle and properties.
[out]	out_handle	Output parameter set to the created OrtExternalMemoryHandle. The caller owns the returned handle and must call ReleaseExternalMemoryHandle to free it.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ ImportSemaphore()

OrtStatus * OrtInteropApi::ImportSemaphore	(	OrtExternalResourceImporter *	importer,
		const OrtExternalSemaphoreDescriptor *	desc,
		OrtExternalSemaphoreHandle **	out_handle
	)

Import an external semaphore into the execution provider.

The returned OrtExternalSemaphoreHandle can be used with WaitSemaphore and an OrtSyncStream to synchronize execution with external operations.

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	desc	Descriptor containing the external semaphore handle and type.
[out]	out_handle	Output parameter set to the created OrtExternalSemaphoreHandle.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ ReleaseExternalMemoryHandle()

void OrtInteropApi::ReleaseExternalMemoryHandle ( OrtExternalMemoryHandle * input )

Release an OrtExternalMemoryHandle instance.

Parameters

[in] input The OrtExternalMemoryHandle instance to release. May be nullptr.

Since: Version 1.24.

◆ ReleaseExternalResourceImporter()

void OrtInteropApi::ReleaseExternalResourceImporter ( OrtExternalResourceImporter * input )

Release an OrtExternalResourceImporter instance.

Parameters

[in] input The OrtExternalResourceImporter instance to release. May be nullptr.

Since: Version 1.24.

◆ ReleaseExternalSemaphoreHandle()

void OrtInteropApi::ReleaseExternalSemaphoreHandle ( OrtExternalSemaphoreHandle * input )

Release an OrtExternalSemaphoreHandle instance.

Parameters

[in] input The OrtExternalSemaphoreHandle instance to release. May be nullptr.

Since: Version 1.24.

◆ SignalSemaphore()

OrtStatus * OrtInteropApi::SignalSemaphore	(	OrtExternalResourceImporter *	importer,
		OrtExternalSemaphoreHandle *	semaphore_handle,
		OrtSyncStream *	stream,
		uint64_t	value
	)

Signal an external semaphore from the EP's stream.

Inserts a signal operation into the EP's stream that sets the semaphore to the specified value when reached. This is used to notify external GPU work (e.g., D3D12 timeline fence) that ORT inference is complete.

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	semaphore_handle	The imported external semaphore.
[in]	stream	The OrtSyncStream to signal from.
[in]	value	The fence/semaphore value to signal.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ WaitSemaphore()

OrtStatus * OrtInteropApi::WaitSemaphore	(	OrtExternalResourceImporter *	importer,
		OrtExternalSemaphoreHandle *	semaphore_handle,
		OrtSyncStream *	stream,
		uint64_t	value
	)

Wait on an external semaphore on the EP's stream.

Inserts a wait operation into the EP's stream that blocks until the semaphore reaches the specified value. This is used to synchronize with external GPU work (e.g., D3D12 timeline fence).

Parameters

[in]	importer	The OrtExternalResourceImporter instance.
[in]	semaphore_handle	The imported external semaphore.
[in]	stream	The OrtSyncStream to wait on.
[in]	value	The fence/semaphore value to wait for.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

Public Member Functions

Detailed Description

Member Function Documentation

◆ CanImportMemory()

◆ CanImportSemaphore()

◆ CreateExternalResourceImporterForDevice()

◆ CreateTensorFromMemory()

◆ ImportMemory()

◆ ImportSemaphore()

◆ ReleaseExternalMemoryHandle()

◆ ReleaseExternalResourceImporter()

◆ ReleaseExternalSemaphoreHandle()

◆ SignalSemaphore()

◆ WaitSemaphore()