The OrtEpApi struct provides functions that are relevant to the implementation of an execution provider. More...

#include <onnxruntime_ep_c_api.h>

Public Member Functions
OrtStatus *	CreateEpDevice (OrtEpFactory ep_factory, const OrtHardwareDevice hardware_device, const OrtKeyValuePairs ep_metadata, const OrtKeyValuePairs ep_options, OrtEpDevice **ep_device)
	Create an OrtEpDevice for the EP and an OrtHardwareDevice.

void	ReleaseEpDevice (OrtEpDevice *input)

OrtStatus *	EpGraphSupportInfo_AddNodesToFuse (OrtEpGraphSupportInfo graph_support_info, const OrtNode const nodes, size_t num_nodes, const OrtNodeFusionOptions node_fusion_options)
	Specify nodes that are supported by an OrtEp and should be fused into one node.

OrtStatus *	EpGraphSupportInfo_AddSingleNode (OrtEpGraphSupportInfo graph_support_info, const OrtNode node)
	Specify a node that is supported by an OrtEp and should be run with a registered EP kernel.

OrtStatus *	EpDevice_AddAllocatorInfo (OrtEpDevice ep_device, const OrtMemoryInfo allocator_memory_info)
	Register an allocator with the OrtEpDevice.

OrtStatus *	CreateHardwareDevice (OrtHardwareDeviceType type, uint32_t vendor_id, uint32_t device_id, const char vendor_name, const OrtKeyValuePairs metadata, OrtHardwareDevice **hardware_device)
	Create an OrtHardwareDevice.

void	ReleaseHardwareDevice (OrtHardwareDevice *input)

Public Attributes
const char (	NodeComputeContext_NodeName )(const OrtNodeComputeContext *context)
	Query a OrtNodeComputeContext for the name of the node that encapsulates the compiled/fused node.

const OrtMemoryDevice (	MemoryInfo_GetMemoryDevice )(const OrtMemoryInfo *memory_info)
	Get the OrtMemoryDevice from an OrtMemoryInfo instance.

const OrtMemoryDevice (	Value_GetMemoryDevice )(const OrtValue *value)
	Get the OrtMemoryDevice from an OrtValue instance if it contains a Tensor.

bool(*	MemoryDevice_AreEqual )(const OrtMemoryDevice a, const OrtMemoryDevice b)
	Compare two OrtMemoryDevice instances for equality.

OrtMemoryInfoDeviceType(*	MemoryDevice_GetDeviceType )(const OrtMemoryDevice *memory_device)
	Get the OrtMemoryInfoDeviceType value from an OrtMemoryDevice instance.

OrtDeviceMemoryType(*	MemoryDevice_GetMemoryType )(const OrtMemoryDevice *memory_device)
	Get the OrtDeviceMemoryType value from an OrtMemoryDevice instance.

uint32_t(*	MemoryDevice_GetVendorId )(const OrtMemoryDevice *memory_device)
	Get the vendor ID from an OrtMemoryDevice instance.

uint32_t(*	MemoryDevice_GetDeviceId )(const OrtMemoryDevice *memory_device)
	Get the device ID from an OrtMemoryDevice instance.

const OrtSyncStreamImpl (	SyncStream_GetImpl )(const OrtSyncStream *stream)
	Get the OrtSyncStreamImpl associated with an OrtSyncStream instance.

uint64_t(*	SyncStream_GetSyncId )(const OrtSyncStream *stream)
	Get the current sync ID for a stream.

uint64_t(*	GetSyncIdForLastWaitOnSyncStream )(const OrtSyncStream producer_stream, const OrtSyncStream consumer_stream)
	Get the sync ID for the last time the consumer_stream waited on the producer_stream.

Detailed Description

The OrtEpApi struct provides functions that are relevant to the implementation of an execution provider.

Since: Version 1.22.

Member Function Documentation

◆ CreateEpDevice()

OrtStatus * OrtEpApi::CreateEpDevice	(	OrtEpFactory *	ep_factory,
		const OrtHardwareDevice *	hardware_device,
		const OrtKeyValuePairs *	ep_metadata,
		const OrtKeyValuePairs *	ep_options,
		OrtEpDevice **	ep_device
	)

Create an OrtEpDevice for the EP and an OrtHardwareDevice.

Parameters

[in]	ep_factory	Execution provider factory that is creating the instance.
[in]	hardware_device	Hardware device that the EP can utilize.
[in]	ep_metadata	Optional OrtKeyValuePairs instance for execution provider metadata that may be used during execution provider selection and passed to CreateEp. ep_device will copy this instance and the user should call ReleaseKeyValuePairs.
[in]	ep_options	Optional OrtKeyValuePairs instance for execution provider options that will be added to the Session configuration options if the execution provider is selected. ep_device will copy this instance and the user should call ReleaseKeyValuePairs.
	ep_device	OrtExecutionDevice that is created.

Since: Version 1.22.

◆ CreateHardwareDevice()

OrtStatus * OrtEpApi::CreateHardwareDevice	(	OrtHardwareDeviceType	type,
		uint32_t	vendor_id,
		uint32_t	device_id,
		const char *	vendor_name,
		const OrtKeyValuePairs *	metadata,
		OrtHardwareDevice **	hardware_device
	)

Create an OrtHardwareDevice.

Note: Called within OrtEpFactory::GetSupportedDevices to create a new hardware device (e.g., virtual).

Parameters

[in]	type	The hardware device type.
[in]	vendor_id	The hardware device's vendor identifier.
[in]	device_id	The hardware device's identifier.
[in]	vendor_name	The hardware device's vendor name as a null-terminated string. Copied by ORT.
[in]	metadata	Optional OrtKeyValuePairs instance for hardware device metadata that may be queried by applications via OrtApi::GetEpDevices(). Refer to onnxruntime_ep_device_ep_metadata_keys.h for common OrtHardwareDevice metadata keys.
[out]	hardware_device	Output parameter set to the new OrtHardwareDevice instance that is created. Must be release with ReleaseHardwareDevice().

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ EpDevice_AddAllocatorInfo()

OrtStatus * OrtEpApi::EpDevice_AddAllocatorInfo	(	OrtEpDevice *	ep_device,
		const OrtMemoryInfo *	allocator_memory_info
	)

Register an allocator with the OrtEpDevice.

This allows an EP to provide OrtMemoryInfo for DEFAULT and HOST_ACCESSIBLE memory type as needed. The registered values will be used in calls to OrtEpFactory::CreateAllocator to ensure the required allocator/s are available for EP usage.

Multiple calls for the same entry type will replace a previous entry.

Available entries:

OrtDeviceAllocator with type of OrtDeviceMemoryType_DEFAULT
OrtDeviceAllocator with type of OrtDeviceMemoryType_HOST_ACCESSIBLE
OrtReadOnlyAllocator with type of OrtDeviceMemoryType_DEFAULT
- if provided this allocator will only be used to copy initializers to the device the EP uses. ORT will use the OrtDeviceAllocator if not provided.

Parameters

[in]	ep_device	The OrtEpDevice instance to register the OrtMemoryInfo with.
[in]	allocator_memory_info	The OrtMemoryInfo information for the allocator.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ EpGraphSupportInfo_AddNodesToFuse()

OrtStatus * OrtEpApi::EpGraphSupportInfo_AddNodesToFuse	(	OrtEpGraphSupportInfo *	graph_support_info,
		const OrtNode const	nodes,
		size_t	num_nodes,
		const OrtNodeFusionOptions *	node_fusion_options
	)

Specify nodes that are supported by an OrtEp and should be fused into one node.

Because the nodes will be fused into one "fused node", there must not exist an unsupported node in a path between two of the provided nodes. Otherwise, the graph will become invalid.

This function can be called multiple times. A subsequent call to this function will force the next set of nodes to be fused into a different node.

Parameters

[in]	graph_support_info	OrtEpGraphSupportInfo instance to which to add the supported nodes.
[in]	nodes	Array of nodes supported by the EP that should be fused/compiled.
[in]	num_nodes	The number of supported nodes.
[in]	node_fusion_options	Optional node fusion options. Ignored if set to NULL.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ EpGraphSupportInfo_AddSingleNode()

OrtStatus * OrtEpApi::EpGraphSupportInfo_AddSingleNode	(	OrtEpGraphSupportInfo *	graph_support_info,
		const OrtNode *	node
	)

Specify a node that is supported by an OrtEp and should be run with a registered EP kernel.

Parameters

[in]	graph_support_info	OrtEpGraphSupportInfo instance to which to add the supported node.
[in]	node	The supported OrtNode instance.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ ReleaseEpDevice()

void OrtEpApi::ReleaseEpDevice ( OrtEpDevice * input )

◆ ReleaseHardwareDevice()

void OrtEpApi::ReleaseHardwareDevice ( OrtHardwareDevice * input )

Member Data Documentation

◆ GetSyncIdForLastWaitOnSyncStream

uint64_t( * OrtEpApi::GetSyncIdForLastWaitOnSyncStream) (const OrtSyncStream *producer_stream, const OrtSyncStream *consumer_stream)

Get the sync ID for the last time the consumer_stream waited on the producer_stream.

When two streams are synchronized, the sync id represents the event used in that synchronization.

Parameters

[in]	producer_stream	The OrtSyncStream that produced the data.
[in]	consumer_stream	The OrtSyncStream that waited on the producer_stream.

Returns: ID for last sync. 0 if no sync has occurred between the two streams.

Since: Version 1.23.

◆ MemoryDevice_AreEqual

bool( * OrtEpApi::MemoryDevice_AreEqual) (const OrtMemoryDevice *a, const OrtMemoryDevice *b)

Compare two OrtMemoryDevice instances for equality.

This is used to check if two memory devices are the same. Used to implement DataTransferImpl::CanCopy.

Parameters

[in]	a	The first OrtMemoryDevice instance to compare.
[in]	b	The second OrtMemoryDevice instance to compare.

Returns: True if the two OrtMemoryDevice instances are equal, false otherwise.

Since: Version 1.23.

◆ MemoryDevice_GetDeviceId

uint32_t( * OrtEpApi::MemoryDevice_GetDeviceId) (const OrtMemoryDevice *memory_device)

Get the device ID from an OrtMemoryDevice instance.

Parameters

[in] memory_device OrtMemoryDevice instance.

Returns: The device ID.

Since: Version 1.23.

◆ MemoryDevice_GetDeviceType

OrtMemoryInfoDeviceType( * OrtEpApi::MemoryDevice_GetDeviceType) (const OrtMemoryDevice *memory_device)

Get the OrtMemoryInfoDeviceType value from an OrtMemoryDevice instance.

Parameters

[in] memory_device OrtMemoryDevice instance.

Returns: The OrtMemoryInfoDeviceType value.

Since: Version 1.23.

◆ MemoryDevice_GetMemoryType

OrtDeviceMemoryType( * OrtEpApi::MemoryDevice_GetMemoryType) (const OrtMemoryDevice *memory_device)

Get the OrtDeviceMemoryType value from an OrtMemoryDevice instance.

Parameters

[in] memory_device OrtMemoryDevice instance.

Returns: The OrtDeviceMemoryType value.

Since: Version 1.23.

◆ MemoryDevice_GetVendorId

uint32_t( * OrtEpApi::MemoryDevice_GetVendorId) (const OrtMemoryDevice *memory_device)

Get the vendor ID from an OrtMemoryDevice instance.

The vendor ID is used to identify the vendor of the device, and is typically set to the PCI vendor ID.

If the device is not vendor specific (e.g. CPU memory) the vendor ID is set to 0.

Parameters

[in] memory_device OrtMemoryDevice instance.

Returns: The vendor ID value.

Since: Version 1.23.

◆ MemoryInfo_GetMemoryDevice

const OrtMemoryDevice *( * OrtEpApi::MemoryInfo_GetMemoryDevice) (const OrtMemoryInfo *memory_info)

Get the OrtMemoryDevice from an OrtMemoryInfo instance.

This is required for OrtDataTransferImpl (which implements onnxruntime::IDataTransfer) where the OrtMemoryDevice is used in the CanCopy and CopyTensors functions.

Parameters

[in] memory_info The OrtMemoryInfo instance to get the memory device from.

Returns: The OrtMemoryDevice associated with the OrtMemoryInfo instance.

Since: Version 1.23.

◆ NodeComputeContext_NodeName

const char *( * OrtEpApi::NodeComputeContext_NodeName) (const OrtNodeComputeContext *context)

Query a OrtNodeComputeContext for the name of the node that encapsulates the compiled/fused node.

Used in OrtNodeComputeInfo::CreateComputeState().

Parameters

[in] context The OrtNodeComputeContext instance to query.

Returns: The node's name.

Note: Returned string is owned by ORT and valid only while OrtNodeComputeInfo::CreateComputeState() is called.

Since: Version 1.23.

◆ SyncStream_GetImpl

const OrtSyncStreamImpl *( * OrtEpApi::SyncStream_GetImpl) (const OrtSyncStream *stream)

Get the OrtSyncStreamImpl associated with an OrtSyncStream instance.

This allows an the plugin library to connect its OrtSyncStreamImpl instance with an OrtSyncStream if needed.

Parameters

[in] stream The OrtSyncStream instance to find an OrtSyncStreamImpl for.

Returns: The associated OrtSyncStreamImpl if found. nullptr otherwise.

Since: Version 1.23.

Remarks: There should always be an OrtSyncStreamImpl associated with an OrtSyncStream instance that the EP gets.

◆ SyncStream_GetSyncId

uint64_t( * OrtEpApi::SyncStream_GetSyncId) (const OrtSyncStream *stream)

Get the current sync ID for a stream.

Parameters

[in] stream The OrtSyncStream to get the sync ID for.

Returns: Current sync ID.

Since: Version 1.23.

◆ Value_GetMemoryDevice

const OrtMemoryDevice *( * OrtEpApi::Value_GetMemoryDevice) (const OrtValue *value)

Get the OrtMemoryDevice from an OrtValue instance if it contains a Tensor.

Parameters

[in] value The OrtValue instance to get the memory device from.

Returns: Memory device if OrtValue contains a Tensor, nullptr otherwise.

Since: Version 1.23.

Public Member Functions

Public Attributes

Detailed Description

Member Function Documentation

◆ CreateEpDevice()

◆ CreateHardwareDevice()

◆ EpDevice_AddAllocatorInfo()

◆ EpGraphSupportInfo_AddNodesToFuse()

◆ EpGraphSupportInfo_AddSingleNode()

◆ ReleaseEpDevice()

◆ ReleaseHardwareDevice()

Member Data Documentation

◆ GetSyncIdForLastWaitOnSyncStream

◆ MemoryDevice_AreEqual

◆ MemoryDevice_GetDeviceId

◆ MemoryDevice_GetDeviceType

◆ MemoryDevice_GetMemoryType

◆ MemoryDevice_GetVendorId

◆ MemoryInfo_GetMemoryDevice

◆ NodeComputeContext_NodeName

◆ SyncStream_GetImpl

◆ SyncStream_GetSyncId

◆ Value_GetMemoryDevice