The OrtEpFactory provides functions to create and manage execution providers. More...

#include <onnxruntime_ep_c_api.h>

Public Member Functions
OrtStatus *	GetSupportedDevices (OrtEpFactory this_ptr, const OrtHardwareDevice const devices, size_t num_devices, OrtEpDevice ep_devices, size_t max_ep_devices, size_t num_ep_devices)
	Get information from the execution provider about OrtHardwareDevice support.

OrtStatus *	CreateEp (OrtEpFactory this_ptr, const OrtHardwareDevice const devices, const OrtKeyValuePairs const ep_metadata_pairs, size_t num_devices, const OrtSessionOptions session_options, const OrtLogger logger, OrtEp *ep)
	Function to create an OrtEp instance for use in a Session.

OrtStatus *	ValidateCompiledModelCompatibilityInfo (OrtEpFactory this_ptr, const OrtHardwareDevice const devices, size_t num_devices, const char compatibility_info, OrtCompiledModelCompatibility *model_compatibility)
	Validate the compatibility of a compiled model with the execution provider factory for one or more devices.

OrtStatus *	CreateAllocator (OrtEpFactory this_ptr, const OrtMemoryInfo memory_info, const OrtKeyValuePairs allocator_options, OrtAllocator *allocator)
	Create an OrtAllocator that can be shared across sessions for the given OrtMemoryInfo.

OrtStatus *	CreateDataTransfer (OrtEpFactory this_ptr, OrtDataTransferImpl *data_transfer)
	Create an OrtDataTransferImpl instance for the factory.

OrtStatus *	CreateSyncStreamForDevice (OrtEpFactory this_ptr, const OrtMemoryDevice memory_device, const OrtKeyValuePairs stream_options, OrtSyncStreamImpl *stream)
	Create a synchronization stream for the given memory device.

OrtStatus *	GetHardwareDeviceIncompatibilityDetails (OrtEpFactory this_ptr, const OrtHardwareDevice hw, OrtDeviceEpIncompatibilityDetails *details)
	Check for known incompatibility reasons between a hardware device and this execution provider.

OrtStatus *	CreateExternalResourceImporterForDevice (OrtEpFactory this_ptr, const OrtEpDevice ep_device, OrtExternalResourceImporterImpl **out_importer)
	Create an OrtExternalResourceImporterImpl for external resource import.

OrtStatus *	GetNumCustomOpDomains (OrtEpFactory this_ptr, size_t num_domains)
	Returns the number of OrtCustomOpDomains that this factory provides.

OrtStatus *	GetCustomOpDomains (OrtEpFactory this_ptr, OrtCustomOpDomain *domains, size_t num_domains)
	Gets the EP-specific OrtCustomOpDomains.

Public Attributes
uint32_t	ort_version_supported
	The ONNX Runtime version the execution provider was compiled with.

const char (	GetName )(const OrtEpFactory *this_ptr)
	Get the name of the execution provider that the factory creates.

const char (	GetVendor )(const OrtEpFactory *this_ptr)
	Get the name of vendor who owns the execution provider that the factory creates.

void(*	ReleaseEp )(OrtEpFactory this_ptr, struct OrtEp ep)
	Release the OrtEp instance.

uint32_t(*	GetVendorId )(const OrtEpFactory *this_ptr)
	Get the vendor id who owns the execution provider that the factory creates.

const char (	GetVersion )(const OrtEpFactory *this_ptr)
	Get the version of the execution provider that the factory creates.

void(*	ReleaseAllocator )(OrtEpFactory this_ptr, OrtAllocator allocator)
	Release an OrtAllocator created by the factory.

bool(*	IsStreamAware )(const OrtEpFactory *this_ptr)
	Check if execution providers created by the factory are stream aware.

Detailed Description

The OrtEpFactory provides functions to create and manage execution providers.

Since: Version 1.22.

Member Function Documentation

◆ CreateAllocator()

OrtStatus * OrtEpFactory::CreateAllocator	(	OrtEpFactory *	this_ptr,
		const OrtMemoryInfo *	memory_info,
		const OrtKeyValuePairs *	allocator_options,
		OrtAllocator **	allocator
	)

Create an OrtAllocator that can be shared across sessions for the given OrtMemoryInfo.

The factory that creates the EP is responsible for providing the allocators required by the EP. The OrtMemoryInfo instance will match one of the values set in the OrtEpDevice using EpDevice_AddAllocatorInfo.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	memory_info	The OrtMemoryInfo to create the allocator for. May be nullptr.
[in]	allocator_options	Optional key-value pairs for allocator options, can be nullptr.
[out]	allocator	The created OrtAllocator instance. Set to nullptr if the default CPU allocator is used.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ CreateDataTransfer()

OrtStatus * OrtEpFactory::CreateDataTransfer	(	OrtEpFactory *	this_ptr,
		OrtDataTransferImpl **	data_transfer
	)

Create an OrtDataTransferImpl instance for the factory.

This is used to create an IDataTransfer implementation that can be used to copy data between devices that the execution provider supports.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[out]	data_transfer	The created OrtDataTransferImpl instance. Set to nullptr if not required.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ CreateEp()

OrtStatus * OrtEpFactory::CreateEp	(	OrtEpFactory *	this_ptr,
		const OrtHardwareDevice const	devices,
		const OrtKeyValuePairs const	ep_metadata_pairs,
		size_t	num_devices,
		const OrtSessionOptions *	session_options,
		const OrtLogger *	logger,
		OrtEp **	ep
	)

Function to create an OrtEp instance for use in a Session.

ORT will call ReleaseEp to release the instance when it is no longer needed.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	devices	The OrtHardwareDevice instances that the execution provider was selected to use. May be a subset of the OrtHardwareDevice instances that the execution provider's factory set as supported in the call to OrtEpFactory::GetSupportedDevices.
[in]	ep_metadata_pairs	Execution provider metadata that was provided to OrtEpApi::CreateEpDevice, for each device.
[in]	num_devices	The number of devices the execution provider was selected for.
[in]	session_options	The OrtSessionOptions instance that contains the configuration options for the session. This will include ep_options from GetSupportedDevices as well as any user provided overrides. Execution provider options will have been added with a prefix of 'ep.[ep name].'. The OrtSessionOptions instance will NOT be valid after this call and should not be stored for later use.
[in]	logger	The OrtLogger instance for the session that the execution provider should use for logging.
[out]	ep	The OrtEp instance created by the factory.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ CreateExternalResourceImporterForDevice()

OrtStatus * OrtEpFactory::CreateExternalResourceImporterForDevice	(	OrtEpFactory *	this_ptr,
		const OrtEpDevice *	ep_device,
		OrtExternalResourceImporterImpl **	out_importer
	)

Create an OrtExternalResourceImporterImpl for external resource import.

This is used to create an external resource importer that enables zero-copy import of external GPU memory (e.g., D3D12 shared resources) and synchronization primitives (e.g., D3D12 timeline fences).

EPs that support external resource import (via CUDA, HIP, Vulkan, or D3D12 APIs) can implement this to allow applications to share GPU resources without copies.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	ep_device	The OrtEpDevice to create the external resource importer for.
[out]	out_importer	The created OrtExternalResourceImporterImpl instance. Set to nullptr if external resource import is not supported.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Note: Implementation of this function is optional. An EP factory should only implement this if it supports external resource import. If not implemented or not supported, return ORT_NOT_IMPLEMENTED or set out_importer to nullptr.

Since: Version 1.24.

◆ CreateSyncStreamForDevice()

OrtStatus * OrtEpFactory::CreateSyncStreamForDevice	(	OrtEpFactory *	this_ptr,
		const OrtMemoryDevice *	memory_device,
		const OrtKeyValuePairs *	stream_options,
		OrtSyncStreamImpl **	stream
	)

Create a synchronization stream for the given memory device.

This is used to create a synchronization stream for the memory device that can be used for operations outside of a session.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	memory_device	The OrtMemoryDevice to create the synchronization stream for.
[in]	stream_options	Options for stream creation. May be nullptr.
[out]	stream	The created OrtSyncStreamImpl instance. nullptr if the execution provider is not stream aware.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ GetCustomOpDomains()

OrtStatus * OrtEpFactory::GetCustomOpDomains	(	OrtEpFactory *	this_ptr,
		OrtCustomOpDomain **	domains,
		size_t	num_domains
	)

Gets the EP-specific OrtCustomOpDomains.

This function is used when running inference on a model that contains EP-specific custom operations.

Workflow:

The EP factory implements this function to supply a list of OrtCustomOpDomain instances.
The application either 1) calls SessionOptionsAppendExecutionProvider_V2() with an OrtEpDevice containing the plugin EP's factory or 2) enables auto ep selection.
1) SessionOptionsAppendExecutionProvider_V2() appends the provided OrtCustomOpDomains to the session options or 2) ORT registers the OrtCustomOpDomains provided by the EP devices that could be potentially selected.

As a result, any session created from these session options will have these custom op domains registered in ORT, ensuring that the custom ops are properly recognized and validated when the model is loaded.

Plugin EPs can provide two types of custom ops:

A full OrtCustomOp with a concrete kernel implementation
- A Plugin EP can supply an OrtCustomOp and a corresponding CustomKernel::Compute() implementation.
- In GetCapability(), it calls EpGraphSupportInfo_AddSingleNode() to inform ORT that the custom node should NOT be fused or compiled. Instead, ORT should invoke the custom node's Compute() function at runtime.
A "placeholder" OrtCustomOp with an empty kernel implementation
- A compile-based Plugin EP can supply an OrtCustomOp whose CustomKernel::Compute() does nothing. The purpose is to satisfy model validation during model loading by registering the custom op as a valid operator in the session.
- In GetCapability(), the EP should call EpGraphSupportInfo_AddNodesToFuse() to notify ORT that this custom node should be fused and compiled by the EP.
- In Compile(), the EP executes its compiled bits to perform inference for the fused custom node.

Note: The OrtCustomOpDomain instances must be valid while any session is using them. EP factory has the responsibility to release OrtCustomOpDomain instances it creates. It happens automatically if using the C++ Ort::CustomOpDomain class.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[out]	domains	Array of `num_domains` elements pre-allocated by ORT that should be filled with OrtCustomOpDomain instances created by the EP. The `num_domains` is the value returned by GetNumCustomOpDomains().
[in]	num_domains	The size of the `domains` array pre-allocated by ORT.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ GetHardwareDeviceIncompatibilityDetails()

OrtStatus * OrtEpFactory::GetHardwareDeviceIncompatibilityDetails	(	OrtEpFactory *	this_ptr,
		const OrtHardwareDevice *	hw,
		OrtDeviceEpIncompatibilityDetails *	details
	)

Check for known incompatibility reasons between a hardware device and this execution provider.

This function allows an execution provider to check if a specific hardware device is compatible with the execution provider. The EP can set specific incompatibility reasons via the OrtDeviceEpIncompatibilityDetails parameter using OrtEpApi::DeviceEpIncompatibilityDetails_SetDetails.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	hw	The hardware device to check for incompatibility.
[in,out]	details	Pre-allocated incompatibility details object created and initialized by ORT. The EP can use OrtEpApi::DeviceEpIncompatibilityDetails_SetDetails to set incompatibility information. If the device is compatible, the EP can leave the object unchanged (it defaults to no incompatibility).

Note: Implementation of this function is optional. If not implemented, ORT will assume the device is compatible with this EP.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ GetNumCustomOpDomains()

OrtStatus * OrtEpFactory::GetNumCustomOpDomains	(	OrtEpFactory *	this_ptr,
		size_t *	num_domains
	)

Returns the number of OrtCustomOpDomains that this factory provides.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[out]	num_domains	Output parameter set to the number of provided OrtCustomOpDomain instances.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.24.

◆ GetSupportedDevices()

OrtStatus * OrtEpFactory::GetSupportedDevices	(	OrtEpFactory *	this_ptr,
		const OrtHardwareDevice const	devices,
		size_t	num_devices,
		OrtEpDevice **	ep_devices,
		size_t	max_ep_devices,
		size_t *	num_ep_devices
	)

Get information from the execution provider about OrtHardwareDevice support.

Parameters

[in]	this_ptr	The OrtEpFactory instance. Non-const as the factory is passed through to the CreateEp call via the OrtEpDevice.
[in]	devices	The OrtHardwareDevice instances that are available.
[in]	num_devices	The number of OrtHardwareDevice instances.
[out]	ep_devices	OrtEpDevice instances for each OrtHardwareDevice that the EP can use. The implementation should call OrtEpApi::CreateEpDevice to create, and add the OrtEpDevice instances to this pre-allocated array. ORT will take ownership of the values returned. i.e. usage is `ep_devices[0] = <ptr to OrtEpDevice created with OrtEpApi::CreateEpDevice>;`
[in]	max_ep_devices	The maximum number of OrtEpDevices that can be added to ep_devices. Current default is 8. This can be increased if needed.
[out]	num_ep_devices	The number of EP devices added to ep_devices.

Returns: true if the factory can create an execution provider that uses device.

Since: Version 1.22.

◆ ValidateCompiledModelCompatibilityInfo()

OrtStatus * OrtEpFactory::ValidateCompiledModelCompatibilityInfo	(	OrtEpFactory *	this_ptr,
		const OrtHardwareDevice const	devices,
		size_t	num_devices,
		const char *	compatibility_info,
		OrtCompiledModelCompatibility *	model_compatibility
	)

Validate the compatibility of a compiled model with the execution provider factory for one or more devices.

Given a compatibility info string produced during model compilation, the EP factory should determine whether the compiled model is compatible with the EP factory when targeting the provided hardware devices. All devices provided must belong to the same execution provider instance that this factory creates.

The EP factory implementation should consider the set of devices (e.g., multi-adapter or multi-GPU scenarios) when evaluating compatibility and set model_compatibility accordingly.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	devices	Array of OrtHardwareDevice pointers that the EP would run on. All must map to this EP.
[in]	num_devices	Number of entries in `devices`.
[in]	compatibility_info	The compatibility information string produced when the model was compiled.
[out]	model_compatibility	OrtCompiledModelCompatibility value describing the compatibility of the model with the EP.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

Member Data Documentation

◆ GetName

const char *( * OrtEpFactory::GetName) (const OrtEpFactory *this_ptr)

Get the name of the execution provider that the factory creates.

The returned string should be a null-terminated, UTF-8 encoded string. ORT will copy it.

Parameters

[in] this_ptr The OrtEpFactory instance.

Returns: The name of the execution provider the factory creates.

Since: Version 1.22.

◆ GetVendor

const char *( * OrtEpFactory::GetVendor) (const OrtEpFactory *this_ptr)

Get the name of vendor who owns the execution provider that the factory creates.

The returned string should be a null-terminated, UTF-8 encoded string. ORT will copy it.

Parameters

[in] this_ptr The OrtEpFactory instance.

Returns: vendor The vendor name of the execution provider the factory creates.

Since: Version 1.22.

◆ GetVendorId

uint32_t( * OrtEpFactory::GetVendorId) (const OrtEpFactory *this_ptr)

Get the vendor id who owns the execution provider that the factory creates.

This is typically the PCI vendor ID. See https://pcisig.com/membership/member-companies

Parameters

[in] this_ptr The OrtEpFactory instance.

Returns: vendor_id The vendor ID of the execution provider the factory creates.

Since: Version 1.23.

◆ GetVersion

const char *( * OrtEpFactory::GetVersion) (const OrtEpFactory *this_ptr)

Get the version of the execution provider that the factory creates.

The version string should adhere to the Semantic Versioning 2.0 specification (https://github.com/semver/semver/blob/v2.0.0/semver.md).

The returned string should be a null-terminated, UTF-8 encoded string. ORT will copy it.

Parameters

[in] this_ptr The OrtEpFactory instance.

Returns: The execution provider version string.

Since: Version 1.23.

◆ IsStreamAware

bool( * OrtEpFactory::IsStreamAware) (const OrtEpFactory *this_ptr)

Check if execution providers created by the factory are stream aware.

Parameters

[in] this_ptr The OrtEpFactory instance.

Returns: True if the factory creates execution providers that are stream aware and it implements CreateSyncStreamForDevice.

Since: Version 1.23.

◆ ort_version_supported

uint32_t OrtEpFactory::ort_version_supported

The ONNX Runtime version the execution provider was compiled with.

Implementation should set to ORT_API_VERSION. ORT will use this to ensure it does not call functions that were not available when the library was compiled.

Since: Version 1.22.

◆ ReleaseAllocator

void( * OrtEpFactory::ReleaseAllocator) (OrtEpFactory *this_ptr, OrtAllocator *allocator)

Release an OrtAllocator created by the factory.

Since: Version 1.23.

◆ ReleaseEp

void( * OrtEpFactory::ReleaseEp) (OrtEpFactory *this_ptr, struct OrtEp *ep)

Release the OrtEp instance.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	ep	The OrtEp instance to release.

Since: Version 1.22.

Public Member Functions

Public Attributes

Detailed Description

Member Function Documentation

◆ CreateAllocator()

◆ CreateDataTransfer()

◆ CreateEp()

◆ CreateExternalResourceImporterForDevice()

◆ CreateSyncStreamForDevice()

◆ GetCustomOpDomains()

◆ GetHardwareDeviceIncompatibilityDetails()

◆ GetNumCustomOpDomains()

◆ GetSupportedDevices()

◆ ValidateCompiledModelCompatibilityInfo()

Member Data Documentation

◆ GetName

◆ GetVendor

◆ GetVendorId

◆ GetVersion

◆ IsStreamAware

◆ ort_version_supported

◆ ReleaseAllocator

◆ ReleaseEp