The OrtEp struct provides functions to implement for an execution provider. More...

#include <onnxruntime_ep_c_api.h>

Public Member Functions
OrtStatus *	GetCapability (OrtEp this_ptr, const OrtGraph graph, OrtEpGraphSupportInfo *graph_support_info)
	Get information about the nodes supported by the OrtEp instance.

OrtStatus *	Compile (OrtEp this_ptr, const OrtGraph graphs, const OrtNode fused_nodes, size_t count, OrtNodeComputeInfo node_compute_infos, OrtNode *ep_context_nodes)
	Compile OrtGraph instances assigned to the OrtEp. Implementer must set a OrtNodeComputeInfo instance for each OrtGraph in order to define its computation function.

OrtStatus *	GetPreferredDataLayout (OrtEp this_ptr, OrtEpDataLayout preferred_data_layout)
	Get the EP's preferred data layout.

OrtStatus *	ShouldConvertDataLayoutForOp (OrtEp this_ptr, const char domain, const char op_type, OrtEpDataLayout target_data_layout, int should_convert)
	Given an op with domain `domain` and type `op_type`, determine whether an associated node's data layout should be converted to `target_data_layout`. If the EP prefers a non-default data layout (see `GetPreferredDataLayout()`), this function will be called during layout transformation with `target_data_layout` set to the EP's preferred data layout.

OrtStatus *	SetDynamicOptions (OrtEp this_ptr, const char const option_keys, const char const *option_values, size_t num_options)
	Set dynamic options on this EP.

OrtStatus *	OnRunStart (OrtEp this_ptr, const OrtRunOptions run_options)
	Called by ORT to notify the EP of the start of a run.

OrtStatus *	OnRunEnd (OrtEp this_ptr, const OrtRunOptions run_options, bool sync_stream)
	Called by ORT to notify the EP of the end of a run.

OrtStatus *	CreateAllocator (OrtEp this_ptr, const OrtMemoryInfo memory_info, OrtAllocator **allocator)
	Create an OrtAllocator for the given OrtMemoryInfo for an OrtSession.

OrtStatus *	CreateSyncStreamForDevice (OrtEp this_ptr, const OrtMemoryDevice memory_device, OrtSyncStreamImpl **stream)
	Create a synchronization stream for the given memory device for an OrtSession.

Public Attributes
uint32_t	ort_version_supported
	The ONNX Runtime version the execution provider was compiled with.

const char (	GetName )(const OrtEp *this_ptr)
	Get the execution provider name.

void(*	ReleaseNodeComputeInfos )(OrtEp this_ptr, OrtNodeComputeInfo *node_compute_infos, size_t num_node_compute_infos)
	Release OrtNodeComputeInfo instances.

const char (	GetCompiledModelCompatibilityInfo )(OrtEp this_ptr, const OrtGraph graph)
	Get a string with details about the EP stack used to produce a compiled model.

Detailed Description

The OrtEp struct provides functions to implement for an execution provider.

Since: Version 1.22.

Member Function Documentation

◆ Compile()

OrtStatus * OrtEp::Compile	(	OrtEp *	this_ptr,
		const OrtGraph **	graphs,
		const OrtNode **	fused_nodes,
		size_t	count,
		OrtNodeComputeInfo **	node_compute_infos,
		OrtNode **	ep_context_nodes
	)

Compile OrtGraph instances assigned to the OrtEp. Implementer must set a OrtNodeComputeInfo instance for each OrtGraph in order to define its computation function.

If the session is configured to generate a pre-compiled model, the execution provider must return EPContext nodes, as OrtNode instances, that ONNX Runtime uses to create a pre-compiled model, known as an "EPContext model". An EPContext model contains EPContext nodes. Each EPContext node encapsulates the pre-compiled binary data for a OrtGraph compiled for a specific execution provider. For more details about the EPContext design, refer to: EPContext design document.

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	graphs	Array of `count` OrtGraph instances to compile. Each graph contains only the nodes for which the execution provider indicated support. Nested subgraphs contained by a node, such as an If or Loop, have separate OrtGraph instances.
[in]	fused_nodes	Array of `count` fused nodes that will replace the compiled graphs. Each fused node is an OrtNode initialized with the intended fused node name and input/output information.
[in]	count	The number of OrtGraph instances to compile.
[out]	node_compute_infos	Array of `count` OrtNodeComputeInfo instances that define each OrtGraph instance's computation function. The implementer allocates the OrtNodeComputeInfo instances. ORT calls ReleaseNodeComputeInfos() to release multiple instances in a batch.
[out]	ep_context_nodes	Output array of `count` OrtNode instances, each representing an EPContext node for a compiled OrtGraph. The execution provider must use OrtModelEditorApi::CreateNode to create the OrtNode instances. ONNX Runtime takes ownership of the OrtNode instances, so the execution provider must NOT call OrtApi::ReleaseNode. Should be ignored if the session is not configured to generate an EPContext model.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Note: Do NOT cache the provided OrtGraph instances in any of the OrtNodeComputeInfo functions because the graphs are only valid for the duration of the call to Compile. Any graph/node/input/output names that are needed by the OrtNodeComputeInfo functions must be copied and stored by the OrtEp.

Since: Version 1.23.

◆ CreateAllocator()

OrtStatus * OrtEp::CreateAllocator	(	OrtEp *	this_ptr,
		const OrtMemoryInfo *	memory_info,
		OrtAllocator **	allocator
	)

Create an OrtAllocator for the given OrtMemoryInfo for an OrtSession.

The OrtMemoryInfo instance will match one of the values set in the OrtEpDevice using EpDevice_AddAllocatorInfo. Any allocator specific options should be read from the session options.

If nullptr OrtEpFactory::CreateAllocator will be used.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	memory_info	The OrtMemoryInfo to create the allocator for. May be nullptr.
[out]	allocator	The created OrtAllocator instance. Set to nullptr if the default CPU allocator is used.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ CreateSyncStreamForDevice()

OrtStatus * OrtEp::CreateSyncStreamForDevice	(	OrtEp *	this_ptr,
		const OrtMemoryDevice *	memory_device,
		OrtSyncStreamImpl **	stream
	)

Create a synchronization stream for the given memory device for an OrtSession.

This is used to create a synchronization stream for the execution provider and is used to synchronize operations on the device during model execution. Any stream specific options should be read from the session options.

If nullptr OrtEpFactory::CreateSyncStreamForDevice will be used.

Parameters

[in]	this_ptr	The OrtEpFactory instance.
[in]	memory_device	The OrtMemoryDevice to create the synchronization stream for.
[out]	stream	The created OrtSyncStreamImpl instance. nullptr if the execution provider is not stream aware.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ GetCapability()

OrtStatus * OrtEp::GetCapability	(	OrtEp *	this_ptr,
		const OrtGraph *	graph,
		OrtEpGraphSupportInfo *	graph_support_info
	)

Get information about the nodes supported by the OrtEp instance.

IMPORTANT: This is not the final version of this API function. This is currently experimental but will be stabilized by the ONNX Runtime 1.23 release.

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	graph	The OrtGraph instance for which to populate node support. The OrtGraph could be a nested subgraph contained by a node (e.g., an If or Loop node). ONNX Runtime calls this function separately for each nested subgraph.
[in,out]	graph_support_info	OrtEpGraphSupportInfo instance that the implementer must fill out in order to specify the supported nodes.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ GetPreferredDataLayout()

OrtStatus * OrtEp::GetPreferredDataLayout	(	OrtEp *	this_ptr,
		OrtEpDataLayout *	preferred_data_layout
	)

Get the EP's preferred data layout.

Note: Implementation of this function is optional. If not implemented, ORT will assume that this EP prefers the data layout OrtEpDataLayout::NCHW.

Parameters

[in]	this_ptr	The OrtEp instance.
[out]	preferred_data_layout	The EP's preferred data layout.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ OnRunEnd()

OrtStatus * OrtEp::OnRunEnd	(	OrtEp *	this_ptr,
		const OrtRunOptions *	run_options,
		bool	sync_stream
	)

Called by ORT to notify the EP of the end of a run.

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	run_options	The run options for this run.
[in]	sync_stream	Whether any associated stream should be synchronized during this call. Only applicable if there is such a stream.

Note: Implementation of this function is optional.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ OnRunStart()

OrtStatus * OrtEp::OnRunStart	(	OrtEp *	this_ptr,
		const OrtRunOptions *	run_options
	)

Called by ORT to notify the EP of the start of a run.

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	run_options	The run options for this run.

Note: Implementation of this function is optional.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ SetDynamicOptions()

OrtStatus * OrtEp::SetDynamicOptions	(	OrtEp *	this_ptr,
		const char const	option_keys,
		const char const	option_values,
		size_t	num_options
	)

Set dynamic options on this EP.

Dynamic options can be set by the user at any time after session creation with OrtApi::SetEpDynamicOptions().

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	option_keys	The dynamic option keys.
[in]	option_values	The dynamic option values.
[in]	num_options	The number of dynamic options.

Note: Implementation of this function is optional. An EP should only implement this if it needs to handle any dynamic options.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ ShouldConvertDataLayoutForOp()

OrtStatus * OrtEp::ShouldConvertDataLayoutForOp	(	OrtEp *	this_ptr,
		const char *	domain,
		const char *	op_type,
		OrtEpDataLayout	target_data_layout,
		int *	should_convert
	)

Given an op with domain domain and type op_type, determine whether an associated node's data layout should be converted to target_data_layout. If the EP prefers a non-default data layout (see GetPreferredDataLayout()), this function will be called during layout transformation with target_data_layout set to the EP's preferred data layout.

Note: Implementation of this function is optional. If an EP prefers a non-default data layout, it may implement this to customize the specific op data layout preferences at a finer granularity.

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	domain	The op domain. An empty string means the ONNX domain.
[in]	op_type	The op type.
[in]	target_data_layout	The target data layout.
[out]	should_convert	Whether the associated node's data layout should be converted to `target_data_layout`. If greater than 0, convert. If 0, don't convert. Otherwise, if less than 0, leave the decision to ORT.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

Member Data Documentation

◆ GetCompiledModelCompatibilityInfo

const char *( * OrtEp::GetCompiledModelCompatibilityInfo) (OrtEp *this_ptr, const OrtGraph *graph)

Get a string with details about the EP stack used to produce a compiled model.

This function gets a compatibility information string that contains details about the execution provider used to compile a given model. This string can later be used with ValidateCompiledModelCompatibilityInfo to determine if a compiled model is compatible with the EP.

The returned string should be a null-terminated, UTF-8 encoded string. ORT will copy it.

Parameters

[in]	this_ptr	The OrtEp instance.
[in]	graph	The OrtGraph instance for which to generate compatibility information.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.23.

◆ GetName

const char *( * OrtEp::GetName) (const OrtEp *this_ptr)

Get the execution provider name.

The returned string should be a null-terminated, UTF-8 encoded string. ORT will copy it.

Parameters

[in] this_ptr The OrtEp instance.

Returns: The execution provider name.

Since: Version 1.22.

◆ ort_version_supported

uint32_t OrtEp::ort_version_supported

The ONNX Runtime version the execution provider was compiled with.

Implementation should set to ORT_API_VERSION. ORT will use this to ensure it does not call functions that were not available when the library was compiled.

Since: Version 1.22.

◆ ReleaseNodeComputeInfos

void( * OrtEp::ReleaseNodeComputeInfos) (OrtEp *this_ptr, OrtNodeComputeInfo **node_compute_infos, size_t num_node_compute_infos)

Release OrtNodeComputeInfo instances.

Parameters

[in]	this_ptr	The OrtEp instance.
[in,out]	node_compute_infos	The OrtNodeComputeInfo instances to release.
[in]	num_node_compute_infos	The number of OrtNodeComputeInfo instances.

Since: Version 1.23.

Public Member Functions

Public Attributes

Detailed Description

Member Function Documentation

◆ Compile()

◆ CreateAllocator()

◆ CreateSyncStreamForDevice()

◆ GetCapability()

◆ GetPreferredDataLayout()

◆ OnRunEnd()

◆ OnRunStart()

◆ SetDynamicOptions()

◆ ShouldConvertDataLayoutForOp()

Member Data Documentation

◆ GetCompiledModelCompatibilityInfo

◆ GetName

◆ ort_version_supported

◆ ReleaseNodeComputeInfos