Struct that an EP implements for profiling support. More...

#include <onnxruntime_ep_c_api.h>

Public Member Functions
OrtStatus *	StartProfiling (OrtEpProfilerImpl *this_ptr, int64_t ep_profiling_start_offset_ns)
	Called when profiling starts.

OrtStatus *	StartEvent (OrtEpProfilerImpl *this_ptr, uint64_t ort_event_correlation_id)
	Called when an ORT event (e.g., session initialization, node kernel execution, etc.) begins.

OrtStatus *	StopEvent (OrtEpProfilerImpl this_ptr, uint64_t ort_event_correlation_id, const OrtProfilingEvent ort_event)
	Called when a profiled ORT event (e.g., session initialization, node kernel execution, etc.) ends.

OrtStatus *	EndProfiling (OrtEpProfilerImpl this_ptr, OrtProfilingEventsContainer events_container)
	Called when profiling ends to collect the EP's new profiling events since the call to StartProfiling.

Public Attributes
uint32_t	ort_version_supported
	Must be initialized to ORT_API_VERSION.

void(*	Release )(OrtEpProfilerImpl *this_ptr)
	Release the OrtEpProfilerImpl instance.

Detailed Description

Struct that an EP implements for profiling support.

An execution provider optionally implements this struct to participate in ONNX Runtime's profiling system. The EP creates and returns an instance of this struct via OrtEp::CreateProfiler.

ORT calls the function pointers at appropriate times during a profiling session:

StartProfiling once when profiling begins.
[Optional] StartEvent / StopEvent around each ORT event (operator executions, session events, etc.).
EndProfiling once when profiling ends to collect EP events.
Release when ORT no longer needs the profiler.

Profiling scenarios:

ORT session profiling: Captures session initialization and one or more runs with a single ORT session.
- Enabled via OrtApi::EnableProfiling(session_options).
- There is one ORT session, one OrtEp, and one OrtEpProfilerImpl
- Concurrency notes: An application may use a single ORT session (and the single OrtEpProfilerImpl) to run multiple inferences concurrently. The OrtEpProfilerImpl::StartEvent and OrtEpProfilerImpl::StopEvent functions will be called with ORT events from multiple concurrent runs. The OrtEpProfilerImpl::EndProfiling function is expected to return all EP events from all runs with correct correlations with the original ORT events.
ORT run profiling: Captures events for a given run. An application can either enable session profiling or run profiling, but not both at the same time.
- Enabled via OrtApi::RunOptionsEnableProfiling(run_options).
- There is one ORT session, one OrtEp, and multiple OrtEpProfilerImpl instances (one per profiled run).
- Concurrency notes: each OrtEpProfilerImpl only receives calls for its specific run. OrtEpProfilerImpl::EndProfiling must only return EP events for its specific run.

Since: Version 1.25.

Member Function Documentation

◆ EndProfiling()

OrtStatus * OrtEpProfilerImpl::EndProfiling	(	OrtEpProfilerImpl *	this_ptr,
		OrtProfilingEventsContainer *	events_container
	)

Called when profiling ends to collect the EP's new profiling events since the call to StartProfiling.

An EP profiler converts its events to OrtProfilingEvent instances and adds them into the provided OrtProfilingEventsContainer container. Call OrtEpApi::CreateProfilingEvent to create a new OrtProfilingEvent instance. Then call OrtEpApi::ProfilingEventsContainer_AddEvents to add one or more events to the container.

After this function returns, ORT appends the EP's events to the profiling timeline.

Parameters

[in]	this_ptr	The OrtEpProfilerImpl instance.
[in]	events_container	Event container to which the EP profiler adds its new events.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Note: An error OrtStatus returned from this function is logged by ORT (does not end execution).; Implementation of this function is required.

Since: Version 1.25.

◆ StartEvent()

OrtStatus * OrtEpProfilerImpl::StartEvent	(	OrtEpProfilerImpl *	this_ptr,
		uint64_t	ort_event_correlation_id
	)

Called when an ORT event (e.g., session initialization, node kernel execution, etc.) begins.

ORT pairs every StartEvent call with a corresponding call to StopEvent with the same ORT event correlation ID. EP profiler implementations may use the calls to StartEvent and StopEvent to maintain a stack of ORT event correlation IDs that can be correlated with EP events (e.g., GPU kernel events). For example:

OrtEpProfilerImpl::StartEvent(x) -> EP ort event stack: [x] <- top of stack [EP events are tagged with 'x'] OrtEpProfilerImpl::StartEvent(y) -> EP ort event stack: [x, y] <- top of stack [EP events are tagged with 'y'] OrtEpProfilerImpl::StopEvent(y) -> EP ort event stack: [x] <- top of stack OrtEpProfilerImpl::StartEvent(z) -> EP ort event stack: [x, z] <- top of stack [EP events are tagged with 'z'] OrtEpProfilerImpl::StopEvent(z) -> EP ort event stack: [x] <- top of stack OrtEpProfilerImpl::StopEvent(x) -> EP ort event stack: [ ] <- top of stack

Tagging EP events with the ORT event correlation ID enables the EP to annotate its own events with metadata from the parent ORT event (e.g., operator name).

Note: Threading: For a given ORT event, StartEvent, the kernel's Compute() entry point, and StopEvent are all invoked on the same CPU thread (Compute() may asynchronously launch device work). Different ORT events may run on different threads (e.g., inter-op parallelism), so correlation stacks (e.g., for CUPTI) should use thread-local storage.; The ORT event correlation ID is an absolute, epoch-based timestamp in microseconds. It is computed from the ORT event's start time using std::chrono::high_resolution_clock (platform-defined epoch). Because it is absolute rather than relative to profiling start, it is practically unique across concurrent profiling sessions within the same process (collisions require sub-microsecond event concurrency) and can be used directly as a correlation ID for EP profiling utilities (e.g., CUPTI or ROCTracer).

Parameters

[in]	this_ptr	Pointer to the OrtEpProfilerImpl instance.
[in]	ort_event_correlation_id	Absolute, epoch-based correlation ID for the ORT event that is starting. The same value is passed to the corresponding StopEvent call.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Note: An error OrtStatus returned from this function is logged by ORT (does not end execution).; Implementation of this function is optional. If set to NULL, it is not called.

Since: Version 1.25.

◆ StartProfiling()

OrtStatus * OrtEpProfilerImpl::StartProfiling	(	OrtEpProfilerImpl *	this_ptr,
		int64_t	ep_profiling_start_offset_ns
	)

Called when profiling starts.

Allows the EP profiler to initialize profiling utilities and record the profiling start time.

An EP profiler should record its own clock's current time when this function is called. This allows the EP to later compute ORT-relative event timestamps by combining ep_profiling_start_offset_ns with the EP's own elapsed time since this call. The formula is:

event_timestamp_us = (ep_profiling_start_offset_ns + (ep_event_time_ns - ep_profiling_start_time_ns)) / 1000

where ep_event_time_ns and ep_profiling_start_time_ns are measured using the EP's own clock.

Parameters

[in]	this_ptr	Pointer to the OrtEpProfilerImpl instance.
[in]	ep_profiling_start_offset_ns	The elapsed time in nanoseconds (using ORT's profiling clock) between ORT's profiling start and this call to StartProfiling.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Note: An error OrtStatus returned from this function is logged by ORT (does not end execution).; Implementation of this function is required.

Since: Version 1.25.

◆ StopEvent()

OrtStatus * OrtEpProfilerImpl::StopEvent	(	OrtEpProfilerImpl *	this_ptr,
		uint64_t	ort_event_correlation_id,
		const OrtProfilingEvent *	ort_event
	)

Called when a profiled ORT event (e.g., session initialization, node kernel execution, etc.) ends.

ORT pairs every StartEvent call with a corresponding call to StopEvent with the same ORT event correlation ID. EP profiler implementations may use the calls to StartEvent and StopEvent to maintain a stack of ORT event correlation IDs that can be correlated with EP events (e.g., GPU kernel events). For example:

OrtEpProfilerImpl::StartEvent(x) -> EP ort event stack: [x] <- top of stack [EP events are tagged with 'x'] OrtEpProfilerImpl::StartEvent(y) -> EP ort event stack: [x, y] <- top of stack [EP events are tagged with 'y'] OrtEpProfilerImpl::StopEvent(y) -> EP ort event stack: [x] <- top of stack OrtEpProfilerImpl::StartEvent(z) -> EP ort event stack: [x, z] <- top of stack [EP events are tagged with 'z'] OrtEpProfilerImpl::StopEvent(z) -> EP ort event stack: [x] <- top of stack OrtEpProfilerImpl::StopEvent(x) -> EP ort event stack: [ ] <- top of stack

Tagging EP events with the ORT event correlation ID enables the EP to annotate its own events with metadata from the parent ORT event (e.g., operator name).

Note: Threading: For a given ORT event, StartEvent, the kernel's Compute() entry point, and StopEvent are all invoked on the same CPU thread (Compute() may asynchronously launch device work). Different ORT events may run on different threads (e.g., inter-op parallelism), so correlation stacks (e.g., for CUPTI) should use thread-local storage.; The ORT event correlation ID is an absolute, epoch-based timestamp in microseconds. It is computed from the ORT event's start time using std::chrono::high_resolution_clock (platform-defined epoch). Because it is absolute rather than relative to profiling start, it is practically unique across concurrent profiling sessions within the same process (collisions require sub-microsecond event concurrency) and can be used directly as a correlation ID for EP profiling utilities (e.g., CUPTI or ROCTracer).

Parameters

[in]	this_ptr	Pointer to the OrtEpProfilerImpl instance.
[in]	ort_event_correlation_id	Absolute, epoch-based correlation ID for the ORT event that is ending. The same value was passed to the corresponding StartEvent call.
[in]	ort_event	Opaque pointer to the ORT profiling event. Valid only during this call. Use OrtEpApi accessor functions to read event fields.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Note: An error OrtStatus returned from this function is logged by ORT (does not end execution).; Implementation of this function is optional. If set to NULL, it is not called.

Since: Version 1.25.

Member Data Documentation

◆ ort_version_supported

uint32_t OrtEpProfilerImpl::ort_version_supported

Must be initialized to ORT_API_VERSION.

◆ Release

void( * OrtEpProfilerImpl::Release) (OrtEpProfilerImpl *this_ptr)

Release the OrtEpProfilerImpl instance.

Called by ORT when the profiler is no longer needed. The implementation should release any resources held by the instance.

Parameters

[in] this_ptr Pointer to the OrtEpProfilerImpl instance.

Note: Implementation of this function is required.

Since: Version 1.25.

Public Member Functions

Public Attributes

Detailed Description

Member Function Documentation

◆ EndProfiling()

◆ StartEvent()

◆ StartProfiling()

◆ StopEvent()

Member Data Documentation

◆ ort_version_supported

◆ Release