OrtDataTransferImpl Struct Reference

Struct that an EP implements for IDataTransfer to copy between devices it uses and CPU. More...

#include <onnxruntime_ep_c_api.h>

Public Member Functions
OrtStatus *	CopyTensors (OrtDataTransferImpl *this_ptr, const OrtValue **src_tensors, OrtValue **dst_tensors, OrtSyncStream **streams, size_t num_tensors)
	Copy tensors from src_tensors to dst_tensors using the provided streams.

Public Attributes
uint32_t	ort_version_supported
	Must be initialized to ORT_API_VERSION.
void(*	Release )(OrtDataTransferImpl *this_ptr)
	Release the OrtDataTransferImpl instance.
bool(*	CanCopy )(const OrtDataTransferImpl *this_ptr, const OrtMemoryDevice *src_memory_device, const OrtMemoryDevice *dst_memory_device)
	Check if the implementation can copy between the source and destination memory devices.

Detailed Description

Struct that an EP implements for IDataTransfer to copy between devices it uses and CPU.

Since

Version 1.23.

Member Function Documentation

CopyTensors()

OrtStatus * OrtDataTransferImpl::CopyTensors	(	OrtDataTransferImpl *	this_ptr,
		const OrtValue **	src_tensors,
		OrtValue **	dst_tensors,
		OrtSyncStream **	streams,
		size_t	num_tensors
	)

Copy tensors from src_tensors to dst_tensors using the provided streams.

The implementation can use the provided streams to perform asynchronous copies if supported. If a stream is not available, the copy is performed synchronously.

Parameters

[in]	this_ptr	Pointer to the OrtDataTransferImpl instance.
[in]	src_tensors	Array of source OrtValue pointers to copy from.
[in]	dst_tensors	Array of destination OrtValue pointers to copy to.
[in]	streams	Array of OrtSyncStream pointers for the copy operations, if the execution provider is stream aware. nullptr if it is not.
[in]	num_tensors	Number of tensors to copy.

Returns

If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since

Version 1.23.

Member Data Documentation

CanCopy

bool( * OrtDataTransferImpl::CanCopy) (const OrtDataTransferImpl *this_ptr, const OrtMemoryDevice *src_memory_device, const OrtMemoryDevice *dst_memory_device)

Check if the implementation can copy between the source and destination memory devices.

Parameters

[in]	this_ptr	Pointer to the OrtDataTransferImpl instance.
[in]	src_memory_device	Source OrtMemoryDevice to copy from.
[in]	dst_memory_device	Destination OrtMemoryDevice to copy to.

Returns

True if the implementation can copy between the devices.

Since

Version 1.23.

ort_version_supported

uint32_t OrtDataTransferImpl::ort_version_supported

Must be initialized to ORT_API_VERSION.

Release

void( * OrtDataTransferImpl::Release) (OrtDataTransferImpl *this_ptr)

Release the OrtDataTransferImpl instance.

This is called by ORT when the OrtDataTransferImpl instance is no longer needed. The implementation should release any resources held by the instance.

Parameters

[in]

this_ptr

Pointer to the OrtDataTransferImpl instance.

Since

Version 1.23.