ONNX Runtime
Loading...
Searching...
No Matches
OrtApi Struct Reference

The C API. More...

#include <onnxruntime_c_api.h>

Public Member Functions

OrtStatusSynchronizeBoundInputs (OrtIoBinding *binding_ptr)
 Synchronize bound inputs. The call may be necessary for some providers, such as cuda, in case the system that allocated bound memory operated on a different stream. However, the operation is provider specific and could be a no-op.
 
OrtStatusSynchronizeBoundOutputs (OrtIoBinding *binding_ptr)
 Synchronize bound outputs. The call may be necessary for some providers, such as cuda, in case the system that allocated bound memory operated on a different stream. However, the operation is provider specific and could be a no-op.
 
OrtStatusSessionOptionsAppendExecutionProvider_MIGraphX (OrtSessionOptions *options, const OrtMIGraphXProviderOptions *migraphx_options)
 Append MIGraphX provider to session options.
 
OrtStatusAddExternalInitializers (OrtSessionOptions *options, const char *const *initializer_names, const OrtValue *const *initializers, size_t num_initializers)
 Replace initialized Tensors with external data with the data provided in initializers.
 
OrtStatusCreateOpAttr (const char *name, const void *data, int len, OrtOpAttrType type, OrtOpAttr **op_attr)
 : Create attribute of onnxruntime operator
 
void ReleaseOpAttr (OrtOpAttr *input)
 
OrtStatusCreateOp (const OrtKernelInfo *info, const char *op_name, const char *domain, int version, const char **type_constraint_names, const ONNXTensorElementDataType *type_constraint_values, int type_constraint_count, const OrtOpAttr *const *attr_values, int attr_count, int input_count, int output_count, OrtOp **ort_op)
 : Create onnxruntime native operator
 
OrtStatusInvokeOp (const OrtKernelContext *context, const OrtOp *ort_op, const OrtValue *const *input_values, int input_count, OrtValue *const *output_values, int output_count)
 : Invoke the operator created by OrtApi::CreateOp The inputs must follow the order as specified in onnx specification
 
void ReleaseOp (OrtOp *input)
 
OrtStatusSessionOptionsAppendExecutionProvider (OrtSessionOptions *options, const char *provider_name, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 : Append execution provider to the session options.
 
OrtStatusCopyKernelInfo (const OrtKernelInfo *info, OrtKernelInfo **info_copy)
 
void ReleaseKernelInfo (OrtKernelInfo *input)
 
OrtStatusSessionOptionsAppendExecutionProvider_CANN (OrtSessionOptions *options, const OrtCANNProviderOptions *cann_options)
 Append CANN provider to session options.
 
OrtStatusCreateCANNProviderOptions (OrtCANNProviderOptions **out)
 Create an OrtCANNProviderOptions.
 
OrtStatusUpdateCANNProviderOptions (OrtCANNProviderOptions *cann_options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Set options in a CANN Execution Provider.
 
OrtStatusGetCANNProviderOptionsAsString (const OrtCANNProviderOptions *cann_options, OrtAllocator *allocator, char **ptr)
 Get serialized CANN provider options string.
 
OrtStatusUpdateEnvWithCustomLogLevel (OrtEnv *ort_env, OrtLoggingLevel log_severity_level)
 
OrtStatusSetGlobalIntraOpThreadAffinity (OrtThreadingOptions *tp_options, const char *affinity_string)
 
OrtStatusRegisterCustomOpsLibrary_V2 (OrtSessionOptions *options, const char *library_name)
 Register custom ops from a shared library.
 
OrtStatusRegisterCustomOpsUsingFunction (OrtSessionOptions *options, const char *registration_func_name)
 Register custom ops by calling a RegisterCustomOpsFn function.
 
OrtStatusSessionOptionsAppendExecutionProvider_Dnnl (OrtSessionOptions *options, const OrtDnnlProviderOptions *dnnl_options)
 Append dnnl provider to session options.
 
OrtStatusCreateDnnlProviderOptions (OrtDnnlProviderOptions **out)
 Create an OrtDnnlProviderOptions.
 
OrtStatusUpdateDnnlProviderOptions (OrtDnnlProviderOptions *dnnl_options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Set options in a oneDNN Execution Provider.
 
OrtStatusGetDnnlProviderOptionsAsString (const OrtDnnlProviderOptions *dnnl_options, OrtAllocator *allocator, char **ptr)
 
OrtStatusKernelInfoGetConstantInput_tensor (const OrtKernelInfo *info, size_t index, int *is_constant, const OrtValue **out)
 Get a OrtValue tensor stored as a constant initializer in the graph node.
 
OrtStatusCastTypeInfoToOptionalTypeInfo (const OrtTypeInfo *type_info, const OrtOptionalTypeInfo **out)
 Get Optional Type information from an OrtTypeInfo.
 
OrtStatusGetOptionalContainedTypeInfo (const OrtOptionalTypeInfo *optional_type_info, OrtTypeInfo **out)
 Get OrtTypeInfo for the allowed contained type from an OrtOptionalTypeInfo.
 
OrtStatusGetResizedStringTensorElementBuffer (OrtValue *value, size_t index, size_t length_in_bytes, char **buffer)
 Set a single string in a string tensor Do not zero terminate the string data.
 
OrtStatusKernelContext_GetAllocator (const OrtKernelContext *context, const OrtMemoryInfo *mem_info, OrtAllocator **out)
 Get Allocator from KernelContext for a specific memoryInfo. Please use C API ReleaseAllocator to release out object.
 

Public Attributes

void(* ReleaseCANNProviderOptions )(OrtCANNProviderOptions *input)
 Release an OrtCANNProviderOptions.
 
void(* MemoryInfoGetDeviceType )(const OrtMemoryInfo *ptr, OrtMemoryInfoDeviceType *out)
 
void(* ReleaseDnnlProviderOptions )(OrtDnnlProviderOptions *input)
 Release an OrtDnnlProviderOptions.
 
const char *(* GetBuildInfoString )(void)
 Returns a null terminated string of the build info including git info and cxx flags.
 

OrtStatus

OrtStatus *(* CreateStatus )(OrtErrorCode code, const char *msg) __attribute__((nonnull))
 Create an OrtStatus from a null terminated string.
 
OrtErrorCode(* GetErrorCode )(const OrtStatus *status) __attribute__((nonnull))
 Get OrtErrorCode from OrtStatus.
 
const char *(* GetErrorMessage )(const OrtStatus *status) __attribute__((nonnull))
 Get error string from OrtStatus.
 
void ReleaseStatus (OrtStatus *input)
 

OrtIoBinding

void(* ClearBoundInputs )(OrtIoBinding *binding_ptr) __attribute__((nonnull))
 Clears any previously set Inputs for an OrtIoBinding.
 
void(* ClearBoundOutputs )(OrtIoBinding *binding_ptr) __attribute__((nonnull))
 Clears any previously set Outputs for an OrtIoBinding.
 
void ReleaseIoBinding (OrtIoBinding *input)
 Release an OrtIoBinding obtained from OrtApi::CreateIoBinding.
 
OrtStatusBindInput (OrtIoBinding *binding_ptr, const char *name, const OrtValue *val_ptr)
 Bind an OrtValue to an OrtIoBinding input.
 
OrtStatusBindOutput (OrtIoBinding *binding_ptr, const char *name, const OrtValue *val_ptr)
 Bind an OrtValue to an OrtIoBinding output.
 
OrtStatusBindOutputToDevice (OrtIoBinding *binding_ptr, const char *name, const OrtMemoryInfo *mem_info_ptr)
 Bind an OrtIoBinding output to a device.
 
OrtStatusGetBoundOutputNames (const OrtIoBinding *binding_ptr, OrtAllocator *allocator, char **buffer, size_t **lengths, size_t *count)
 Get the names of an OrtIoBinding's outputs.
 
OrtStatusGetBoundOutputValues (const OrtIoBinding *binding_ptr, OrtAllocator *allocator, OrtValue ***output, size_t *output_count)
 Get the output OrtValue objects from an OrtIoBinding.
 

OrtTensorRTProviderOptionsV2

void(* ReleaseTensorRTProviderOptions )(OrtTensorRTProviderOptionsV2 *input)
 Release an OrtTensorRTProviderOptionsV2.
 
OrtStatusCreateTensorRTProviderOptions (OrtTensorRTProviderOptionsV2 **out)
 Create an OrtTensorRTProviderOptionsV2.
 
OrtStatusUpdateTensorRTProviderOptions (OrtTensorRTProviderOptionsV2 *tensorrt_options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Set options in a TensorRT Execution Provider.
 
OrtStatusGetTensorRTProviderOptionsAsString (const OrtTensorRTProviderOptionsV2 *tensorrt_options, OrtAllocator *allocator, char **ptr)
 Get serialized TensorRT provider options string.
 

OrtCUDAProviderOptionsV2

void(* ReleaseCUDAProviderOptions )(OrtCUDAProviderOptionsV2 *input)
 Release an OrtCUDAProviderOptionsV2.
 
OrtStatusCreateCUDAProviderOptions (OrtCUDAProviderOptionsV2 **out)
 Create an OrtCUDAProviderOptionsV2.
 
OrtStatusUpdateCUDAProviderOptions (OrtCUDAProviderOptionsV2 *cuda_options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Set options in a CUDA Execution Provider.
 
OrtStatusGetCUDAProviderOptionsAsString (const OrtCUDAProviderOptionsV2 *cuda_options, OrtAllocator *allocator, char **ptr)
 

Ort Training

const OrtTrainingApi *(* GetTrainingApi )(uint32_t version)
 Gets the Training C Api struct.
 

OrtROCMProviderOptions

void(* ReleaseROCMProviderOptions )(OrtROCMProviderOptions *input)
 Release an OrtROCMProviderOptions.
 
OrtStatusCreateROCMProviderOptions (OrtROCMProviderOptions **out)
 Create an OrtROCMProviderOptions.
 
OrtStatusUpdateROCMProviderOptions (OrtROCMProviderOptions *rocm_options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Set options in a ROCm Execution Provider.
 
OrtStatusGetROCMProviderOptionsAsString (const OrtROCMProviderOptions *rocm_options, OrtAllocator *allocator, char **ptr)
 
OrtStatusCreateAndRegisterAllocatorV2 (OrtEnv *env, const char *provider_type, const OrtMemoryInfo *mem_info, const OrtArenaCfg *arena_cfg, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Create an allocator with specific type and register it with the OrtEnv This API enhance CreateAndRegisterAllocator that it can create an allocator with specific type, not just CPU allocator Enables sharing the allocator between multiple sessions that use the same env instance. Lifetime of the created allocator will be valid for the duration of the environment. Returns an error if an allocator with the same OrtMemoryInfo is already registered.
 
OrtStatusRunAsync (OrtSession *session, const OrtRunOptions *run_options, const char *const *input_names, const OrtValue *const *input, size_t input_len, const char *const *output_names, size_t output_names_len, OrtValue **output, RunAsyncCallbackFn run_async_callback, void *user_data)
 Run the model asynchronously in a thread owned by intra op thread pool.
 
OrtStatusUpdateTensorRTProviderOptionsWithValue (OrtTensorRTProviderOptionsV2 *tensorrt_options, const char *key, void *value)
 
OrtStatusGetTensorRTProviderOptionsByName (const OrtTensorRTProviderOptionsV2 *tensorrt_options, const char *key, void **ptr)
 
OrtStatusUpdateCUDAProviderOptionsWithValue (OrtCUDAProviderOptionsV2 *cuda_options, const char *key, void *value)
 
OrtStatusGetCUDAProviderOptionsByName (const OrtCUDAProviderOptionsV2 *cuda_options, const char *key, void **ptr)
 
OrtStatusKernelContext_GetResource (const OrtKernelContext *context, int resource_version, int resource_id, void **resource)
 
OrtStatusSetUserLoggingFunction (OrtSessionOptions *options, OrtLoggingFunction user_logging_function, void *user_logging_param)
 Set user logging function.
 
OrtStatusShapeInferContext_GetInputCount (const OrtShapeInferContext *context, size_t *out)
 
OrtStatusShapeInferContext_GetInputTypeShape (const OrtShapeInferContext *context, size_t index, OrtTensorTypeAndShapeInfo **info)
 
OrtStatusShapeInferContext_GetAttribute (const OrtShapeInferContext *context, const char *attr_name, const OrtOpAttr **attr)
 
OrtStatusShapeInferContext_SetOutputTypeShape (const OrtShapeInferContext *context, size_t index, const OrtTensorTypeAndShapeInfo *info)
 
OrtStatusSetSymbolicDimensions (OrtTensorTypeAndShapeInfo *info, const char *dim_params[], size_t dim_params_length)
 
OrtStatusReadOpAttr (const OrtOpAttr *op_attr, OrtOpAttrType type, void *data, size_t len, size_t *out)
 
OrtStatusSetDeterministicCompute (OrtSessionOptions *options, bool value)
 Set whether to use deterministic compute.
 
OrtStatusKernelContext_ParallelFor (const OrtKernelContext *context, void(*fn)(void *, size_t), size_t total, size_t num_batch, void *usr_data)
 
OrtStatusSessionOptionsAppendExecutionProvider_OpenVINO_V2 (OrtSessionOptions *options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Append OpenVINO execution provider to the session options.
 
OrtStatusSessionOptionsAppendExecutionProvider_VitisAI (OrtSessionOptions *options, const char *const *provider_options_keys, const char *const *provider_options_values, size_t num_keys)
 Append VitisAI provider to session options.
 
OrtStatusKernelContext_GetScratchBuffer (const OrtKernelContext *context, const OrtMemoryInfo *mem_info, size_t count_or_bytes, void **out)
 Get scratch buffer from the corresponding allocator under the sepcific OrtMemoryInfo object. NOTE: callers are responsible to release this scratch buffer from the corresponding allocator.
 
OrtStatusKernelInfoGetAllocator (const OrtKernelInfo *info, OrtMemType mem_type, OrtAllocator **out)
 Get allocator from KernelInfo for a specific memory type. Please use C API ReleaseAllocator to release out object.
 
OrtStatusAddExternalInitializersFromFilesInMemory (OrtSessionOptions *options, const char *const *external_initializer_file_names, char *const *external_initializer_file_buffer_array, const size_t *external_initializer_file_lengths, size_t num_external_initializer_files)
 Replace initialized Tensors with external data with the provided files in memory.
 
OrtStatusCreateLoraAdapter (const char *adapter_file_path, OrtAllocator *allocator, OrtLoraAdapter **out)
 Create an OrtLoraAdapter.
 
OrtStatusCreateLoraAdapterFromArray (const void *bytes, size_t num_bytes, OrtAllocator *allocator, OrtLoraAdapter **out)
 Create an OrtLoraAdapter.
 
void ReleaseLoraAdapter (OrtLoraAdapter *input)
 Release an OrtLoraAdapter obtained from OrtApi::CreateLoraAdapter.
 
OrtStatusRunOptionsAddActiveLoraAdapter (OrtRunOptions *options, const OrtLoraAdapter *adapter)
 Add the Lora Adapter to the list of active adapters.
 

OrtEnv

OrtStatusCreateEnv (OrtLoggingLevel log_severity_level, const char *logid, OrtEnv **out)
 Create an OrtEnv.
 
OrtStatusCreateEnvWithCustomLogger (OrtLoggingFunction logging_function, void *logger_param, OrtLoggingLevel log_severity_level, const char *logid, OrtEnv **out)
 Create an OrtEnv.
 
OrtStatusEnableTelemetryEvents (const OrtEnv *env)
 Enable Telemetry.
 
OrtStatusDisableTelemetryEvents (const OrtEnv *env)
 Disable Telemetry.
 
void ReleaseEnv (OrtEnv *input)
 
OrtStatusCreateEnvWithGlobalThreadPools (OrtLoggingLevel log_severity_level, const char *logid, const OrtThreadingOptions *tp_options, OrtEnv **out)
 Create an OrtEnv.
 
OrtStatusCreateAndRegisterAllocator (OrtEnv *env, const OrtMemoryInfo *mem_info, const OrtArenaCfg *arena_cfg)
 Create an allocator and register it with the OrtEnv.
 
OrtStatusSetLanguageProjection (const OrtEnv *ort_env, OrtLanguageProjection projection)
 Set language projection.
 
OrtStatusCreateEnvWithCustomLoggerAndGlobalThreadPools (OrtLoggingFunction logging_function, void *logger_param, OrtLoggingLevel log_severity_level, const char *logid, const struct OrtThreadingOptions *tp_options, OrtEnv **out)
 

OrtSession

OrtStatusCreateSession (const OrtEnv *env, const char *model_path, const OrtSessionOptions *options, OrtSession **out)
 Create an OrtSession from a model file.
 
OrtStatusCreateSessionFromArray (const OrtEnv *env, const void *model_data, size_t model_data_length, const OrtSessionOptions *options, OrtSession **out)
 Create an OrtSession from memory.
 
OrtStatusRun (OrtSession *session, const OrtRunOptions *run_options, const char *const *input_names, const OrtValue *const *inputs, size_t input_len, const char *const *output_names, size_t output_names_len, OrtValue **outputs)
 Run the model in an OrtSession.
 
OrtStatusSessionGetInputCount (const OrtSession *session, size_t *out)
 Get input count for a session.
 
OrtStatusSessionGetOutputCount (const OrtSession *session, size_t *out)
 Get output count for a session.
 
OrtStatusSessionGetOverridableInitializerCount (const OrtSession *session, size_t *out)
 Get overridable initializer count.
 
OrtStatusSessionGetInputTypeInfo (const OrtSession *session, size_t index, OrtTypeInfo **type_info)
 Get input type information.
 
OrtStatusSessionGetOutputTypeInfo (const OrtSession *session, size_t index, OrtTypeInfo **type_info)
 Get output type information.
 
OrtStatusSessionGetOverridableInitializerTypeInfo (const OrtSession *session, size_t index, OrtTypeInfo **type_info)
 Get overridable initializer type information.
 
OrtStatusSessionGetInputName (const OrtSession *session, size_t index, OrtAllocator *allocator, char **value)
 Get input name.
 
OrtStatusSessionGetOutputName (const OrtSession *session, size_t index, OrtAllocator *allocator, char **value)
 Get output name.
 
OrtStatusSessionGetOverridableInitializerName (const OrtSession *session, size_t index, OrtAllocator *allocator, char **value)
 Get overridable initializer name.
 
void ReleaseSession (OrtSession *input)
 
OrtStatusSessionEndProfiling (OrtSession *session, OrtAllocator *allocator, char **out)
 End profiling and return filename of the profile data.
 
OrtStatusSessionGetModelMetadata (const OrtSession *session, OrtModelMetadata **out)
 Get OrtModelMetadata from an OrtSession.
 
OrtStatusRunWithBinding (OrtSession *session, const OrtRunOptions *run_options, const OrtIoBinding *binding_ptr)
 Run a model using Io Bindings for the inputs & outputs.
 
OrtStatusCreateIoBinding (OrtSession *session, OrtIoBinding **out)
 Create an OrtIoBinding instance.
 
OrtStatusSessionGetProfilingStartTimeNs (const OrtSession *session, uint64_t *out)
 Return the time that profiling was started.
 
OrtStatusCreateSessionWithPrepackedWeightsContainer (const OrtEnv *env, const char *model_path, const OrtSessionOptions *options, OrtPrepackedWeightsContainer *prepacked_weights_container, OrtSession **out)
 Create session with prepacked weights container.
 
OrtStatusCreateSessionFromArrayWithPrepackedWeightsContainer (const OrtEnv *env, const void *model_data, size_t model_data_length, const OrtSessionOptions *options, OrtPrepackedWeightsContainer *prepacked_weights_container, OrtSession **out)
 Create session from memory with prepacked weights container.
 

OrtSessionOptions

Custom operator APIs

OrtStatusCreateSessionOptions (OrtSessionOptions **options)
 Create an OrtSessionOptions object.
 
OrtStatusSetOptimizedModelFilePath (OrtSessionOptions *options, const char *optimized_model_filepath)
 Set filepath to save optimized model after graph level transformations.
 
OrtStatusCloneSessionOptions (const OrtSessionOptions *in_options, OrtSessionOptions **out_options)
 Create a copy of an existing OrtSessionOptions.
 
OrtStatusSetSessionExecutionMode (OrtSessionOptions *options, ExecutionMode execution_mode)
 Set execution mode.
 
OrtStatusEnableProfiling (OrtSessionOptions *options, const char *profile_file_prefix)
 Enable profiling for a session.
 
OrtStatusDisableProfiling (OrtSessionOptions *options)
 Disable profiling for a session.
 
OrtStatusEnableMemPattern (OrtSessionOptions *options)
 Enable the memory pattern optimization.
 
OrtStatusDisableMemPattern (OrtSessionOptions *options)
 Disable the memory pattern optimization.
 
OrtStatusEnableCpuMemArena (OrtSessionOptions *options)
 Enable the memory arena on CPU.
 
OrtStatusDisableCpuMemArena (OrtSessionOptions *options)
 Disable the memory arena on CPU.
 
OrtStatusSetSessionLogId (OrtSessionOptions *options, const char *logid)
 Set session log id.
 
OrtStatusSetSessionLogVerbosityLevel (OrtSessionOptions *options, int session_log_verbosity_level)
 Set session log verbosity level.
 
OrtStatusSetSessionLogSeverityLevel (OrtSessionOptions *options, int session_log_severity_level)
 Set session log severity level.
 
OrtStatusSetSessionGraphOptimizationLevel (OrtSessionOptions *options, GraphOptimizationLevel graph_optimization_level)
 Set the optimization level to apply when loading a graph.
 
OrtStatusSetIntraOpNumThreads (OrtSessionOptions *options, int intra_op_num_threads)
 Sets the number of threads used to parallelize the execution within nodes.
 
OrtStatusSetInterOpNumThreads (OrtSessionOptions *options, int inter_op_num_threads)
 Sets the number of threads used to parallelize the execution of the graph.
 
OrtStatusAddCustomOpDomain (OrtSessionOptions *options, OrtCustomOpDomain *custom_op_domain)
 Add custom op domain to a session options.
 
OrtStatusRegisterCustomOpsLibrary (OrtSessionOptions *options, const char *library_path, void **library_handle)
 
OrtStatusAddFreeDimensionOverride (OrtSessionOptions *options, const char *dim_denotation, int64_t dim_value)
 Override session symbolic dimensions.
 
void ReleaseSessionOptions (OrtSessionOptions *input)
 
OrtStatusDisablePerSessionThreads (OrtSessionOptions *options)
 Use global thread pool on a session.
 
OrtStatusAddFreeDimensionOverrideByName (OrtSessionOptions *options, const char *dim_name, int64_t dim_value)
 
OrtStatusAddSessionConfigEntry (OrtSessionOptions *options, const char *config_key, const char *config_value)
 Set a session configuration entry as a pair of strings.
 
OrtStatusAddInitializer (OrtSessionOptions *options, const char *name, const OrtValue *val)
 Add a pre-allocated initializer to a session.
 
OrtStatusSessionOptionsAppendExecutionProvider_CUDA (OrtSessionOptions *options, const OrtCUDAProviderOptions *cuda_options)
 Append CUDA provider to session options.
 
OrtStatusSessionOptionsAppendExecutionProvider_ROCM (OrtSessionOptions *options, const OrtROCMProviderOptions *rocm_options)
 Append ROCM execution provider to the session options.
 
OrtStatusSessionOptionsAppendExecutionProvider_OpenVINO (OrtSessionOptions *options, const OrtOpenVINOProviderOptions *provider_options)
 Append OpenVINO execution provider to the session options.
 
OrtStatusSessionOptionsAppendExecutionProvider_TensorRT (OrtSessionOptions *options, const OrtTensorRTProviderOptions *tensorrt_options)
 Append TensorRT provider to session options.
 
OrtStatusSessionOptionsAppendExecutionProvider_TensorRT_V2 (OrtSessionOptions *options, const OrtTensorRTProviderOptionsV2 *tensorrt_options)
 Append TensorRT execution provider to the session options.
 
OrtStatusEnableOrtCustomOps (OrtSessionOptions *options)
 Enable custom operators.
 
OrtStatusHasValue (const OrtValue *value, int *out)
 Sets out to 1 iff an optional type OrtValue has an element, 0 otherwise (OrtValue is None) Use this API to find if the optional type OrtValue is None or not. If the optional type OrtValue is not None, use the OrtValue just like any other OrtValue. For example, if you get an OrtValue that corresponds to Optional(tensor) and if HasValue() returns true, use it as tensor and so on.
 
OrtStatusSessionOptionsAppendExecutionProvider_CUDA_V2 (OrtSessionOptions *options, const OrtCUDAProviderOptionsV2 *cuda_options)
 Append CUDA execution provider to the session options.
 
OrtStatusHasSessionConfigEntry (const OrtSessionOptions *options, const char *config_key, int *out)
 Checks if the given session configuration entry exists.
 
OrtStatusGetSessionConfigEntry (const OrtSessionOptions *options, const char *config_key, char *config_value, size_t *size)
 Get a session configuration value.
 

OrtCustomOpDomain

OrtStatusCreateCustomOpDomain (const char *domain, OrtCustomOpDomain **out)
 Create a custom op domain.
 
OrtStatusCustomOpDomain_Add (OrtCustomOpDomain *custom_op_domain, const OrtCustomOp *op)
 Add a custom op to a custom op domain.
 
void ReleaseCustomOpDomain (OrtCustomOpDomain *input)
 

OrtRunOptions

OrtStatusCreateRunOptions (OrtRunOptions **out)
 Create an OrtRunOptions.
 
OrtStatusRunOptionsSetRunLogVerbosityLevel (OrtRunOptions *options, int log_verbosity_level)
 Set per-run log verbosity level.
 
OrtStatusRunOptionsSetRunLogSeverityLevel (OrtRunOptions *options, int log_severity_level)
 Set per-run log severity level.
 
OrtStatusRunOptionsSetRunTag (OrtRunOptions *options, const char *run_tag)
 Set per-run tag.
 
OrtStatusRunOptionsGetRunLogVerbosityLevel (const OrtRunOptions *options, int *log_verbosity_level)
 Get per-run log verbosity level.
 
OrtStatusRunOptionsGetRunLogSeverityLevel (const OrtRunOptions *options, int *log_severity_level)
 Get per-run log severity level.
 
OrtStatusRunOptionsGetRunTag (const OrtRunOptions *options, const char **run_tag)
 Get per-run tag.
 
OrtStatusRunOptionsSetTerminate (OrtRunOptions *options)
 Set terminate flag.
 
OrtStatusRunOptionsUnsetTerminate (OrtRunOptions *options)
 Clears the terminate flag.
 
void ReleaseRunOptions (OrtRunOptions *input)
 
OrtStatusAddRunConfigEntry (OrtRunOptions *options, const char *config_key, const char *config_value)
 Set a single run configuration entry as a pair of strings.
 

OrtValue

OrtStatusCreateTensorAsOrtValue (OrtAllocator *allocator, const int64_t *shape, size_t shape_len, ONNXTensorElementDataType type, OrtValue **out)
 Create a tensor.
 
OrtStatusCreateTensorWithDataAsOrtValue (const OrtMemoryInfo *info, void *p_data, size_t p_data_len, const int64_t *shape, size_t shape_len, ONNXTensorElementDataType type, OrtValue **out)
 Create a tensor backed by a user supplied buffer.
 
OrtStatusIsTensor (const OrtValue *value, int *out)
 Return if an OrtValue is a tensor type.
 
OrtStatusGetTensorMutableData (OrtValue *value, void **out)
 Get a pointer to the raw data inside a tensor.
 
OrtStatusFillStringTensor (OrtValue *value, const char *const *s, size_t s_len)
 Set all strings at once in a string tensor.
 
OrtStatusGetStringTensorDataLength (const OrtValue *value, size_t *len)
 Get total byte length for all strings in a string tensor.
 
OrtStatusGetStringTensorContent (const OrtValue *value, void *s, size_t s_len, size_t *offsets, size_t offsets_len)
 Get all strings from a string tensor.
 
OrtStatusGetTensorTypeAndShape (const OrtValue *value, OrtTensorTypeAndShapeInfo **out)
 Get type and shape information from a tensor OrtValue.
 
OrtStatusGetTypeInfo (const OrtValue *value, OrtTypeInfo **out)
 Get type information of an OrtValue.
 
OrtStatusGetValueType (const OrtValue *value, enum ONNXType *out)
 Get ONNXType of an OrtValue.
 
OrtStatusGetValue (const OrtValue *value, int index, OrtAllocator *allocator, OrtValue **out)
 Get non tensor data from an OrtValue.
 
OrtStatusGetValueCount (const OrtValue *value, size_t *out)
 Get non tensor value count from an OrtValue.
 
OrtStatusCreateValue (const OrtValue *const *in, size_t num_values, enum ONNXType value_type, OrtValue **out)
 Create a map or sequence OrtValue.
 
OrtStatusCreateOpaqueValue (const char *domain_name, const char *type_name, const void *data_container, size_t data_container_size, OrtValue **out)
 Create an opaque (custom user defined type) OrtValue.
 
OrtStatusGetOpaqueValue (const char *domain_name, const char *type_name, const OrtValue *in, void *data_container, size_t data_container_size)
 Get internal data from an opaque (custom user defined type) OrtValue.
 
void ReleaseValue (OrtValue *input)
 
OrtStatusGetStringTensorElementLength (const OrtValue *value, size_t index, size_t *out)
 Get the length of a single string in a string tensor.
 
OrtStatusGetStringTensorElement (const OrtValue *value, size_t s_len, size_t index, void *s)
 Get a single string from a string tensor.
 
OrtStatusFillStringTensorElement (OrtValue *value, const char *s, size_t index)
 Set a single string in a string tensor.
 
OrtStatusTensorAt (OrtValue *value, const int64_t *location_values, size_t location_values_count, void **out)
 Direct memory access to a specified tensor element.
 
OrtStatusIsSparseTensor (const OrtValue *value, int *out)
 Sets *out to 1 iff an OrtValue is a SparseTensor, and 0 otherwise.
 
OrtStatusCreateSparseTensorAsOrtValue (OrtAllocator *allocator, const int64_t *dense_shape, size_t dense_shape_len, ONNXTensorElementDataType type, OrtValue **out)
 Create an OrtValue with a sparse tensor that is empty.
 
OrtStatusFillSparseTensorCoo (OrtValue *ort_value, const OrtMemoryInfo *data_mem_info, const int64_t *values_shape, size_t values_shape_len, const void *values, const int64_t *indices_data, size_t indices_num)
 
OrtStatusFillSparseTensorCsr (OrtValue *ort_value, const OrtMemoryInfo *data_mem_info, const int64_t *values_shape, size_t values_shape_len, const void *values, const int64_t *inner_indices_data, size_t inner_indices_num, const int64_t *outer_indices_data, size_t outer_indices_num)
 
OrtStatusFillSparseTensorBlockSparse (OrtValue *ort_value, const OrtMemoryInfo *data_mem_info, const int64_t *values_shape, size_t values_shape_len, const void *values, const int64_t *indices_shape_data, size_t indices_shape_len, const int32_t *indices_data)
 
OrtStatusCreateSparseTensorWithValuesAsOrtValue (const OrtMemoryInfo *info, void *p_data, const int64_t *dense_shape, size_t dense_shape_len, const int64_t *values_shape, size_t values_shape_len, ONNXTensorElementDataType type, OrtValue **out)
 
OrtStatusUseCooIndices (OrtValue *ort_value, int64_t *indices_data, size_t indices_num)
 
OrtStatusUseCsrIndices (OrtValue *ort_value, int64_t *inner_data, size_t inner_num, int64_t *outer_data, size_t outer_num)
 
OrtStatusUseBlockSparseIndices (OrtValue *ort_value, const int64_t *indices_shape, size_t indices_shape_len, int32_t *indices_data)
 
OrtStatusGetSparseTensorFormat (const OrtValue *ort_value, enum OrtSparseFormat *out)
 Returns sparse tensor format enum iff a given ort value contains an instance of sparse tensor.
 
OrtStatusGetSparseTensorValuesTypeAndShape (const OrtValue *ort_value, OrtTensorTypeAndShapeInfo **out)
 Returns data type and shape of sparse tensor values (nnz) iff OrtValue contains a SparseTensor.
 
OrtStatusGetSparseTensorValues (const OrtValue *ort_value, const void **out)
 Returns numeric data for sparse tensor values (nnz). For string values use GetStringTensor*().
 
OrtStatusGetSparseTensorIndicesTypeShape (const OrtValue *ort_value, enum OrtSparseIndicesFormat indices_format, OrtTensorTypeAndShapeInfo **out)
 Returns data type, shape for the type of indices specified by indices_format.
 
OrtStatusGetSparseTensorIndices (const OrtValue *ort_value, enum OrtSparseIndicesFormat indices_format, size_t *num_indices, const void **indices)
 Returns indices data for the type of the indices specified by indices_format.
 

OrtTypeInfo

OrtStatusCastTypeInfoToTensorInfo (const OrtTypeInfo *type_info, const OrtTensorTypeAndShapeInfo **out)
 Get OrtTensorTypeAndShapeInfo from an OrtTypeInfo.
 
OrtStatusGetOnnxTypeFromTypeInfo (const OrtTypeInfo *type_info, enum ONNXType *out)
 Get ONNXType from OrtTypeInfo.
 
void ReleaseTypeInfo (OrtTypeInfo *input)
 
OrtStatusGetDenotationFromTypeInfo (const OrtTypeInfo *type_info, const char **const denotation, size_t *len)
 Get denotation from type information.
 
OrtStatusCastTypeInfoToMapTypeInfo (const OrtTypeInfo *type_info, const OrtMapTypeInfo **out)
 Get detailed map information from an OrtTypeInfo.
 
OrtStatusCastTypeInfoToSequenceTypeInfo (const OrtTypeInfo *type_info, const OrtSequenceTypeInfo **out)
 Cast OrtTypeInfo to an OrtSequenceTypeInfo.
 

OrtTensorTypeAndShapeInfo

OrtStatusCreateTensorTypeAndShapeInfo (OrtTensorTypeAndShapeInfo **out)
 Create an OrtTensorTypeAndShapeInfo object.
 
OrtStatusSetTensorElementType (OrtTensorTypeAndShapeInfo *info, enum ONNXTensorElementDataType type)
 Set element type in OrtTensorTypeAndShapeInfo.
 
OrtStatusSetDimensions (OrtTensorTypeAndShapeInfo *info, const int64_t *dim_values, size_t dim_count)
 Set shape information in OrtTensorTypeAndShapeInfo.
 
OrtStatusGetTensorElementType (const OrtTensorTypeAndShapeInfo *info, enum ONNXTensorElementDataType *out)
 Get element type in OrtTensorTypeAndShapeInfo.
 
OrtStatusGetDimensionsCount (const OrtTensorTypeAndShapeInfo *info, size_t *out)
 Get dimension count in OrtTensorTypeAndShapeInfo.
 
OrtStatusGetDimensions (const OrtTensorTypeAndShapeInfo *info, int64_t *dim_values, size_t dim_values_length)
 Get dimensions in OrtTensorTypeAndShapeInfo.
 
OrtStatusGetSymbolicDimensions (const OrtTensorTypeAndShapeInfo *info, const char *dim_params[], size_t dim_params_length)
 Get symbolic dimension names in OrtTensorTypeAndShapeInfo.
 
OrtStatusGetTensorShapeElementCount (const OrtTensorTypeAndShapeInfo *info, size_t *out)
 Get total number of elements in a tensor shape from an OrtTensorTypeAndShapeInfo.
 
void ReleaseTensorTypeAndShapeInfo (OrtTensorTypeAndShapeInfo *input)
 

OrtMemoryInfo

OrtStatusCreateMemoryInfo (const char *name, enum OrtAllocatorType type, int id, enum OrtMemType mem_type, OrtMemoryInfo **out)
 Create an OrtMemoryInfo.
 
OrtStatusCreateCpuMemoryInfo (enum OrtAllocatorType type, enum OrtMemType mem_type, OrtMemoryInfo **out)
 Create an OrtMemoryInfo for CPU memory.
 
OrtStatusCompareMemoryInfo (const OrtMemoryInfo *info1, const OrtMemoryInfo *info2, int *out)
 Compare OrtMemoryInfo objects for equality.
 
OrtStatusMemoryInfoGetName (const OrtMemoryInfo *ptr, const char **out)
 Get name from OrtMemoryInfo.
 
OrtStatusMemoryInfoGetId (const OrtMemoryInfo *ptr, int *out)
 Get the id from OrtMemoryInfo.
 
OrtStatusMemoryInfoGetMemType (const OrtMemoryInfo *ptr, OrtMemType *out)
 Get the OrtMemType from OrtMemoryInfo.
 
OrtStatusMemoryInfoGetType (const OrtMemoryInfo *ptr, OrtAllocatorType *out)
 Get the OrtAllocatorType from OrtMemoryInfo.
 
void ReleaseMemoryInfo (OrtMemoryInfo *input)
 

OrtAllocator

OrtStatusAllocatorAlloc (OrtAllocator *ort_allocator, size_t size, void **out)
 Calls OrtAllocator::Alloc function.
 
OrtStatusAllocatorFree (OrtAllocator *ort_allocator, void *p)
 Calls OrtAllocator::Free function.
 
OrtStatusAllocatorGetInfo (const OrtAllocator *ort_allocator, const struct OrtMemoryInfo **out)
 Calls OrtAllocator::Info function.
 
OrtStatusGetAllocatorWithDefaultOptions (OrtAllocator **out)
 Get the default allocator.
 
OrtStatusCreateAllocator (const OrtSession *session, const OrtMemoryInfo *mem_info, OrtAllocator **out)
 Create an allocator for an OrtSession following an OrtMemoryInfo.
 
void ReleaseAllocator (OrtAllocator *input)
 Release an OrtAllocator obtained from OrtApi::CreateAllocator.
 
OrtStatusRegisterAllocator (OrtEnv *env, OrtAllocator *allocator)
 Register a custom allocator.
 
OrtStatusUnregisterAllocator (OrtEnv *env, const OrtMemoryInfo *mem_info)
 Unregister a custom allocator.
 

OrtKernelInfo

Custom operator APIs.

OrtStatusKernelInfoGetAttribute_float (const OrtKernelInfo *info, const char *name, float *out)
 Get a float stored as an attribute in the graph node.
 
OrtStatusKernelInfoGetAttribute_int64 (const OrtKernelInfo *info, const char *name, int64_t *out)
 Fetch a 64-bit int stored as an attribute in the graph node.
 
OrtStatusKernelInfoGetAttribute_string (const OrtKernelInfo *info, const char *name, char *out, size_t *size)
 Fetch a string stored as an attribute in the graph node.
 
OrtStatusKernelInfoGetAttributeArray_float (const OrtKernelInfo *info, const char *name, float *out, size_t *size)
 Fetch an array of int64_t values stored as an attribute in the graph node.
 
OrtStatusKernelInfoGetAttributeArray_int64 (const OrtKernelInfo *info, const char *name, int64_t *out, size_t *size)
 Fetch an array of int64_t values stored as an attribute in the graph node.
 
OrtStatusKernelInfo_GetInputCount (const OrtKernelInfo *info, size_t *out)
 Get the number of inputs from OrtKernelInfo.
 
OrtStatusKernelInfo_GetOutputCount (const OrtKernelInfo *info, size_t *out)
 Get the number of outputs from OrtKernelInfo.
 
OrtStatusKernelInfo_GetInputName (const OrtKernelInfo *info, size_t index, char *out, size_t *size)
 Get the name of a OrtKernelInfo's input.
 
OrtStatusKernelInfo_GetOutputName (const OrtKernelInfo *info, size_t index, char *out, size_t *size)
 Get the name of a OrtKernelInfo's output.
 
OrtStatusKernelInfo_GetInputTypeInfo (const OrtKernelInfo *info, size_t index, OrtTypeInfo **type_info)
 Get the type information for a OrtKernelInfo's input.
 
OrtStatusKernelInfo_GetOutputTypeInfo (const OrtKernelInfo *info, size_t index, OrtTypeInfo **type_info)
 Get the type information for a OrtKernelInfo's output.
 
OrtStatusKernelInfoGetAttribute_tensor (const OrtKernelInfo *info, const char *name, OrtAllocator *allocator, OrtValue **out)
 Get a OrtValue tensor stored as an attribute in the graph node.
 
OrtStatusKernelInfo_GetNodeName (const OrtKernelInfo *info, char *out, size_t *size)
 Get the graph node name from OrtKernelInfo.
 
OrtStatusKernelInfo_GetLogger (const OrtKernelInfo *info, const OrtLogger **logger)
 Get the session logger from OrtKernelInfo.
 

OrtKernelContext

Custom operator APIs.

OrtStatusKernelContext_GetInputCount (const OrtKernelContext *context, size_t *out)
 Used for custom operators, get the input count of a kernel.
 
OrtStatusKernelContext_GetOutputCount (const OrtKernelContext *context, size_t *out)
 Used for custom operators, get the output count of a kernel.
 
OrtStatusKernelContext_GetInput (const OrtKernelContext *context, size_t index, const OrtValue **out)
 Used for custom operators, get an input of a kernel.
 
OrtStatusKernelContext_GetOutput (OrtKernelContext *context, size_t index, const int64_t *dim_values, size_t dim_count, OrtValue **out)
 Used for custom operators, get an output of a kernel.
 
OrtStatusKernelContext_GetGPUComputeStream (const OrtKernelContext *context, void **out)
 Used for custom operators, gets the GPU compute stream to use to launch the custom a GPU kernel.
 
OrtStatusKernelContext_GetLogger (const OrtKernelContext *context, const OrtLogger **logger)
 Get the runtime logger from OrtKernelContext.
 

OrtMapTypeInfo

OrtStatusGetMapKeyType (const OrtMapTypeInfo *map_type_info, enum ONNXTensorElementDataType *out)
 Get key type from an OrtMapTypeInfo.
 
OrtStatusGetMapValueType (const OrtMapTypeInfo *map_type_info, OrtTypeInfo **type_info)
 Get the value type from an OrtMapTypeInfo.
 
void ReleaseMapTypeInfo (OrtMapTypeInfo *input)
 

OrtSequenceTypeInfo

OrtStatusGetSequenceElementType (const OrtSequenceTypeInfo *sequence_type_info, OrtTypeInfo **type_info)
 Get element type from an OrtSequenceTypeInfo.
 
void ReleaseSequenceTypeInfo (OrtSequenceTypeInfo *input)
 

OrtModelMetadata

OrtStatusModelMetadataGetProducerName (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, char **value)
 Get producer name from an OrtModelMetadata.
 
OrtStatusModelMetadataGetGraphName (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, char **value)
 Get graph name from an OrtModelMetadata.
 
OrtStatusModelMetadataGetDomain (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, char **value)
 Get domain from an OrtModelMetadata.
 
OrtStatusModelMetadataGetDescription (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, char **value)
 Get description from an OrtModelMetadata.
 
OrtStatusModelMetadataLookupCustomMetadataMap (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, const char *key, char **value)
 Return data for a key in the custom metadata map in an OrtModelMetadata.
 
OrtStatusModelMetadataGetVersion (const OrtModelMetadata *model_metadata, int64_t *value)
 Get version number from an OrtModelMetadata.
 
void ReleaseModelMetadata (OrtModelMetadata *input)
 
OrtStatusModelMetadataGetCustomMetadataMapKeys (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, char ***keys, int64_t *num_keys)
 
OrtStatusModelMetadataGetGraphDescription (const OrtModelMetadata *model_metadata, OrtAllocator *allocator, char **value)
 

OrtThreadingOptions

OrtStatusCreateThreadingOptions (OrtThreadingOptions **out)
 Create an OrtThreadingOptions.
 
void ReleaseThreadingOptions (OrtThreadingOptions *input)
 
OrtStatusSetGlobalIntraOpNumThreads (OrtThreadingOptions *tp_options, int intra_op_num_threads)
 Set global intra-op thread count.
 
OrtStatusSetGlobalInterOpNumThreads (OrtThreadingOptions *tp_options, int inter_op_num_threads)
 Set global inter-op thread count.
 
OrtStatusSetGlobalSpinControl (OrtThreadingOptions *tp_options, int allow_spinning)
 Set global spin control options.
 
OrtStatusSetGlobalDenormalAsZero (OrtThreadingOptions *tp_options)
 Set threading flush-to-zero and denormal-as-zero.
 
OrtStatusSetGlobalCustomCreateThreadFn (OrtThreadingOptions *tp_options, OrtCustomCreateThreadFn ort_custom_create_thread_fn)
 Set custom thread creation function for global thread pools.
 
OrtStatusSetGlobalCustomThreadCreationOptions (OrtThreadingOptions *tp_options, void *ort_custom_thread_creation_options)
 Set custom thread creation options for global thread pools.
 
OrtStatusSetGlobalCustomJoinThreadFn (OrtThreadingOptions *tp_options, OrtCustomJoinThreadFn ort_custom_join_thread_fn)
 Set custom thread join function for global thread pools.
 

Misc

OrtStatusGetAvailableProviders (char ***out_ptr, int *provider_length)
 Get the names of all available providers.
 
OrtStatusReleaseAvailableProviders (char **ptr, int providers_length)
 Release data from OrtApi::GetAvailableProviders. This API will never fail so you can rely on it in a noexcept code.
 
OrtStatusSetCurrentGpuDeviceId (int device_id)
 Set current GPU device ID.
 
OrtStatusGetCurrentGpuDeviceId (int *device_id)
 Get current GPU device ID.
 

OrtArenaCfg

OrtStatusCreateArenaCfg (size_t max_mem, int arena_extend_strategy, int initial_chunk_size_bytes, int max_dead_bytes_per_chunk, OrtArenaCfg **out)
 
void ReleaseArenaCfg (OrtArenaCfg *input)
 
OrtStatusCreateArenaCfgV2 (const char *const *arena_config_keys, const size_t *arena_config_values, size_t num_keys, OrtArenaCfg **out)
 Create an OrtArenaCfg.
 

OrtPrepackedWeightsContainer

OrtStatusCreatePrepackedWeightsContainer (OrtPrepackedWeightsContainer **out)
 Create an OrtPrepackedWeightsContainer.
 
void ReleasePrepackedWeightsContainer (OrtPrepackedWeightsContainer *input)
 Release OrtPrepackedWeightsContainer instance.
 

GetTensorMemoryInfo

OrtStatusGetTensorMemoryInfo (const OrtValue *value, const OrtMemoryInfo **mem_info)
 Returns a pointer to the OrtMemoryInfo of a Tensor.
 

GetExecutionProviderApi

OrtStatusGetExecutionProviderApi (const char *provider_name, uint32_t version, const void **provider_api)
 Get a pointer to the requested version of the Execution Provider specific API extensions to the OrtApi.
 

SessionOptions

OrtStatusSessionOptionsSetCustomCreateThreadFn (OrtSessionOptions *options, OrtCustomCreateThreadFn ort_custom_create_thread_fn)
 Set custom thread creation function.
 
OrtStatusSessionOptionsSetCustomThreadCreationOptions (OrtSessionOptions *options, void *ort_custom_thread_creation_options)
 Set creation options for custom thread.
 
OrtStatusSessionOptionsSetCustomJoinThreadFn (OrtSessionOptions *options, OrtCustomJoinThreadFn ort_custom_join_thread_fn)
 Set custom thread join function.
 

OrtLogger

Custom operator APIs.

OrtStatusLogger_LogMessage (const OrtLogger *logger, OrtLoggingLevel log_severity_level, const char *message, const char *file_path, int line_number, const char *func_name)
 Logs a message at the given severity level using the provided OrtLogger.
 
OrtStatusLogger_GetLoggingSeverityLevel (const OrtLogger *logger, OrtLoggingLevel *out)
 Get the logging severity level of the OrtLogger.
 

OrtEpDynamicOptions

OrtStatusSetEpDynamicOptions (OrtSession *sess, const char *const *keys, const char *const *values, size_t kv_len)
 Set DynamicOptions for EPs (Execution Providers)
 

Detailed Description

The C API.

All C API functions are defined inside this structure as pointers to functions. Call OrtApiBase::GetApi to get a pointer to it

Member Function Documentation

◆ AddCustomOpDomain()

OrtStatus * OrtApi::AddCustomOpDomain ( OrtSessionOptions options,
OrtCustomOpDomain custom_op_domain 
)

Add custom op domain to a session options.

Note
The OrtCustomOpDomain* must not be deleted until all sessions using it are released
Parameters
[in]options
[in]custom_op_domain

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ AddExternalInitializers()

OrtStatus * OrtApi::AddExternalInitializers ( OrtSessionOptions options,
const char *const *  initializer_names,
const OrtValue *const *  initializers,
size_t  num_initializers 
)

Replace initialized Tensors with external data with the data provided in initializers.

The function will find the initialized TensorProtos with external data in the graph with the provided names and replace them with the provided tensors. The API verifies that the TensorProto being replaced has an external data reference and has the same name, dimensions and data type as its replacement. The replacement will occur before any of the optimizations take place. The data will be copied into the graph since TensorProto can't refer to the user provided buffers.

Once the model has been loaded, the OrtValue(s) added to SessionOptions instance will be removed from the internal SessionOptions copy to save memory, the user provided buffers can then be deallocated and the SessionOptions instance that refers to them can be destroyed.

Parameters
[in]options
[in]initializer_namesArray of null terminated UTF-8 encoded strings of the initializers names.
[in]initializersArray of OrtValue type
[in]num_initializersNumber of elements in the initializer_names and initializers

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.12.

◆ AddExternalInitializersFromFilesInMemory()

OrtStatus * OrtApi::AddExternalInitializersFromFilesInMemory ( OrtSessionOptions options,
const char *const *  external_initializer_file_names,
char *const *  external_initializer_file_buffer_array,
const size_t *  external_initializer_file_lengths,
size_t  num_external_initializer_files 
)

Replace initialized Tensors with external data with the provided files in memory.

The function will find the initialized TensorProtos with external data in the graph with the provided external file names and the file content in memory. The API gets the external file name, offset, data length from TensorProto, and locate the tensor data from the file in memory buffer. It creates a Tensor to replace the existing Tensor in graph. The replacement will occur before any of the optimizations take place. The data will be copied into the graph since TensorProto can't refer to the user provided buffers.

Parameters
[in]options
[in]external_initializer_file_namesArray of null terminated UTF-8 encoded strings of the file names which holds the external initializers.
[in]external_initializer_file_buffer_arrayArray of pointers to the buffer of the file content. The buffer can be freed after session creation.
[in]external_initializer_file_lengthsArray of size_t to indicate the length of file content
[in]num_external_initializer_filesNumber of external files

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ AddFreeDimensionOverride()

OrtStatus * OrtApi::AddFreeDimensionOverride ( OrtSessionOptions options,
const char *  dim_denotation,
int64_t  dim_value 
)

Override session symbolic dimensions.

Override symbolic dimensions (by specific denotation strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)

Parameters
[in]options
[in]dim_denotation
[in]dim_value

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ AddFreeDimensionOverrideByName()

OrtStatus * OrtApi::AddFreeDimensionOverrideByName ( OrtSessionOptions options,
const char *  dim_name,
int64_t  dim_value 
)

Override symbolic dimensions (by specific name strings) with actual values if known at session initialization time to enable optimizations that can take advantage of fixed values (such as memory planning, etc)

◆ AddInitializer()

OrtStatus * OrtApi::AddInitializer ( OrtSessionOptions options,
const char *  name,
const OrtValue val 
)

Add a pre-allocated initializer to a session.

If a model contains an initializer with a name that is same as the name passed to this call, ORT will use this initializer instance instead of deserializing one from the model file. This is useful when you want to share the same initializer across sessions.

Parameters
[in]options
[in]nameNull terminated string of the initializer name
[in]valOrtValue containing the initializer. Its lifetime and the underlying initializer buffer must be managed by the user (created using the OrtApi::CreateTensorWithDataAsOrtValue) and it must outlive the session object to which it is added.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ AddRunConfigEntry()

OrtStatus * OrtApi::AddRunConfigEntry ( OrtRunOptions options,
const char *  config_key,
const char *  config_value 
)

Set a single run configuration entry as a pair of strings.

If a configuration with same key exists, this will overwrite the configuration with the given config_value

The config_key and the format of config_value are defined in onnxruntime_run_options_config_keys.h

Parameters
[in]options
[in]config_keyA null terminated string representation of the config key
[in]config_valueA null terminated string representation of the config value

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ AddSessionConfigEntry()

OrtStatus * OrtApi::AddSessionConfigEntry ( OrtSessionOptions options,
const char *  config_key,
const char *  config_value 
)

Set a session configuration entry as a pair of strings.

If a configuration with same key exists, this will overwrite the configuration with the given config_value.

The config_key and the format of config_value are defined in onnxruntime_session_options_config_keys.h

Parameters
[in]options
[in]config_keyA null terminated string representation of the config key
[in]config_valueA null terminated string representation of the config value

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ AllocatorAlloc()

OrtStatus * OrtApi::AllocatorAlloc ( OrtAllocator ort_allocator,
size_t  size,
void **  out 
)

Calls OrtAllocator::Alloc function.

◆ AllocatorFree()

OrtStatus * OrtApi::AllocatorFree ( OrtAllocator ort_allocator,
void *  p 
)

Calls OrtAllocator::Free function.

◆ AllocatorGetInfo()

OrtStatus * OrtApi::AllocatorGetInfo ( const OrtAllocator ort_allocator,
const struct OrtMemoryInfo **  out 
)

Calls OrtAllocator::Info function.

◆ BindInput()

OrtStatus * OrtApi::BindInput ( OrtIoBinding binding_ptr,
const char *  name,
const OrtValue val_ptr 
)

Bind an OrtValue to an OrtIoBinding input.

When using OrtApi::RunWithBinding this value is used for the named input

Parameters
[in]binding_ptr
[in]nameName for the model input
[in]val_ptrOrtValue of Tensor type.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ BindOutput()

OrtStatus * OrtApi::BindOutput ( OrtIoBinding binding_ptr,
const char *  name,
const OrtValue val_ptr 
)

Bind an OrtValue to an OrtIoBinding output.

When using OrtApi::RunWithBinding this value is used for the named output

Parameters
[in]binding_ptr
[in]nameNull terminated string of the model output name
[in]val_ptrOrtValue of Tensor type.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ BindOutputToDevice()

OrtStatus * OrtApi::BindOutputToDevice ( OrtIoBinding binding_ptr,
const char *  name,
const OrtMemoryInfo mem_info_ptr 
)

Bind an OrtIoBinding output to a device.

Binds the OrtValue to a device which is specified by OrtMemoryInfo. You can either create an instance of OrtMemoryInfo with a device id or obtain one from the allocator that you have created/are using This is useful when one or more outputs have dynamic shapes and, it is hard to pre-allocate and bind a chunk of memory within OrtValue ahead of time.

See also
OrtApi::RunWithBinding
Parameters
[in]binding_ptr
[in]nameNull terminated string of the device name
[in]mem_info_ptr

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CastTypeInfoToMapTypeInfo()

OrtStatus * OrtApi::CastTypeInfoToMapTypeInfo ( const OrtTypeInfo type_info,
const OrtMapTypeInfo **  out 
)

Get detailed map information from an OrtTypeInfo.

This augments OrtTypeInfo to return an OrtMapTypeInfo when the type is a map. The OrtMapTypeInfo has additional information about the map's key type and value type.

This is used by WinML to support model reflection APIs.

Parameters
[out]type_info
[out]outA pointer to the OrtMapTypeInfo. Do not free this value. If type_info does not contain a map, this value will be set to nullptr.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CastTypeInfoToOptionalTypeInfo()

OrtStatus * OrtApi::CastTypeInfoToOptionalTypeInfo ( const OrtTypeInfo type_info,
const OrtOptionalTypeInfo **  out 
)

Get Optional Type information from an OrtTypeInfo.

This augments OrtTypeInfo to return an OrtOptionalTypeInfo when the type is optional. The OrtOptionalTypeInfo also has a nested OrtTypeInfo that describes the type of the optional value. OrtOptionalTypeInfo type can only appear within model metadata to describe inputs/outputs. The actual OrtValues that are supplied in place of optional type inputs should contain specific type that is described by OrtOptionalTypeInfo.

So the picture: OrtTypeInfo -> OrtOptionalTypeInfo -> OrtTypeInfo (describes the type that can be supplied in place of the optional type when creating the actual OrtValue).

Parameters
[in]type_info
[out]outA pointer to the OrtOptionalTypeInfo. Do not free this value, it is owned by OrtTypeInfo instance. When the type_info does not represent optional type, nullptr is returned in out.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ CastTypeInfoToSequenceTypeInfo()

OrtStatus * OrtApi::CastTypeInfoToSequenceTypeInfo ( const OrtTypeInfo type_info,
const OrtSequenceTypeInfo **  out 
)

Cast OrtTypeInfo to an OrtSequenceTypeInfo.

This api augments OrtTypeInfo to return an OrtSequenceTypeInfo when the type is a sequence. The OrtSequenceTypeInfo has additional information about the sequence's element type.

This is used by WinML to support model reflection APIs.

Parameters
[in]type_info
[out]outA pointer to the OrtSequenceTypeInfo. Do not free this value. If type_info doesn not contain a sequence, this value will be set to nullptr.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CastTypeInfoToTensorInfo()

OrtStatus * OrtApi::CastTypeInfoToTensorInfo ( const OrtTypeInfo type_info,
const OrtTensorTypeAndShapeInfo **  out 
)

Get OrtTensorTypeAndShapeInfo from an OrtTypeInfo.

Parameters
[in]type_info
[out]outDo not free this value, it will be valid until type_info is freed. If type_info does not represent tensor, this value will be set to nullptr.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CloneSessionOptions()

OrtStatus * OrtApi::CloneSessionOptions ( const OrtSessionOptions in_options,
OrtSessionOptions **  out_options 
)

Create a copy of an existing OrtSessionOptions.

Parameters
[in]in_optionsOrtSessionOptions to copy
[out]out_optionsReturned newly created OrtSessionOptions. Must be freed with OrtApi::ReleaseSessionOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CompareMemoryInfo()

OrtStatus * OrtApi::CompareMemoryInfo ( const OrtMemoryInfo info1,
const OrtMemoryInfo info2,
int *  out 
)

Compare OrtMemoryInfo objects for equality.

Compares all settings of each OrtMemoryInfo for equality

Parameters
[in]info1
[in]info2
[out]outSet to 0 if equal, -1 if not equal

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CopyKernelInfo()

OrtStatus * OrtApi::CopyKernelInfo ( const OrtKernelInfo info,
OrtKernelInfo **  info_copy 
)

◆ CreateAllocator()

OrtStatus * OrtApi::CreateAllocator ( const OrtSession session,
const OrtMemoryInfo mem_info,
OrtAllocator **  out 
)

Create an allocator for an OrtSession following an OrtMemoryInfo.

Parameters
[in]session
[in]mem_infovalid OrtMemoryInfo instance
[out]outNewly created OrtAllocator. Must be freed with OrtApi::ReleaseAllocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateAndRegisterAllocator()

OrtStatus * OrtApi::CreateAndRegisterAllocator ( OrtEnv env,
const OrtMemoryInfo mem_info,
const OrtArenaCfg arena_cfg 
)

Create an allocator and register it with the OrtEnv.

Enables sharing the allocator between multiple sessions that use the same env instance. Lifetime of the created allocator will be valid for the duration of the environment. Returns an error if an allocator with the same OrtMemoryInfo is already registered.

See https://onnxruntime.ai/docs/get-started/with-c.html for details.

Parameters
[in]envOrtEnv instance
[in]mem_info
[in]arena_cfgPass nullptr for defaults

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateAndRegisterAllocatorV2()

OrtStatus * OrtApi::CreateAndRegisterAllocatorV2 ( OrtEnv env,
const char *  provider_type,
const OrtMemoryInfo mem_info,
const OrtArenaCfg arena_cfg,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Create an allocator with specific type and register it with the OrtEnv This API enhance CreateAndRegisterAllocator that it can create an allocator with specific type, not just CPU allocator Enables sharing the allocator between multiple sessions that use the same env instance. Lifetime of the created allocator will be valid for the duration of the environment. Returns an error if an allocator with the same OrtMemoryInfo is already registered.

Parameters
[in]envOrtEnv instance
[in]provider_typeExecutionProvider type
[in]mem_infoOrtMemoryInfo instance
[in]arena_cfgArena configuration
[in]provider_options_keyskey of the provider options map
[in]provider_options_valuesvalue of the provider options map
[in]num_keysLength of the provider options map

◆ CreateArenaCfg()

OrtStatus * OrtApi::CreateArenaCfg ( size_t  max_mem,
int  arena_extend_strategy,
int  initial_chunk_size_bytes,
int  max_dead_bytes_per_chunk,
OrtArenaCfg **  out 
)
Deprecated:
Use OrtApi::CreateArenaCfgV2

This will create the configuration of an arena that can eventually be used to define an arena based allocator's behavior

Parameters
[in]max_memUse 0 to allow ORT to choose the default
[in]arena_extend_strategyUse -1 to allow ORT to choose the default, 0 = kNextPowerOfTwo, 1 = kSameAsRequested
[in]initial_chunk_size_bytesUse -1 to allow ORT to choose the default
[in]max_dead_bytes_per_chunkUse -1 to allow ORT to choose the default
[in]outA pointer to an OrtArenaCfg instance

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateArenaCfgV2()

OrtStatus * OrtApi::CreateArenaCfgV2 ( const char *const *  arena_config_keys,
const size_t *  arena_config_values,
size_t  num_keys,
OrtArenaCfg **  out 
)

Create an OrtArenaCfg.

Create the configuration of an arena that can eventually be used to define an arena based allocator's behavior.

Supported keys are (See https://onnxruntime.ai/docs/get-started/with-c.html for details on what the following parameters mean and how to choose these values.): "max_mem": Maximum memory that can be allocated by the arena based allocator. Use 0 for ORT to pick the best value. Default is 0. "arena_extend_strategy": 0 = kNextPowerOfTwo, 1 = kSameAsRequested. Use -1 to allow ORT to choose the default. "initial_chunk_size_bytes": (Possible) Size of the first allocation in the arena. Only relevant if arena strategy is kNextPowerOfTwo. Use -1 to allow ORT to choose the default. Ultimately, the first allocation size is determined by the allocation memory request. "max_dead_bytes_per_chunk": Threshold of unused memory in an allocated chunk of arena memory after crossing which the current chunk is chunked into 2. "initial_growth_chunk_size_bytes": (Possible) Size of the second allocation in the arena. Only relevant if arena strategy is kNextPowerOfTwo. Use -1 to allow ORT to choose the default. "max_power_of_two_extend_bytes": The maximum enxtend size if arena strategy is kNextPowerOfTwo. It is not an allocation limit, it is only a limit for extension when requested byte is less than the limit. When requested bytes is more than the limit, allocator will still return as requested. Use -1 to allow ORT to choose the default 1GB for max_power_of_two_extend_bytes. Ultimately, the allocation size is determined by the allocation memory request. Further allocation sizes are governed by the arena extend strategy.

Parameters
[in]arena_config_keysKeys to configure the arena
[in]arena_config_valuesValues to configure the arena
[in]num_keysNumber of keys in arena_config_keys and arena_config_values
[out]outNewly created OrtArenaCfg. Must be freed with OrtApi::ReleaseArenaCfg

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateCANNProviderOptions()

OrtStatus * OrtApi::CreateCANNProviderOptions ( OrtCANNProviderOptions **  out)

Create an OrtCANNProviderOptions.

Parameters
[out]outcreated OrtCANNProviderOptions. Must be released with OrtApi::ReleaseCANNProviderOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.13.

◆ CreateCpuMemoryInfo()

OrtStatus * OrtApi::CreateCpuMemoryInfo ( enum OrtAllocatorType  type,
enum OrtMemType  mem_type,
OrtMemoryInfo **  out 
)

Create an OrtMemoryInfo for CPU memory.

Special case version of OrtApi::CreateMemoryInfo for CPU based memory. Same as using OrtApi::CreateMemoryInfo with name = "Cpu" and id = 0.

Parameters
[in]type
[in]mem_type
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateCUDAProviderOptions()

OrtStatus * OrtApi::CreateCUDAProviderOptions ( OrtCUDAProviderOptionsV2 **  out)

Create an OrtCUDAProviderOptionsV2.

Parameters
[out]outNewly created OrtCUDAProviderOptionsV2. Must be released with OrtApi::ReleaseCudaProviderOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.11.

◆ CreateCustomOpDomain()

OrtStatus * OrtApi::CreateCustomOpDomain ( const char *  domain,
OrtCustomOpDomain **  out 
)

Create a custom op domain.

Parameters
[in]domain
[out]outNewly created domain. Must be freed with OrtApi::ReleaseCustomOpDomain

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateDnnlProviderOptions()

OrtStatus * OrtApi::CreateDnnlProviderOptions ( OrtDnnlProviderOptions **  out)

Create an OrtDnnlProviderOptions.

Parameters
[out]outNewly created OrtDnnlProviderOptions. Must be released with OrtApi::ReleaseDnnlProviderOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ CreateEnv()

OrtStatus * OrtApi::CreateEnv ( OrtLoggingLevel  log_severity_level,
const char *  logid,
OrtEnv **  out 
)

Create an OrtEnv.

Note
Invoking this function will return the same instance of the environment as that returned by a previous call to another env creation function; all arguments to this function will be ignored.
Parameters
[in]log_severity_levelThe log severity level.
[in]logidThe log identifier.
[out]outReturned newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateEnvWithCustomLogger()

OrtStatus * OrtApi::CreateEnvWithCustomLogger ( OrtLoggingFunction  logging_function,
void *  logger_param,
OrtLoggingLevel  log_severity_level,
const char *  logid,
OrtEnv **  out 
)

Create an OrtEnv.

Note
Invoking this function will return the same instance of the environment as that returned by a previous call to another env creation function; all arguments to this function will be ignored. If you want to provide your own logging function, consider setting it using the SetUserLoggingFunction API instead.
Parameters
[in]logging_functionA pointer to a logging function.
[in]logger_paramA pointer to arbitrary data passed as the OrtLoggingFunction param parameter to logging_function. This parameter is optional.
[in]log_severity_levelThe log severity level.
[in]logidThe log identifier.
[out]outReturned newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateEnvWithCustomLoggerAndGlobalThreadPools()

OrtStatus * OrtApi::CreateEnvWithCustomLoggerAndGlobalThreadPools ( OrtLoggingFunction  logging_function,
void *  logger_param,
OrtLoggingLevel  log_severity_level,
const char *  logid,
const struct OrtThreadingOptions tp_options,
OrtEnv **  out 
)

Create a custom environment with global threadpools and logger that will be shared across sessions. Use this in conjunction with OrtApi::DisablePerSessionThreads or else the session will use its own thread pools.

Parameters
[in]logging_functionA pointer to a logging function.
[in]logger_paramA pointer to arbitrary data passed as the OrtLoggingFunction param parameter to logging_function.
[in]log_severity_levelThe log severity level.
[in]logidThe log identifier.
[in]tp_options
[out]outNewly created OrtEnv. Must be freed with OrtApi::ReleaseEnv

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateEnvWithGlobalThreadPools()

OrtStatus * OrtApi::CreateEnvWithGlobalThreadPools ( OrtLoggingLevel  log_severity_level,
const char *  logid,
const OrtThreadingOptions tp_options,
OrtEnv **  out 
)

Create an OrtEnv.

Create an environment with global threadpools that will be shared across sessions. Use this in conjunction with OrtApi::DisablePerSessionThreads or else the session will use its own thread pools.

Parameters
[in]log_severity_levelThe log severity level.
[in]logidThe log identifier.
[in]tp_options
[out]outReturned newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateIoBinding()

OrtStatus * OrtApi::CreateIoBinding ( OrtSession session,
OrtIoBinding **  out 
)

Create an OrtIoBinding instance.

An IoBinding object allows one to bind pre-allocated OrtValues to input names. Thus if you want to use a raw on device buffer as input or output you can avoid extra copy during runtime.

Parameters
[in]session
[out]outNewly created OrtIoBinding. Must be freed with OrtApi::ReleaseIoBinding

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateLoraAdapter()

OrtStatus * OrtApi::CreateLoraAdapter ( const char *  adapter_file_path,
OrtAllocator allocator,
OrtLoraAdapter **  out 
)

Create an OrtLoraAdapter.

The function attempts to locate file specified by adapter_file_path, read it and create an OrtLoraAdapter instance. The adapter_file_path should be a valid path to a file that contains a valid Lora Adapter format. The function attempts to validate the format at load time. The file will always be memory mapped, unless the platform does not support memory mapping, in which case the file will be read into memory.

Parameters
[in]adapter_file_pathadapter file path.
[in]allocatoroptional pointer to a device allocator. If specified data is copied to the device at some point before Run() is invoked. If nullptr, data stays on CPU. The data would still be copied to device if required by the model at inference time.
[out]outA pointer to a newly created OrtLoraAdapter instance. Must be released with OrtApi::ReleaseLoraAdapter.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateLoraAdapterFromArray()

OrtStatus * OrtApi::CreateLoraAdapterFromArray ( const void *  bytes,
size_t  num_bytes,
OrtAllocator allocator,
OrtLoraAdapter **  out 
)

Create an OrtLoraAdapter.

The function copies the bytes from the array and creates an OrtLoraAdapter instance.

Parameters
[in]bytespointer to a valid Lora Adapter format buffer.
[in]num_byteslength of bytes buffer.
[in]allocatoroptional pointer to a device allocator. If specified data is copied to the device at some point before Run() is invoked. If nullptr, data stays on CPU. The data would still be copied to device if required by the model at inference time.
[out]outA pointer to a newly created OrtLoraAdapter instance. Must be released with OrtApi::ReleaseLoraAdapter.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateMemoryInfo()

OrtStatus * OrtApi::CreateMemoryInfo ( const char *  name,
enum OrtAllocatorType  type,
int  id,
enum OrtMemType  mem_type,
OrtMemoryInfo **  out 
)

Create an OrtMemoryInfo.

Parameters
[in]name
[in]type
[in]id
[in]mem_type
[out]outNewly created OrtMemoryInfo. Must be freed with OrtAPi::ReleaseMemoryInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateOp()

OrtStatus * OrtApi::CreateOp ( const OrtKernelInfo info,
const char *  op_name,
const char *  domain,
int  version,
const char **  type_constraint_names,
const ONNXTensorElementDataType type_constraint_values,
int  type_constraint_count,
const OrtOpAttr *const *  attr_values,
int  attr_count,
int  input_count,
int  output_count,
OrtOp **  ort_op 
)

: Create onnxruntime native operator

Parameters
[in]infoKernel info
[in]op_nameOperator name
[in]domainOperator domain
[in]versionOperator opset version
[in]type_constraint_namesName of the type contraints, such as "T" or "T1"
[in]type_constraint_valuesType of each contraints
[in]type_constraint_countNumber of contraints
[in]attr_valuesAttributes used to initialize the operator
[in]attr_countNumber of the attributes
[in]input_countNumber of inputs
[in]output_countNumber of outputs
[out]ort_opOperator that has been created
Since
Version 1.12.

◆ CreateOpaqueValue()

OrtStatus * OrtApi::CreateOpaqueValue ( const char *  domain_name,
const char *  type_name,
const void *  data_container,
size_t  data_container_size,
OrtValue **  out 
)

Create an opaque (custom user defined type) OrtValue.

Constructs an OrtValue that contains a value of non-standard type created for experiments or while awaiting standardization. OrtValue in this case would contain an internal representation of the Opaque type. Opaque types are distinguished from each other by two strings 1) domain and 2) type name. The combination of the two must be unique, so the type representation is properly identified internally. The combination must be properly registered from within ORT at both compile/run time or by another API.

To construct the OrtValue pass domain and type names, also a pointer to a data container the type of which must be known to both ORT and the client program. That data container may or may not match the internal representation of the Opaque type. The sizeof(data_container) is passed for verification purposes.

Parameters
[in]domain_nameNull terminated string of the domain name
[in]type_nameNull terminated string of the type name
[in]data_containerUser pointer Data to populate OrtValue
[in]data_container_sizeSize in bytes of what data_container points to
[out]outNewly created OrtValue. Must be freed with OrtApi::ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateOpAttr()

OrtStatus * OrtApi::CreateOpAttr ( const char *  name,
const void *  data,
int  len,
OrtOpAttrType  type,
OrtOpAttr **  op_attr 
)

: Create attribute of onnxruntime operator

Parameters
[in]nameName of the attribute
[in]dataData content of the attribute
[in]lenNumber of bytes stored in data
[in]typeData type
[out]op_attrAttribute that has been created, which must be released by OrtApi::ReleaseOpAttr
Since
Version 1.12.

◆ CreatePrepackedWeightsContainer()

OrtStatus * OrtApi::CreatePrepackedWeightsContainer ( OrtPrepackedWeightsContainer **  out)

Create an OrtPrepackedWeightsContainer.

This container will hold pre-packed buffers of shared initializers for sharing between sessions (i.e.) if there are shared initializers that can be shared between sessions, the pre-packed buffers of these (if any) may possibly be shared to provide memory footprint savings. Pass this container to sessions that you would like to share pre-packed buffers of shared initializers at session creation time.

Parameters
[out]outNewly created OrtPrepackedWeightsContainer. Must be freed with OrtApi::ReleasePrepackedWeightsContainer

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateROCMProviderOptions()

OrtStatus * OrtApi::CreateROCMProviderOptions ( OrtROCMProviderOptions **  out)

Create an OrtROCMProviderOptions.

Parameters
[out]outNewly created OrtROCMProviderOptions. Must be released with OrtApi::ReleaseROCMProviderOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.16.

◆ CreateRunOptions()

OrtStatus * OrtApi::CreateRunOptions ( OrtRunOptions **  out)

Create an OrtRunOptions.

Parameters
[out]outReturned newly created OrtRunOptions. Must be freed with OrtApi::ReleaseRunOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSession()

OrtStatus * OrtApi::CreateSession ( const OrtEnv env,
const char *  model_path,
const OrtSessionOptions options,
OrtSession **  out 
)

Create an OrtSession from a model file.

Parameters
[in]env
[in]model_path
[in]options
[out]outReturned newly created OrtSession. Must be freed with OrtApi::ReleaseSession

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSessionFromArray()

OrtStatus * OrtApi::CreateSessionFromArray ( const OrtEnv env,
const void *  model_data,
size_t  model_data_length,
const OrtSessionOptions options,
OrtSession **  out 
)

Create an OrtSession from memory.

Parameters
[in]env
[in]model_data
[in]model_data_length
[in]options
[out]outReturned newly created OrtSession. Must be freed with OrtApi::ReleaseSession

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSessionFromArrayWithPrepackedWeightsContainer()

OrtStatus * OrtApi::CreateSessionFromArrayWithPrepackedWeightsContainer ( const OrtEnv env,
const void *  model_data,
size_t  model_data_length,
const OrtSessionOptions options,
OrtPrepackedWeightsContainer prepacked_weights_container,
OrtSession **  out 
)

Create session from memory with prepacked weights container.

Same functionality offered by OrtApi::CreateSessionFromArray except that a container that contains pre-packed weights' buffers is written into/read from by the created session. This is useful when used in conjunction with OrtApi::AddInitializer which injects shared initializer info into sessions. Wherever possible, the pre-packed versions of these shared initializers are cached in this container so that multiple sessions can just re-use these instead of duplicating these in memory.

Parameters
[in]env
[in]model_dataArray of bytes holding the model
[in]model_data_lengthNumber of bytes in model_data_model
[in]options
[in]prepacked_weights_container
[out]outNewly created OrtSession. Must be freed with OrtApi::ReleaseSession

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSessionOptions()

OrtStatus * OrtApi::CreateSessionOptions ( OrtSessionOptions **  options)

Create an OrtSessionOptions object.

To use additional providers, you must build ORT with the extra providers enabled. Then call one of these functions to enable them in the session:
OrtSessionOptionsAppendExecutionProvider_CPU
OrtSessionOptionsAppendExecutionProvider_CUDA
OrtSessionOptionsAppendExecutionProvider_(remaining providers...)
The order they are called indicates the preference order as well. In other words call this method on your most preferred execution provider first followed by the less preferred ones. If none are called Ort will use its internal CPU execution provider.

Parameters
[out]optionsThe newly created OrtSessionOptions. Must be freed with OrtApi::ReleaseSessionOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSessionWithPrepackedWeightsContainer()

OrtStatus * OrtApi::CreateSessionWithPrepackedWeightsContainer ( const OrtEnv env,
const char *  model_path,
const OrtSessionOptions options,
OrtPrepackedWeightsContainer prepacked_weights_container,
OrtSession **  out 
)

Create session with prepacked weights container.

Same functionality offered by OrtApi::CreateSession except that a container that contains pre-packed weights' buffers is written into/read from by the created session. This is useful when used in conjunction with OrtApi::AddInitializer which injects shared initializer info into sessions. Wherever possible, the pre-packed versions of these shared initializers are cached in this container so that multiple sessions can just re-use these instead of duplicating these in memory.

Parameters
[in]envOrtEnv instance instance
[in]model_pathNull terminated string of the path (wchar on Windows, char otherwise)
[in]options
[in]prepacked_weights_container
[out]outNewly created OrtSession. Must be freed with OrtApi::ReleaseSession

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSparseTensorAsOrtValue()

OrtStatus * OrtApi::CreateSparseTensorAsOrtValue ( OrtAllocator allocator,
const int64_t *  dense_shape,
size_t  dense_shape_len,
ONNXTensorElementDataType  type,
OrtValue **  out 
)

Create an OrtValue with a sparse tensor that is empty.

Use FillSparseTensor<Format>() functions to populate sparse tensor with non-zero values and format specific indices data. Use ReleaseValue to destroy the sparse tensor, this will also release the buffer inside the output value if any was allocated.

Parameters
[in,out]allocatorallocator to use when performing an allocation. Allocation will be performed by FillSparseTensor<Format>() APIs. The lifespan of the allocator instance must eclipse the lifespan this sparse tensor instance as the same allocator will be used to free memory.
[in]dense_shapeshape of the original dense tensor
[in]dense_shape_lennumber of shape dimensions being passed
[in]typemust be one of TENSOR_ELEMENT_DATA_TYPE_xxxx
[out]outShould be freed by calling ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateSparseTensorWithValuesAsOrtValue()

OrtStatus * OrtApi::CreateSparseTensorWithValuesAsOrtValue ( const OrtMemoryInfo info,
void *  p_data,
const int64_t *  dense_shape,
size_t  dense_shape_len,
const int64_t *  values_shape,
size_t  values_shape_len,
ONNXTensorElementDataType  type,
OrtValue **  out 
)

Create an OrtValue with a sparse tensor. This is the first step. Next, use Use<Format>Indices() functions to supply sparse tensor with format specific indices data and set its sparse format to a specific enum value. This will not perform memory allocations. It will use supplied user buffer which should outlive the created sparse tensor. Use OrtApi::ReleaseValue to destroy the sparse tensor. It would not release the supplied values buffer. This function can not be used to map strings from the user allocated memory. Strings must always be copied and have UTF-8 encoding. Therefore, use OrtApi::CreateSparseTensorAsOrtValue above and then fill it with data using appropriate Make*() function.

Parameters
[in]infomemory info where sparse values reside.
[in,out]p_datapointer to a user allocated buffer with values. To create a full sparse tensor with no non-zero values, pass nullptr
[in]dense_shapeshape of the original dense tensor
[in]dense_shape_lennumber of shape dimensions being passed
[in]values_shapeshape of the values data. To create a fully sparse tensor with no non-zero values, pass {0} shape.
[in]values_shape_lennumber of values shape dimensions
[in]typemust be one of TENSOR_ELEMENT_DATA_TYPE_xxxx
[out]outShould be freed by calling ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateTensorAsOrtValue()

OrtStatus * OrtApi::CreateTensorAsOrtValue ( OrtAllocator allocator,
const int64_t *  shape,
size_t  shape_len,
ONNXTensorElementDataType  type,
OrtValue **  out 
)

Create a tensor.

Create a tensor using a supplied OrtAllocator

Parameters
[in]allocator
[in]shapePointer to the tensor shape dimensions.
[in]shape_lenThe number of tensor shape dimensions.
[in]type
[out]outReturns newly created OrtValue. Must be freed with OrtApi::ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateTensorRTProviderOptions()

OrtStatus * OrtApi::CreateTensorRTProviderOptions ( OrtTensorRTProviderOptionsV2 **  out)

Create an OrtTensorRTProviderOptionsV2.

Parameters
[out]outNewly created OrtTensorRTProviderOptionsV2. Must be released with OrtApi::ReleaseTensorRTProviderOptions

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateTensorTypeAndShapeInfo()

OrtStatus * OrtApi::CreateTensorTypeAndShapeInfo ( OrtTensorTypeAndShapeInfo **  out)

Create an OrtTensorTypeAndShapeInfo object.

Parameters
[out]outReturns newly created OrtTensorTypeAndShapeInfo. Must be freed with OrtApi::ReleaseTensorTypeAndShapeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateTensorWithDataAsOrtValue()

OrtStatus * OrtApi::CreateTensorWithDataAsOrtValue ( const OrtMemoryInfo info,
void *  p_data,
size_t  p_data_len,
const int64_t *  shape,
size_t  shape_len,
ONNXTensorElementDataType  type,
OrtValue **  out 
)

Create a tensor backed by a user supplied buffer.

Create a tensor with user's buffer. You can fill the buffer either before calling this function or after. p_data is owned by caller. ReleaseValue won't release p_data.

Parameters
[in]infoMemory description of where the p_data buffer resides (CPU vs GPU etc).
[in]p_dataPointer to the data buffer.
[in]p_data_lenThe number of bytes in the data buffer.
[in]shapePointer to the tensor shape dimensions.
[in]shape_lenThe number of tensor shape dimensions.
[in]typeThe data type.
[out]outReturns newly created OrtValue. Must be freed with OrtApi::ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateThreadingOptions()

OrtStatus * OrtApi::CreateThreadingOptions ( OrtThreadingOptions **  out)

Create an OrtThreadingOptions.

Parameters
[out]outNewly created OrtThreadingOptions. Must be freed with OrtApi::ReleaseThreadingOptions
Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CreateValue()

OrtStatus * OrtApi::CreateValue ( const OrtValue *const *  in,
size_t  num_values,
enum ONNXType  value_type,
OrtValue **  out 
)

Create a map or sequence OrtValue.

To construct a map (ONNX_TYPE_MAP), use num_values = 2 and in should be an array of 2 OrtValues representing keys and values.

To construct a sequence (ONNX_TYPE_SEQUENCE), use num_values = N where N is the number of the elements in the sequence. 'in' should be an array of N OrtValues.

Parameters
[in]inSee above for details
[in]num_values
[in]value_typeMust be either ONNX_TYPE_MAP or ONNX_TYPE_SEQUENCE
[out]outNewly created OrtValue. Must be freed with OrtApi::ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ CustomOpDomain_Add()

OrtStatus * OrtApi::CustomOpDomain_Add ( OrtCustomOpDomain custom_op_domain,
const OrtCustomOp op 
)

Add a custom op to a custom op domain.

Note
The OrtCustomOp* pointer must remain valid until the OrtCustomOpDomain using it is released
Parameters
[in]custom_op_domain
[in]op

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ DisableCpuMemArena()

OrtStatus * OrtApi::DisableCpuMemArena ( OrtSessionOptions options)

Disable the memory arena on CPU.

Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ DisableMemPattern()

OrtStatus * OrtApi::DisableMemPattern ( OrtSessionOptions options)

Disable the memory pattern optimization.

See also
OrtApi::EnableMemPattern
Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ DisablePerSessionThreads()

OrtStatus * OrtApi::DisablePerSessionThreads ( OrtSessionOptions options)

Use global thread pool on a session.

Disable using per session thread pool and use the shared global threadpool. This should be used in conjunction with OrtApi::CreateEnvWithGlobalThreadPools.

Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ DisableProfiling()

OrtStatus * OrtApi::DisableProfiling ( OrtSessionOptions options)

Disable profiling for a session.

Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ DisableTelemetryEvents()

OrtStatus * OrtApi::DisableTelemetryEvents ( const OrtEnv env)

Disable Telemetry.

See also
OrtApi::EnableTelemetryEvents
Parameters
[in]env

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ EnableCpuMemArena()

OrtStatus * OrtApi::EnableCpuMemArena ( OrtSessionOptions options)

Enable the memory arena on CPU.

Arena may pre-allocate memory for future usage.

Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ EnableMemPattern()

OrtStatus * OrtApi::EnableMemPattern ( OrtSessionOptions options)

Enable the memory pattern optimization.

The idea is if the input shapes are the same, we could trace the internal memory allocation and generate a memory pattern for future request. So next time we could just do one allocation with a big chunk for all the internal memory allocation.

Note
Memory pattern optimization is only available when Sequential Execution mode is enabled (see OrtApi::SetSessionExecutionMode)
See also
OrtApi::DisableMemPattern
Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ EnableOrtCustomOps()

OrtStatus * OrtApi::EnableOrtCustomOps ( OrtSessionOptions options)

Enable custom operators.

See onnxruntime-extensions: https://github.com/microsoft/onnxruntime-extensions.git

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ EnableProfiling()

OrtStatus * OrtApi::EnableProfiling ( OrtSessionOptions options,
const char *  profile_file_prefix 
)

Enable profiling for a session.

Parameters
[in]options
[in]profile_file_prefix

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ EnableTelemetryEvents()

OrtStatus * OrtApi::EnableTelemetryEvents ( const OrtEnv env)

Enable Telemetry.

Note
Telemetry events are on by default since they are lightweight
Parameters
[in]env

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ FillSparseTensorBlockSparse()

OrtStatus * OrtApi::FillSparseTensorBlockSparse ( OrtValue ort_value,
const OrtMemoryInfo data_mem_info,
const int64_t *  values_shape,
size_t  values_shape_len,
const void *  values,
const int64_t *  indices_shape_data,
size_t  indices_shape_len,
const int32_t *  indices_data 
)

This fills populates an empty tensor that was created using OrtApi::CreateSparseTensorAsOrtValue. This will allocate required memory and copy the supplied NNZ values and BlockSparse indices into that memory allocation. Memory allocation is performed using the allocator that was specified with OrtApi::CreateSparseTensorAsOrtValue.

Parameters
[in,out]ort_valueOrtValue to populate with data
[in]data_mem_infoserves to identify the location of the data to be copied. If the allocator specified at the creation time has memory info that is not the same as mem_info argument to this function a X-device copy will be performed. String data is assumed to be on CPU and will only be copied into a CPU allocated buffer.
[in]values_shape
[in]values_shape_len
[in]valuesstructure with values information
[in]indices_shape_datapointer to a location of indices shape
[in]indices_shape_lenlength of the block sparse indices shape
[in]indices_datapointer to a location of indices data. Shape will determine the length of the indices data.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ FillSparseTensorCoo()

OrtStatus * OrtApi::FillSparseTensorCoo ( OrtValue ort_value,
const OrtMemoryInfo data_mem_info,
const int64_t *  values_shape,
size_t  values_shape_len,
const void *  values,
const int64_t *  indices_data,
size_t  indices_num 
)

This fills populates an empty tensor that was created using OrtApi::CreateSparseTensorAsOrtValue. This will allocate required memory and copy the supplied NNZ values and COO indices into that memory allocation. Memory allocation is performed using the allocator that was specified with OrtApi::CreateSparseTensorAsOrtValue.

Parameters
[in,out]ort_valueOrtValue to populate with data
[in]data_mem_infoserves to identify the location of the data to be copied. If the allocator specified at the creation time has memory info that is not the same as mem_info argument to this function a X-device copy will be performed. String data is assumed to be on CPU and will only be copied into a CPU allocated buffer.
[in]values_shapepointer to values shape array
[in]values_shape_lenlength of the values_shape
[in]valuespointer to an array of values. For strings, pass const char**.
[in]indices_datapointer to a location of COO indices
[in]indices_numnumber of COO indices

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ FillSparseTensorCsr()

OrtStatus * OrtApi::FillSparseTensorCsr ( OrtValue ort_value,
const OrtMemoryInfo data_mem_info,
const int64_t *  values_shape,
size_t  values_shape_len,
const void *  values,
const int64_t *  inner_indices_data,
size_t  inner_indices_num,
const int64_t *  outer_indices_data,
size_t  outer_indices_num 
)

This fills populates an empty tensor that was created using OrtApi::CreateSparseTensorAsOrtValue. This will allocate required memory and copy the supplied NNZ values and CSR indices into that memory allocation. Memory allocation is performed using the allocator that was specified with OrtApi::CreateSparseTensorAsOrtValue.

Parameters
[in,out]ort_valueOrtValue to populate with data
[in]data_mem_infoserves to identify the location of the data to be copied. If the allocator specified at the creation time has memory info that is not the same as mem_info argument to this function a X-device copy will be performed. String data is assumed to be on CPU and will only be copied into a CPU allocated buffer.
[in]values_shapepointer to values shape array
[in]values_shape_lenlength of the values_shape
[in]values- pointer to an array of values. For strings, pass const char**.
[in]inner_indices_datapointer to a location of CSR inner indices
[in]inner_indices_numnumber of CSR inner indices
[in]outer_indices_datapointer to a location of CSR outer indices
[in]outer_indices_numnumber of CSR outer indices

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ FillStringTensor()

OrtStatus * OrtApi::FillStringTensor ( OrtValue value,
const char *const *  s,
size_t  s_len 
)

Set all strings at once in a string tensor.

Parameters
[in,out]valueA tensor of type ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING
[in]sAn array of strings. Each string in this array must be null terminated.
[in]s_lenCount of strings in s (Must match the size of value's tensor shape)

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ FillStringTensorElement()

OrtStatus * OrtApi::FillStringTensorElement ( OrtValue value,
const char *  s,
size_t  index 
)

Set a single string in a string tensor.

Parameters
[in]valueA string tensor
[in]sA null terminated UTF-8 encoded string
[in]indexIndex of the string in the tensor to set

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetAllocatorWithDefaultOptions()

OrtStatus * OrtApi::GetAllocatorWithDefaultOptions ( OrtAllocator **  out)

Get the default allocator.

The default allocator is a CPU based, non-arena. Always returns the same pointer to the same default allocator.

Parameters
[out]outReturned value should NOT be freed

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetAvailableProviders()

OrtStatus * OrtApi::GetAvailableProviders ( char ***  out_ptr,
int *  provider_length 
)

Get the names of all available providers.

Note
The providers in the list are not guaranteed to be usable. They may fail to load due to missing system dependencies. For example, if the CUDA/cuDNN libraries are not installed, the CUDA provider will report an error when it is added to the session options.
Parameters
[out]out_ptrSet to a pointer to an array of null terminated strings of the available providers. The entries and the array itself must be freed using OrtApi::ReleaseAvailableProviders
[out]provider_lengthSet to the number of entries in the out_ptr array

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetBoundOutputNames()

OrtStatus * OrtApi::GetBoundOutputNames ( const OrtIoBinding binding_ptr,
OrtAllocator allocator,
char **  buffer,
size_t **  lengths,
size_t *  count 
)

Get the names of an OrtIoBinding's outputs.

Returns the names of the outputs in the order they were bound. This is useful after running the model with bound outputs because the returned names are in order in which output OrtValue are returned. This is useful if the order of outputs and their names is not known.

Parameters
[in]binding_ptr
[in]allocatorAllocator used to allocate continuous buffers for output strings and lengths.
[out]bufferReturns an array of non-null terminated UTF-8 strings. The number of strings stored is returned in the count parameter. This buffer is allocated using allocator and must be freed using it.
[out]lengthsReturns an array of count lengths of the strings returned in buffer This buffer is allocated using allocator and must be freed using it.
[out]countNumber of strings returned. If binding_ptr has no bound outputs, zero is returned, no memory allocation is performed and buffer and lengths are set to nullptr.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetBoundOutputValues()

OrtStatus * OrtApi::GetBoundOutputValues ( const OrtIoBinding binding_ptr,
OrtAllocator allocator,
OrtValue ***  output,
size_t *  output_count 
)

Get the output OrtValue objects from an OrtIoBinding.

Returns an array of pointers to individually allocated OrtValues that contain results of a model execution with OrtApi::RunWithBinding The array contains the same number of OrtValues and they are in the same order as they were bound with OrtApi::BindOutput or OrtApi::BindOutputToDevice.

The returned OrtValues must be released using OrtApi::ReleaseValue after they are no longer needed. The array is allocated using the specified instance of the allocator and must be freed using the same allocator after all the OrtValues contained therein are individually released.

Parameters
[in]binding_ptr
[in]allocatorAllocator used to allocate output array
[out]outputSet to the allocated array of allocated OrtValue outputs. Set to nullptr if there are 0 outputs.
[out]output_countSet to number of OrtValues returned

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetCANNProviderOptionsAsString()

OrtStatus * OrtApi::GetCANNProviderOptionsAsString ( const OrtCANNProviderOptions cann_options,
OrtAllocator allocator,
char **  ptr 
)

Get serialized CANN provider options string.

Parameters
[in]cann_optionsOrtCANNProviderOptions instance
[in]allocatora ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions(), the specified allocator will be used to allocate continuous buffers for output strings and lengths.
[out]ptris a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.13.

◆ GetCUDAProviderOptionsAsString()

OrtStatus * OrtApi::GetCUDAProviderOptionsAsString ( const OrtCUDAProviderOptionsV2 cuda_options,
OrtAllocator allocator,
char **  ptr 
)

Get serialized CUDA provider options string.

For example, "device_id=0;arena_extend_strategy=0;......"

Parameters
cuda_options- OrtCUDAProviderOptionsV2 instance
allocator- a ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions() the specified allocator will be used to allocate continuous buffers for output strings and lengths.
ptr- is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.11.

◆ GetCUDAProviderOptionsByName()

OrtStatus * OrtApi::GetCUDAProviderOptionsByName ( const OrtCUDAProviderOptionsV2 cuda_options,
const char *  key,
void **  ptr 
)

Get CUDA EP provider option where its data type is pointer. If the data type of the provider option can be represented by string please use GetCUDAProviderOptionsAsString.

Parameters
cuda_options- OrtCUDAProviderOptionsV2 instance
key- Name of the provider option
ptr- A pointer to the instance that is kept by the provider option
Since
Version 1.16.

◆ GetCurrentGpuDeviceId()

OrtStatus * OrtApi::GetCurrentGpuDeviceId ( int *  device_id)

Get current GPU device ID.

Get the current device id of the GPU execution provider (CUDA/tensorrt/rocm).

See also
OrtApi::SetCurrentGpuDeviceId
Parameters
[out]device_id

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetDenotationFromTypeInfo()

OrtStatus * OrtApi::GetDenotationFromTypeInfo ( const OrtTypeInfo type_info,
const char **const  denotation,
size_t *  len 
)

Get denotation from type information.

Augments OrtTypeInfo to return denotations on the type.

This is used by WinML to determine if an input/output is intended to be an Image or a Tensor.

Parameters
[in]type_info
[out]denotationPointer to the null terminated denotation string is written to this pointer. This pointer is valid until the object is destroyed or the name is changed, do not free.
[out]lenLength in bytes of the string returned in denotation

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetDimensions()

OrtStatus * OrtApi::GetDimensions ( const OrtTensorTypeAndShapeInfo info,
int64_t *  dim_values,
size_t  dim_values_length 
)

Get dimensions in OrtTensorTypeAndShapeInfo.

Parameters
[in]info
[out]dim_valuesArray with dim_values_length elements. On return, filled with the dimensions stored in the OrtTensorTypeAndShapeInfo
[in]dim_values_lengthNumber of elements in dim_values. Use OrtApi::GetDimensionsCount to get this value

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetDimensionsCount()

OrtStatus * OrtApi::GetDimensionsCount ( const OrtTensorTypeAndShapeInfo info,
size_t *  out 
)

Get dimension count in OrtTensorTypeAndShapeInfo.

See also
OrtApi::GetDimensions
Parameters
[in]info
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetDnnlProviderOptionsAsString()

OrtStatus * OrtApi::GetDnnlProviderOptionsAsString ( const OrtDnnlProviderOptions dnnl_options,
OrtAllocator allocator,
char **  ptr 
)

Get serialized oneDNN provider options string.

For example, "use_arena=1;......"

Parameters
dnnl_options- OrtDnnlProviderOptions instance
allocator- a ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions() the specified allocator will be used to allocate continuous buffers for output strings and lengths.
ptr- is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ GetExecutionProviderApi()

OrtStatus * OrtApi::GetExecutionProviderApi ( const char *  provider_name,
uint32_t  version,
const void **  provider_api 
)

Get a pointer to the requested version of the Execution Provider specific API extensions to the OrtApi.

Parameters
[in]provider_nameThe name of the execution provider name. Currently only the following values are supported: "DML".
[in]versionMust be ORT_API_VERSION.
[out]provider_apiA void pointer containing a reference to the execution provider versioned api structure. For example, the provider_api pointer can be cast to the OrtDmlApi* when the provider_name is "DML".

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetMapKeyType()

OrtStatus * OrtApi::GetMapKeyType ( const OrtMapTypeInfo map_type_info,
enum ONNXTensorElementDataType out 
)

Get key type from an OrtMapTypeInfo.

Key types are restricted to being scalar types.

This is used by WinML to support model reflection APIs.

Parameters
[in]map_type_info
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetMapValueType()

OrtStatus * OrtApi::GetMapValueType ( const OrtMapTypeInfo map_type_info,
OrtTypeInfo **  type_info 
)

Get the value type from an OrtMapTypeInfo.

Parameters
[in]map_type_info
[out]type_info

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetOnnxTypeFromTypeInfo()

OrtStatus * OrtApi::GetOnnxTypeFromTypeInfo ( const OrtTypeInfo type_info,
enum ONNXType out 
)

Get ONNXType from OrtTypeInfo.

Parameters
[in]type_info
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetOpaqueValue()

OrtStatus * OrtApi::GetOpaqueValue ( const char *  domain_name,
const char *  type_name,
const OrtValue in,
void *  data_container,
size_t  data_container_size 
)

Get internal data from an opaque (custom user defined type) OrtValue.

Copies internal data from an opaque value into a user provided buffer

See also
OrtApi::CreateOpaqueValue
Parameters
[in]domain_nameNull terminated string of the domain name
[in]type_nameNull terminated string of the type name
[in]inThe opaque OrtValue
[out]data_containerBuffer to copy data into
[out]data_container_sizeSize in bytes of the buffer pointed to by data_container. Must match the size of the internal buffer.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetOptionalContainedTypeInfo()

OrtStatus * OrtApi::GetOptionalContainedTypeInfo ( const OrtOptionalTypeInfo optional_type_info,
OrtTypeInfo **  out 
)

Get OrtTypeInfo for the allowed contained type from an OrtOptionalTypeInfo.

This augments OrtOptionalTypeInfo to return an OrtTypeInfo for the contained type. The OrtOptionalTypeInfo has a nested OrtTypeInfo that describes the type of the optional value. OrtOptionalTypeInfo type can only appear within model metadata to describe inputs/outputs. The actual OrtValues that are supplied in place of optional type inputs should contain specific type that is described by the returned OrtTypeInfo.

Parameters
[in]optional_type_info
[out]outA pointer to the OrtTypeInfo for what the optional value could be. it is owned by OrtOptionalTypeInfo instance.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ GetResizedStringTensorElementBuffer()

OrtStatus * OrtApi::GetResizedStringTensorElementBuffer ( OrtValue value,
size_t  index,
size_t  length_in_bytes,
char **  buffer 
)

Set a single string in a string tensor Do not zero terminate the string data.

Parameters
[in]valueA string tensor
[in]index- flat index of the element
[in]length_in_byteslength of the buffer in utf-8 bytes (without the null terminator)
[in,out]buffer- address of return value

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetROCMProviderOptionsAsString()

OrtStatus * OrtApi::GetROCMProviderOptionsAsString ( const OrtROCMProviderOptions rocm_options,
OrtAllocator allocator,
char **  ptr 
)

Get serialized ROCm provider options string.

For example, "device_id=0;arena_extend_strategy=0;......"

Parameters
rocm_options- OrtROCMProviderOptions instance
allocator- a ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions() the specified allocator will be used to allocate continuous buffers for output strings and lengths.
ptr- is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.16.

◆ GetSequenceElementType()

OrtStatus * OrtApi::GetSequenceElementType ( const OrtSequenceTypeInfo sequence_type_info,
OrtTypeInfo **  type_info 
)

Get element type from an OrtSequenceTypeInfo.

This is used by WinML to support model reflection APIs.

Parameters
[in]sequence_type_info
[out]type_info

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetSessionConfigEntry()

OrtStatus * OrtApi::GetSessionConfigEntry ( const OrtSessionOptions options,
const char *  config_key,
char *  config_value,
size_t *  size 
)

Get a session configuration value.

Returns a failure status if the configuration key does not exist. The config_key and the format of config_value are defined in onnxruntime_session_options_config_keys.h

If config_value is nullptr, the value of size is set to the true size of the string value (including null-terminator), and a success status is returned.

If the size parameter is greater than or equal to the actual string value's size, the value of size is set to the true size of the string value, the provided memory is filled with the value's contents, and a success status is returned.

If the size parameter is less than the actual string value's size and config_value is not nullptr, the value of size is set to the true size of the string value and a failure status is returned.

Can be used in a custom operator library to get session configuration entries that target one or more custom operators in the library. Example: The config entry custom_op.myop.some_key targets a custom op named "myop".

Parameters
[in]optionsThe session options.
[in]config_keyA null-terminated UTF-8 string representation of the config key.
[in]config_valuePointer to memory where the null-terminated UTF-8 string value will be stored.
[in,out]sizePointer to the size of the config_value buffer. See above comments for details.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ GetSparseTensorFormat()

OrtStatus * OrtApi::GetSparseTensorFormat ( const OrtValue ort_value,
enum OrtSparseFormat out 
)

Returns sparse tensor format enum iff a given ort value contains an instance of sparse tensor.

Parameters
[in]ort_valueOrtValue that contains an instance of sparse tensor
[out]outpointer to out parameter

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetSparseTensorIndices()

OrtStatus * OrtApi::GetSparseTensorIndices ( const OrtValue ort_value,
enum OrtSparseIndicesFormat  indices_format,
size_t *  num_indices,
const void **  indices 
)

Returns indices data for the type of the indices specified by indices_format.

Parameters
[in]ort_valueOrtValue containing sparse tensor.
[in]indices_formatOne of the indices formats. It is an error to request a format that the sparse tensor does not contain.
[out]num_indicesPointer to where the number of indices entries is returned
[out]indicesReturned pointer to the indices data. Do not free the returned pointer as it refers to internal data owned by the OrtValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetSparseTensorIndicesTypeShape()

OrtStatus * OrtApi::GetSparseTensorIndicesTypeShape ( const OrtValue ort_value,
enum OrtSparseIndicesFormat  indices_format,
OrtTensorTypeAndShapeInfo **  out 
)

Returns data type, shape for the type of indices specified by indices_format.

Parameters
[in]ort_valueOrtValue containing sparse tensor.
[in]indices_formatOne of the indices formats. It is an error to request a format that the sparse tensor does not contain.
[out]outan instance of OrtTensorTypeAndShapeInfo. Must be freed by OrtApi::ReleaseTensorTypeAndShapeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetSparseTensorValues()

OrtStatus * OrtApi::GetSparseTensorValues ( const OrtValue ort_value,
const void **  out 
)

Returns numeric data for sparse tensor values (nnz). For string values use GetStringTensor*().

Parameters
[in]ort_valuean instance of OrtValue containing sparse tensor
[out]outreturns a pointer to values data. Do not attempt to free this ptr.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetSparseTensorValuesTypeAndShape()

OrtStatus * OrtApi::GetSparseTensorValuesTypeAndShape ( const OrtValue ort_value,
OrtTensorTypeAndShapeInfo **  out 
)

Returns data type and shape of sparse tensor values (nnz) iff OrtValue contains a SparseTensor.

Parameters
[in]ort_valueAn OrtValue that contains a fully constructed sparse tensor
[out]outMust be freed by OrtApi::ReleaseTensorTypeAndShapeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetStringTensorContent()

OrtStatus * OrtApi::GetStringTensorContent ( const OrtValue value,
void *  s,
size_t  s_len,
size_t *  offsets,
size_t  offsets_len 
)

Get all strings from a string tensor.

An example of the results:
Given value is a string tensor with the strings { "This" "is" "a" "test" }
s must have a size of 11 bytes
offsets must have 4 elements
After the call, these values will be filled in:
s will contain "Thisisatest"
offsets will contain { 0, 4, 6, 7 }
The length of the last string is just s_len - offsets[last]

Parameters
[in]valueA tensor of type ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING
[in]sBuffer to sequentially write all tensor strings to. Each string is NOT null-terminated.
[in]s_lenNumber of bytes of buffer pointed to by s (Get it from OrtApi::GetStringTensorDataLength)
[out]offsetsArray of start offsets into the strings written to s
[in]offsets_lenNumber of elements in offsets

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetStringTensorDataLength()

OrtStatus * OrtApi::GetStringTensorDataLength ( const OrtValue value,
size_t *  len 
)

Get total byte length for all strings in a string tensor.

Typically used with OrtApi::GetStringTensorContent

Parameters
[in]valueA tensor of type ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING
[out]lenTotal byte length of all strings (does not include trailing nulls)

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetStringTensorElement()

OrtStatus * OrtApi::GetStringTensorElement ( const OrtValue value,
size_t  s_len,
size_t  index,
void *  s 
)

Get a single string from a string tensor.

Parameters
[in]valueA string tensor
[in]s_lenNumber of bytes in the s buffer. Must match the value returned by OrtApi::GetStringTensorElementLength.
[in]indexIndex of the string in the tensor
[out]sThe string element contents in UTF-8 encoding. The string is NOT null-terminated.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetStringTensorElementLength()

OrtStatus * OrtApi::GetStringTensorElementLength ( const OrtValue value,
size_t  index,
size_t *  out 
)

Get the length of a single string in a string tensor.

Parameters
[in]valueA string tensor
[in]indexIndex of the string in the tensor
[out]outSet to number of bytes of the string element

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetSymbolicDimensions()

OrtStatus * OrtApi::GetSymbolicDimensions ( const OrtTensorTypeAndShapeInfo info,
const char *  dim_params[],
size_t  dim_params_length 
)

Get symbolic dimension names in OrtTensorTypeAndShapeInfo.

Parameters
[in]info
[in]dim_paramsArray with dim_params_length elements. On return filled with pointers to null terminated strings of the dimension names
[in]dim_params_lengthNumber of elements in dim_params. Use OrtApi::GetDimensionsCount to get this value

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTensorElementType()

OrtStatus * OrtApi::GetTensorElementType ( const OrtTensorTypeAndShapeInfo info,
enum ONNXTensorElementDataType out 
)

Get element type in OrtTensorTypeAndShapeInfo.

See also
OrtApi::SetTensorElementType
Parameters
[in]info
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTensorMemoryInfo()

OrtStatus * OrtApi::GetTensorMemoryInfo ( const OrtValue value,
const OrtMemoryInfo **  mem_info 
)

Returns a pointer to the OrtMemoryInfo of a Tensor.

Parameters
[in]valueOrtValue containing tensor.
[out]mem_infoOrtMemoryInfo of the tensor. Do NOT free the returned pointer. It is valid for the lifetime of the OrtValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTensorMutableData()

OrtStatus * OrtApi::GetTensorMutableData ( OrtValue value,
void **  out 
)

Get a pointer to the raw data inside a tensor.

Used to read/write/modify the internal tensor data directly.

Note
The returned pointer is valid until the value is destroyed.
Parameters
[in]valueA tensor type (string tensors are not supported)
[out]outFilled in with a pointer to the internal storage

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTensorRTProviderOptionsAsString()

OrtStatus * OrtApi::GetTensorRTProviderOptionsAsString ( const OrtTensorRTProviderOptionsV2 tensorrt_options,
OrtAllocator allocator,
char **  ptr 
)

Get serialized TensorRT provider options string.

For example, "trt_max_workspace_size=2147483648;trt_max_partition_iterations=10;trt_int8_enable=1;......"

Parameters
tensorrt_options- OrtTensorRTProviderOptionsV2 instance
allocator- a ptr to an instance of OrtAllocator obtained with OrtApi::CreateAllocator or OrtApi::GetAllocatorWithDefaultOptions the specified allocator will be used to allocate continuous buffers for output strings and lengths.
ptr- is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTensorRTProviderOptionsByName()

OrtStatus * OrtApi::GetTensorRTProviderOptionsByName ( const OrtTensorRTProviderOptionsV2 tensorrt_options,
const char *  key,
void **  ptr 
)

Get TensorRT EP provider option where its data type is pointer. If the data type of the provider option can be represented by string please use GetTensorRTProviderOptionsAsString.

Parameters
tensorrt_options- OrtTensorRTProviderOptionsV2 instance
key- Name of the provider option
ptr- A pointer to the instance that is kept by the provider option
Since
Version 1.16.

◆ GetTensorShapeElementCount()

OrtStatus * OrtApi::GetTensorShapeElementCount ( const OrtTensorTypeAndShapeInfo info,
size_t *  out 
)

Get total number of elements in a tensor shape from an OrtTensorTypeAndShapeInfo.

Return the number of elements specified by the tensor shape (all dimensions multiplied by each other). For 0 dimensions, 1 is returned. If any dimension is less than 0, the result is always -1.

Examples:
[] = 1
[1,3,4] = 12
[2,0,4] = 0
[-1,3,4] = -1

Parameters
[in]info
[out]outNumber of elements

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTensorTypeAndShape()

OrtStatus * OrtApi::GetTensorTypeAndShape ( const OrtValue value,
OrtTensorTypeAndShapeInfo **  out 
)

Get type and shape information from a tensor OrtValue.

Parameters
[in]valueMust be a tensor (not a map/sequence/etc) or will return failure
[out]outNewly created OrtTensorTypeAndShapeInfo. Must be freed with OrtApi::ReleaseTensorTypeAndShapeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetTypeInfo()

OrtStatus * OrtApi::GetTypeInfo ( const OrtValue value,
OrtTypeInfo **  out 
)

Get type information of an OrtValue.

Parameters
[in]value
[out]outNewly created OrtTypeInfo. Must be freed with OrtApi::ReleaseTypeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetValue()

OrtStatus * OrtApi::GetValue ( const OrtValue value,
int  index,
OrtAllocator allocator,
OrtValue **  out 
)

Get non tensor data from an OrtValue.

If value is of type ONNX_TYPE_MAP, you need to retrieve the keys and values separately. Use index=0 to retrieve keys and index=1 to retrieve values. If value is of type ONNX_TYPE_SEQUENCE, use index to retrieve the index'th element of the sequence.

Parameters
[in]value
[in]indexSee above for usage based on value type
[in]allocatorAllocator used to allocate OrtValue
[out]outCreated OrtValue that holds the element requested. Must be freed with OrtApi::ReleaseValue

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetValueCount()

OrtStatus * OrtApi::GetValueCount ( const OrtValue value,
size_t *  out 
)

Get non tensor value count from an OrtValue.

If value is of type ONNX_TYPE_MAP 2 will always be returned. For ONNX_TYPE_SEQUENCE the number of elements in the sequence will be returned

Parameters
[in]value
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ GetValueType()

OrtStatus * OrtApi::GetValueType ( const OrtValue value,
enum ONNXType out 
)

Get ONNXType of an OrtValue.

Parameters
[in]value
[out]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ HasSessionConfigEntry()

OrtStatus * OrtApi::HasSessionConfigEntry ( const OrtSessionOptions options,
const char *  config_key,
int *  out 
)

Checks if the given session configuration entry exists.

The config_key formats are defined in onnxruntime_session_options_config_keys.h

Can be used in a custom operator library to check for session configuration entries that target one or more custom operators in the library. Example: The config entry custom_op.myop.some_key targets a custom op named "myop".

Parameters
[in]optionsThe OrtSessionOptions instance.
[in]config_keyA null-terminated UTF-8 string representation of the configuration key.
[out]outPointer set to 1 if the entry exists and 0 otherwise.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ HasValue()

OrtStatus * OrtApi::HasValue ( const OrtValue value,
int *  out 
)

Sets out to 1 iff an optional type OrtValue has an element, 0 otherwise (OrtValue is None) Use this API to find if the optional type OrtValue is None or not. If the optional type OrtValue is not None, use the OrtValue just like any other OrtValue. For example, if you get an OrtValue that corresponds to Optional(tensor) and if HasValue() returns true, use it as tensor and so on.

Parameters
[in]valueInput OrtValue.
[out]outindicating if the input OrtValue contains data (1) or if it is a None (0)

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ InvokeOp()

OrtStatus * OrtApi::InvokeOp ( const OrtKernelContext context,
const OrtOp ort_op,
const OrtValue *const *  input_values,
int  input_count,
OrtValue *const *  output_values,
int  output_count 
)

: Invoke the operator created by OrtApi::CreateOp The inputs must follow the order as specified in onnx specification

Parameters
[in]contextKernel context
[in]ort_opOperator that has been created
[in]input_valuesArray of inputs
[in]input_countNumber of inputs
[in]output_valuesArray of outputs
[in]output_countNumber of outputs
Since
Version 1.12.

◆ IsSparseTensor()

OrtStatus * OrtApi::IsSparseTensor ( const OrtValue value,
int *  out 
)

Sets *out to 1 iff an OrtValue is a SparseTensor, and 0 otherwise.

Parameters
[in]valueexisting OrtValue
[out]outunless an error occurs, contains 1 iff the value contains an instance of sparse tensor or 0 otherwise.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ IsTensor()

OrtStatus * OrtApi::IsTensor ( const OrtValue value,
int *  out 
)

Return if an OrtValue is a tensor type.

Parameters
[in]valueA tensor type (string tensors are not supported)
[out]outSet to 1 iff OrtValue is a tensor, 0 otherwise

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelContext_GetAllocator()

OrtStatus * OrtApi::KernelContext_GetAllocator ( const OrtKernelContext context,
const OrtMemoryInfo mem_info,
OrtAllocator **  out 
)

Get Allocator from KernelContext for a specific memoryInfo. Please use C API ReleaseAllocator to release out object.

Parameters
[in]contextOrtKernelContext instance
[in]mem_infoOrtMemoryInfo instance
[out]outA pointer to OrtAllocator.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ KernelContext_GetGPUComputeStream()

OrtStatus * OrtApi::KernelContext_GetGPUComputeStream ( const OrtKernelContext context,
void **  out 
)

Used for custom operators, gets the GPU compute stream to use to launch the custom a GPU kernel.

See also
OrtCustomOp
Parameters
[in]contextOrtKernelContext instance
[out]outReturns pointer to a GPU compute stream that can be used to launch the custom GPU kernel. If retrieving the GPU compute stream is not relevant (GPU not enabled in the build, kernel partitioned to some other EP), then a nullptr is returned as the output param. Do not free or mutate the returned pointer as it refers to internal data owned by the underlying session. Only use it for custom kernel launching.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelContext_GetInput()

OrtStatus * OrtApi::KernelContext_GetInput ( const OrtKernelContext context,
size_t  index,
const OrtValue **  out 
)

Used for custom operators, get an input of a kernel.

The function attempts fetches the input of the kernel. If the input is optional and not present, the function returns success and out is set to nullptr.

Parameters
[in]contextOrtKernelContext instance
[in]indexSee KernelContext_GetInputCount for boundaries check.
[out]outOrtValue if the input is present otherwise is set nullptr

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelContext_GetInputCount()

OrtStatus * OrtApi::KernelContext_GetInputCount ( const OrtKernelContext context,
size_t *  out 
)

Used for custom operators, get the input count of a kernel.

See also
OrtCustomOp

◆ KernelContext_GetLogger()

OrtStatus * OrtApi::KernelContext_GetLogger ( const OrtKernelContext context,
const OrtLogger **  logger 
)

Get the runtime logger from OrtKernelContext.

Used in the KernelCompute callback of an OrtCustomOp to get a logger that can be used to log messages during inference.

Parameters
[in]contextAn instance of OrtKernelContext.
[out]loggerPointer set to the kernel context's OrtLogger. Owned by ONNX Runtime, so do not free.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15

◆ KernelContext_GetOutput()

OrtStatus * OrtApi::KernelContext_GetOutput ( OrtKernelContext context,
size_t  index,
const int64_t *  dim_values,
size_t  dim_count,
OrtValue **  out 
)

Used for custom operators, get an output of a kernel.

The function attempts fetches the output of the kernel. If the output is optional and not present, the function returns success and out is set to nullptr.

Parameters
[in]contextOrtKernelContext instance
[in]indexSee KernelContext_GetOutputCount for boundaries check.
[in]dim_valuesoutput dimensions
[in]dim_countnumber of dimensions
[out]outa ptr to OrtValue to output otherwise set to nullptr

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelContext_GetOutputCount()

OrtStatus * OrtApi::KernelContext_GetOutputCount ( const OrtKernelContext context,
size_t *  out 
)

Used for custom operators, get the output count of a kernel.

See also
OrtCustomOp

◆ KernelContext_GetResource()

OrtStatus * OrtApi::KernelContext_GetResource ( const OrtKernelContext context,
int  resource_version,
int  resource_id,
void **  resource 
)

Get a EP resource. E.g. a cuda stream or a cublas handle

Parameters
context- Kernel context
resource_version- Version of the resource
resource_id- Type of resource
resource- A pointer to returned resource
Since
Version 1.16.

◆ KernelContext_GetScratchBuffer()

OrtStatus * OrtApi::KernelContext_GetScratchBuffer ( const OrtKernelContext context,
const OrtMemoryInfo mem_info,
size_t  count_or_bytes,
void **  out 
)

Get scratch buffer from the corresponding allocator under the sepcific OrtMemoryInfo object. NOTE: callers are responsible to release this scratch buffer from the corresponding allocator.

Parameters
[in]contextOrtKernelContext instance
[in]mem_infoOrtMemoryInfo instance
[in]count_or_bytesHow many bytes is this scratch buffer
[out]outA pointer to the scrach buffer
Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelContext_ParallelFor()

OrtStatus * OrtApi::KernelContext_ParallelFor ( const OrtKernelContext context,
void(*)(void *, size_t)  fn,
size_t  total,
size_t  num_batch,
void *  usr_data 
)

Run fn in parallel

Parameters
[in]context
[in]fnFunction accepting usr_data and an integer as iterator
[in]totalThe number of times fn is to be invoked
[in]num_batchNumber of batches by which the "total" is to be divided in maximum. When zero, there is no limit
[in]usr_dataUser data to be passed back to fn
Since
Version 1.17.

◆ KernelInfo_GetInputCount()

OrtStatus * OrtApi::KernelInfo_GetInputCount ( const OrtKernelInfo info,
size_t *  out 
)

Get the number of inputs from OrtKernelInfo.

Used in the CreateKernel callback of an OrtCustomOp to query the number of inputs during kernel/session creation.

Parameters
[in]infoInstance of OrtKernelInfo.
[out]outPointer to variable assigned with the result on success.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ KernelInfo_GetInputName()

OrtStatus * OrtApi::KernelInfo_GetInputName ( const OrtKernelInfo info,
size_t  index,
char *  out,
size_t *  size 
)

Get the name of a OrtKernelInfo's input.

Used in the CreateKernel callback of an OrtCustomOp to query an input's name during kernel/session creation.

If out is nullptr, the value of size is set to the size of the name string (including null-terminator), and a success status is returned.

If the size parameter is greater than or equal to the name string's size, the value of size is set to the true size of the string (including null-terminator), the provided memory is filled with the string's contents, and a success status is returned.

If the size parameter is less than the actual string's size and out is not nullptr, the value of size is set to the true size of the string and a failure status is returned.

Parameters
[in]infoAn instance of OrtKernelInfo.
[in]indexThe index of the input name to get. Returns a failure status if out-of-bounds.
[out]outMemory location into which to write the UTF-8 null-terminated string representing the input's name.
[in,out]sizePointer to the size of the out buffer. See above comments for details.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ KernelInfo_GetInputTypeInfo()

OrtStatus * OrtApi::KernelInfo_GetInputTypeInfo ( const OrtKernelInfo info,
size_t  index,
OrtTypeInfo **  type_info 
)

Get the type information for a OrtKernelInfo's input.

Used in the CreateKernel callback of an OrtCustomOp to query the shape and type information of an input during kernel/session creation.

Parameters
[in]infoAn instance of OrtKernelInfo.
[in]indexWhich input to get the type information for
[out]type_infoPointer set to the resulting OrtTypeInfo. Must be freed with OrtApi::ReleaseTypeInfo.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ KernelInfo_GetLogger()

OrtStatus * OrtApi::KernelInfo_GetLogger ( const OrtKernelInfo info,
const OrtLogger **  logger 
)

Get the session logger from OrtKernelInfo.

Used in the CreateKernel callback of an OrtCustomOp to get a logger that can be used to log messages.

Parameters
[in]infoAn instance of OrtKernelInfo.
[out]loggerPointer set to the session's OrtLogger. Owned by ONNX Runtime, so do not free.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15

◆ KernelInfo_GetNodeName()

OrtStatus * OrtApi::KernelInfo_GetNodeName ( const OrtKernelInfo info,
char *  out,
size_t *  size 
)

Get the graph node name from OrtKernelInfo.

If out is nullptr, the value of size is set to the size of the name string (including null-terminator), and a success status is returned.

If the size parameter is greater than or equal to the name string's size, the value of size is set to the true size of the string (including null-terminator), the provided memory is filled with the string's contents, and a success status is returned.

If the size parameter is less than the actual string's size and out is not nullptr, the value of size is set to the true size of the string and a failure status is returned.

Can be used in a custom operator's CreateKernel callback to get the name of the operator's node name in the graph.

Parameters
[in]infoAn instance of OrtKernelInfo.
[out]outMemory location into which to write the UTF-8 null-terminated string representing the name.
[in,out]sizePointer to the size of the out buffer. See above comments for details.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15

◆ KernelInfo_GetOutputCount()

OrtStatus * OrtApi::KernelInfo_GetOutputCount ( const OrtKernelInfo info,
size_t *  out 
)

Get the number of outputs from OrtKernelInfo.

Used in the CreateKernel callback of an OrtCustomOp to query the number of outputs during kernel/session creation.

Parameters
[in]infoInstance of OrtKernelInfo.
[out]outPointer to variable assigned with the result on success.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ KernelInfo_GetOutputName()

OrtStatus * OrtApi::KernelInfo_GetOutputName ( const OrtKernelInfo info,
size_t  index,
char *  out,
size_t *  size 
)

Get the name of a OrtKernelInfo's output.

Used in the CreateKernel callback of an OrtCustomOp to query an output's name during kernel/session creation.

If out is nullptr, the value of size is set to the size of the name string (including null-terminator), and a success status is returned.

If the size parameter is greater than or equal to the name string's size, the value of size is set to the true size of the string (including null-terminator), the provided memory is filled with the string's contents, and a success status is returned.

If the size parameter is less than the actual string's size and out is not nullptr, the value of size is set to the true size of the string and a failure status is returned.

Parameters
[in]infoAn instance of OrtKernelInfo.
[in]indexThe index of the output name to get. Returns a failure status if out-of-bounds.
[out]outMemory location into which to write the UTF-8 null-terminated string representing the output's name.
[in,out]sizePointer to the size of the out buffer. See above comments for details.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ KernelInfo_GetOutputTypeInfo()

OrtStatus * OrtApi::KernelInfo_GetOutputTypeInfo ( const OrtKernelInfo info,
size_t  index,
OrtTypeInfo **  type_info 
)

Get the type information for a OrtKernelInfo's output.

Used in the CreateKernel callback of an OrtCustomOp to query the shape and type information of an output during kernel/session creation.

Parameters
[in]infoAn instance of OrtKernelInfo.
[in]indexWhich input to get the type information for
[out]type_infoPointer set to the resulting OrtTypeInfo. Must be freed with OrtApi::ReleaseTypeInfo.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ KernelInfoGetAllocator()

OrtStatus * OrtApi::KernelInfoGetAllocator ( const OrtKernelInfo info,
OrtMemType  mem_type,
OrtAllocator **  out 
)

Get allocator from KernelInfo for a specific memory type. Please use C API ReleaseAllocator to release out object.

Parameters
[in]infoOrtKernelInfo instance
[in]mem_typeOrtMemType object
[out]outA pointer to OrtAllocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetAttribute_float()

OrtStatus * OrtApi::KernelInfoGetAttribute_float ( const OrtKernelInfo info,
const char *  name,
float *  out 
)

Get a float stored as an attribute in the graph node.

Parameters
[in]infoOrtKernelInfo instance
[in]nameNull terminated string of the name of the attribute
[out]outPointer to memory where the attribute will be stored

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetAttribute_int64()

OrtStatus * OrtApi::KernelInfoGetAttribute_int64 ( const OrtKernelInfo info,
const char *  name,
int64_t *  out 
)

Fetch a 64-bit int stored as an attribute in the graph node.

Parameters
[in]infoOrtKernelInfo instance
[in]nameNull terminated string of the name of the attribute
[out]outPointer to memory where the attribute will be stored

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetAttribute_string()

OrtStatus * OrtApi::KernelInfoGetAttribute_string ( const OrtKernelInfo info,
const char *  name,
char *  out,
size_t *  size 
)

Fetch a string stored as an attribute in the graph node.

If out is nullptr, the value of size is set to the true size of the string attribute, and a success status is returned.

If the size parameter is greater than or equal to the actual string attribute's size, the value of size is set to the true size of the string attribute, the provided memory is filled with the attribute's contents, and a success status is returned.

If the size parameter is less than the actual string attribute's size and out is not nullptr, the value of size is set to the true size of the string attribute and a failure status is returned.)

Parameters
[in]infoOrtKernelInfo instance
[in]nameNull terminated string of the name of the attribute
[out]outPointer to memory where the attribute will be stored
[in,out]sizeSee above comments for details

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetAttribute_tensor()

OrtStatus * OrtApi::KernelInfoGetAttribute_tensor ( const OrtKernelInfo info,
const char *  name,
OrtAllocator allocator,
OrtValue **  out 
)

Get a OrtValue tensor stored as an attribute in the graph node.

Used in the CreateKernel callback of an OrtCustomOp to get a tensor attribute.

Parameters
[in]infoOrtKernelInfo instance.
[in]nameUTF-8 null-terminated string representing the attribute's name.
[in]allocatorAllocator used to allocate the internal tensor state.
[out]outReturns newly created OrtValue. Must be freed with OrtApi::ReleaseValue, which will also free internal tensor state allocated with the provided allocator.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetAttributeArray_float()

OrtStatus * OrtApi::KernelInfoGetAttributeArray_float ( const OrtKernelInfo info,
const char *  name,
float *  out,
size_t *  size 
)

Fetch an array of int64_t values stored as an attribute in the graph node.

If out is nullptr, the value of size is set to the true size of the attribute array's size, and a success status is returned.

If the size parameter is greater than or equal to the actual attribute array's size, the value of size is set to the true size of the attribute array's size, the provided memory is filled with the attribute's contents, and a success status is returned.

If the size parameter is less than the actual attribute array's size and out is not nullptr, the value of size is set to the true size of the attribute array's size and a failure status is returned.)

Parameters
[in]infoinstance
[in]namename of the attribute to be parsed
[out]outpointer to memory where the attribute's contents are to be stored
[in,out]sizeactual size of attribute array

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetAttributeArray_int64()

OrtStatus * OrtApi::KernelInfoGetAttributeArray_int64 ( const OrtKernelInfo info,
const char *  name,
int64_t *  out,
size_t *  size 
)

Fetch an array of int64_t values stored as an attribute in the graph node.

If out is nullptr, the value of size is set to the true size of the attribute array's size, and a success status is returned.

If the size parameter is greater than or equal to the actual attribute array's size, the value of size is set to the true size of the attribute array's size, the provided memory is filled with the attribute's contents, and a success status is returned.

If the size parameter is less than the actual attribute array's size and out is not nullptr, the value of size is set to the true size of the attribute array's size and a failure status is returned.)

Parameters
[in]infoinstance
[in]namename of the attribute to be parsed
[out]outpointer to memory where the attribute's contents are to be stored
[in,out]sizeactual size of attribute array

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ KernelInfoGetConstantInput_tensor()

OrtStatus * OrtApi::KernelInfoGetConstantInput_tensor ( const OrtKernelInfo info,
size_t  index,
int *  is_constant,
const OrtValue **  out 
)

Get a OrtValue tensor stored as a constant initializer in the graph node.

Used in the CreateKernel callback of an OrtCustomOp to get a tensor value.

Parameters
[in]infoOrtKernelInfo instance.
[in]indexThe node index.
[out]is_constantIs it a constant node input or not.
[out]outThe OrtValue tensor value.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ Logger_GetLoggingSeverityLevel()

OrtStatus * OrtApi::Logger_GetLoggingSeverityLevel ( const OrtLogger logger,
OrtLoggingLevel out 
)

Get the logging severity level of the OrtLogger.

Can be used in a custom operator to get the logging serverity level of the OrtLogger associated with the OrtKernelInfo.

Parameters
[in]loggerThe OrtLogger instance.
[out]outPointer to variable assigned with the logging severity level on success.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15

◆ Logger_LogMessage()

OrtStatus * OrtApi::Logger_LogMessage ( const OrtLogger logger,
OrtLoggingLevel  log_severity_level,
const char *  message,
const char *  file_path,
int  line_number,
const char *  func_name 
)

Logs a message at the given severity level using the provided OrtLogger.

Only messages with a severity level equal or greater than the OrtLogger's logging severity level are logged. Use OrtApi::Logger_GetLoggingSeverityLevel to get the OrtLogger's logging severity level.

Can be used in custom operators to log messages with the logger retrieved via OrtApi::KernelInfo_GetLogger.

Parameters
[in]loggerThe OrtLogger instance.
[in]log_severity_levelThe message's severity level.
[in]messageThe message to log.
[in]file_pathThe filepath of the file in which the message is logged. Usually the value of ORT_FILE.
[in]line_numberThe file line number in which the message is logged. Usually the value of LINE.
[in]func_nameThe name of the function in which the message is logged. Usually the value of FUNCTION.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15

◆ MemoryInfoGetId()

OrtStatus * OrtApi::MemoryInfoGetId ( const OrtMemoryInfo ptr,
int *  out 
)

Get the id from OrtMemoryInfo.

◆ MemoryInfoGetMemType()

OrtStatus * OrtApi::MemoryInfoGetMemType ( const OrtMemoryInfo ptr,
OrtMemType out 
)

Get the OrtMemType from OrtMemoryInfo.

◆ MemoryInfoGetName()

OrtStatus * OrtApi::MemoryInfoGetName ( const OrtMemoryInfo ptr,
const char **  out 
)

Get name from OrtMemoryInfo.

Parameters
[in]ptr
[out]outWrites null terminated string to this pointer. Do NOT free the returned pointer. It is valid for the lifetime of the OrtMemoryInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ MemoryInfoGetType()

OrtStatus * OrtApi::MemoryInfoGetType ( const OrtMemoryInfo ptr,
OrtAllocatorType out 
)

◆ ModelMetadataGetCustomMetadataMapKeys()

OrtStatus * OrtApi::ModelMetadataGetCustomMetadataMapKeys ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
char ***  keys,
int64_t *  num_keys 
)
Parameters
[in]model_metadata
[in]allocator
[out]keysArray of null terminated strings (array count = num_keys) allocated using allocator. The strings and the pointer array must be freed using allocator keys will be set to nullptr if the custom metadata map is empty.
[out]num_keysSet to the number of elements in the keys array

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataGetDescription()

OrtStatus * OrtApi::ModelMetadataGetDescription ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
char **  value 
)

Get description from an OrtModelMetadata.

Parameters
[in]model_metadata
[in]allocator
[out]valueSet to a null terminated string allocated using allocator. Must be freed using allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataGetDomain()

OrtStatus * OrtApi::ModelMetadataGetDomain ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
char **  value 
)

Get domain from an OrtModelMetadata.

Parameters
[in]model_metadata
[in]allocator
[out]valueSet to a null terminated string allocated using allocator. Must be freed using allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataGetGraphDescription()

OrtStatus * OrtApi::ModelMetadataGetGraphDescription ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
char **  value 
)

Use this to obtain the description of the graph present in the model (doc_string field of the GraphProto message within the ModelProto message). If it doesn't exist, an empty string will be returned.

Parameters
[in]model_metadataAn instance of OrtModelMetadata
[in]allocatorAllocator used to allocate the string that will be returned back
[out]valueSet to a null terminated string allocated using allocator. The caller is responsible for freeing it using allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataGetGraphName()

OrtStatus * OrtApi::ModelMetadataGetGraphName ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
char **  value 
)

Get graph name from an OrtModelMetadata.

Parameters
[in]model_metadata
[in]allocator
[out]valueSet to a null terminated string allocated using allocator. Must be freed using allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataGetProducerName()

OrtStatus * OrtApi::ModelMetadataGetProducerName ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
char **  value 
)

Get producer name from an OrtModelMetadata.

Parameters
[in]model_metadata
[in]allocator
[out]valueSet to a null terminated string allocated using allocator. Must be freed using allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataGetVersion()

OrtStatus * OrtApi::ModelMetadataGetVersion ( const OrtModelMetadata model_metadata,
int64_t *  value 
)

Get version number from an OrtModelMetadata.

Parameters
[in]model_metadata
[out]valueSet to the version number

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ModelMetadataLookupCustomMetadataMap()

OrtStatus * OrtApi::ModelMetadataLookupCustomMetadataMap ( const OrtModelMetadata model_metadata,
OrtAllocator allocator,
const char *  key,
char **  value 
)

Return data for a key in the custom metadata map in an OrtModelMetadata.

Parameters
[in]model_metadata
[in]allocator
[in]keyNull terminated string
[out]valueSet to a null terminated string allocated using allocator. Must be freed using allocator value will be set to nullptr if the given key is not found in the custom metadata map.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ReadOpAttr()

OrtStatus * OrtApi::ReadOpAttr ( const OrtOpAttr op_attr,
OrtOpAttrType  type,
void *  data,
size_t  len,
size_t *  out 
)

Read contents of an attribute to data

Parameters
[in]op_attr
[in]typeAttribute type
[out]dataMemory address to save raw content of the attribute
[in]lenNumber of bytes allowed to store in data
[out]outNumber of bytes required to save the data when the call failed, or the real number of bytes saved to data on success
Since
Version 1.17.

◆ RegisterAllocator()

OrtStatus * OrtApi::RegisterAllocator ( OrtEnv env,
OrtAllocator allocator 
)

Register a custom allocator.

Enables sharing between multiple sessions that use the same env instance. Returns an error if an allocator with the same OrtMemoryInfo is already registered.

The behavior of this is exactly the same as OrtApi::CreateAndRegisterAllocator except instead of ORT creating an allocator based on provided info, in this case ORT uses the user-provided custom allocator. See https://onnxruntime.ai/docs/get-started/with-c.html for details.

Parameters
[in]env
[in]allocatorUser provided allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RegisterCustomOpsLibrary()

OrtStatus * OrtApi::RegisterCustomOpsLibrary ( OrtSessionOptions options,
const char *  library_path,
void **  library_handle 
)
Deprecated:
Use OrtApi::RegisterCustomOpsLibrary_V2.

Registers custom ops from a shared library.

Loads a shared library (dll on windows, so on linux, etc) named 'library_path' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions * options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base. The handle to the loaded library is returned in library_handle. It can be freed by the caller after all sessions using the passed in session options are destroyed, or if an error occurs and it is non null.

Parameters
[in]options
[in]library_path
[out]library_handleOS specific handle to the loaded library (Use FreeLibrary on Windows, dlclose on Linux, etc.. to unload)

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RegisterCustomOpsLibrary_V2()

OrtStatus * OrtApi::RegisterCustomOpsLibrary_V2 ( OrtSessionOptions options,
const char *  library_name 
)

Register custom ops from a shared library.

Loads a shared library (.dll on windows, .so on linux, etc) named 'library_name' and looks for this entry point: OrtStatus* RegisterCustomOps(OrtSessionOptions * options, const OrtApiBase* api); It then passes in the provided session options to this function along with the api base.

The handle to the loaded library is automatically released by ORT when the last OrtSession that references the library handle is released. If no OrtSession is created, then the library handle is released when the provided OrtSessionOptions is released.

Parameters
[in]optionsThe session options.
[in]library_nameThe name of the shared library to load and register. Refer to OS-specific dynamic library loading utilities (e.g., LoadLibraryEx on Windows or dlopen on Linux/MacOS) for information on the format of library names and search paths.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ RegisterCustomOpsUsingFunction()

OrtStatus * OrtApi::RegisterCustomOpsUsingFunction ( OrtSessionOptions options,
const char *  registration_func_name 
)

Register custom ops by calling a RegisterCustomOpsFn function.

Searches for registration_func_name and if found calls it.

The library containing the function must either be linked against or previously loaded by the executable.

If you want ONNX Runtime to load the library and manage its lifetime, use RegisterCustomOpsLibrary_V2.

RegisterCustomOpsUsingFunction can be used in scenarios where it may not be possible for ONNX Runtime to load the library from a path. e.g. mobile platforms where the library must be linked into the app.

The registration function must have the signature of RegisterCustomOpsFn: OrtStatus* (fn)(OrtSessionOptions options, const OrtApiBase* api);

See https://onnxruntime.ai/docs/reference/operators/add-custom-op.html for details on how the registration function should be implemented.

Parameters
[in]optionsOrtSessionOptions that is passed through as the first argument in the call to the registration function.
[in]registration_func_nameName of registration function to use.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.14

◆ ReleaseAllocator()

void OrtApi::ReleaseAllocator ( OrtAllocator input)

Release an OrtAllocator obtained from OrtApi::CreateAllocator.

◆ ReleaseArenaCfg()

void OrtApi::ReleaseArenaCfg ( OrtArenaCfg input)

◆ ReleaseAvailableProviders()

OrtStatus * OrtApi::ReleaseAvailableProviders ( char **  ptr,
int  providers_length 
)

Release data from OrtApi::GetAvailableProviders. This API will never fail so you can rely on it in a noexcept code.

Parameters
[in]ptrThe out_ptr result from OrtApi::GetAvailableProviders.
[in]providers_lengthThe provider_length result from OrtApi::GetAvailableProviders

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ ReleaseCustomOpDomain()

void OrtApi::ReleaseCustomOpDomain ( OrtCustomOpDomain input)

◆ ReleaseEnv()

void OrtApi::ReleaseEnv ( OrtEnv input)

◆ ReleaseIoBinding()

void OrtApi::ReleaseIoBinding ( OrtIoBinding input)

Release an OrtIoBinding obtained from OrtApi::CreateIoBinding.

◆ ReleaseKernelInfo()

void OrtApi::ReleaseKernelInfo ( OrtKernelInfo input)

◆ ReleaseLoraAdapter()

void OrtApi::ReleaseLoraAdapter ( OrtLoraAdapter input)

Release an OrtLoraAdapter obtained from OrtApi::CreateLoraAdapter.

◆ ReleaseMapTypeInfo()

void OrtApi::ReleaseMapTypeInfo ( OrtMapTypeInfo input)

◆ ReleaseMemoryInfo()

void OrtApi::ReleaseMemoryInfo ( OrtMemoryInfo input)

◆ ReleaseModelMetadata()

void OrtApi::ReleaseModelMetadata ( OrtModelMetadata input)

◆ ReleaseOp()

void OrtApi::ReleaseOp ( OrtOp input)

◆ ReleaseOpAttr()

void OrtApi::ReleaseOpAttr ( OrtOpAttr input)

◆ ReleasePrepackedWeightsContainer()

void OrtApi::ReleasePrepackedWeightsContainer ( OrtPrepackedWeightsContainer input)

Release OrtPrepackedWeightsContainer instance.

Note
instance must not be released until the sessions using it are released

◆ ReleaseRunOptions()

void OrtApi::ReleaseRunOptions ( OrtRunOptions input)

◆ ReleaseSequenceTypeInfo()

void OrtApi::ReleaseSequenceTypeInfo ( OrtSequenceTypeInfo input)

◆ ReleaseSession()

void OrtApi::ReleaseSession ( OrtSession input)

◆ ReleaseSessionOptions()

void OrtApi::ReleaseSessionOptions ( OrtSessionOptions input)

◆ ReleaseStatus()

void OrtApi::ReleaseStatus ( OrtStatus input)

◆ ReleaseTensorTypeAndShapeInfo()

void OrtApi::ReleaseTensorTypeAndShapeInfo ( OrtTensorTypeAndShapeInfo input)

◆ ReleaseThreadingOptions()

void OrtApi::ReleaseThreadingOptions ( OrtThreadingOptions input)

◆ ReleaseTypeInfo()

void OrtApi::ReleaseTypeInfo ( OrtTypeInfo input)

◆ ReleaseValue()

void OrtApi::ReleaseValue ( OrtValue input)

◆ Run()

OrtStatus * OrtApi::Run ( OrtSession session,
const OrtRunOptions run_options,
const char *const *  input_names,
const OrtValue *const *  inputs,
size_t  input_len,
const char *const *  output_names,
size_t  output_names_len,
OrtValue **  outputs 
)

Run the model in an OrtSession.

Will not return until the model run has completed. Multiple threads might be used to run the model based on the options in the OrtSession and settings used when creating the OrtEnv

Parameters
[in]session
[in]run_optionsIf nullptr, will use a default OrtRunOptions
[in]input_namesArray of null terminated UTF8 encoded strings of the input names
[in]inputsArray of OrtValues of the input values
[in]input_lenNumber of elements in the input_names and inputs arrays
[in]output_namesArray of null terminated UTF8 encoded strings of the output names
[in]output_names_lenNumber of elements in the output_names and outputs array
[out]outputsArray of OrtValues that the outputs are stored in. This can also be an array of nullptr values, in this case OrtValue objects will be allocated and pointers to them will be set into the outputs array.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RunAsync()

OrtStatus * OrtApi::RunAsync ( OrtSession session,
const OrtRunOptions run_options,
const char *const *  input_names,
const OrtValue *const *  input,
size_t  input_len,
const char *const *  output_names,
size_t  output_names_len,
OrtValue **  output,
RunAsyncCallbackFn  run_async_callback,
void *  user_data 
)

Run the model asynchronously in a thread owned by intra op thread pool.

Parameters
[in]session
[in]run_optionsIf nullptr, will use a default OrtRunOptions
[in]input_namesArray of null terminated UTF8 encoded strings of the input names
[in]inputArray of OrtValues of the input values
[in]input_lenNumber of elements in the input_names and inputs arrays
[in]output_namesArray of null terminated UTF8 encoded strings of the output names
[in]output_names_lenNumber of elements in the output_names and outputs array
[out]outputOrtValue* array of size output_names_len. On calling RunAsync, output[i] could either be a null or a pointer to a preallocated OrtValue. Later, the output array will be passed to run_async_callback with all null(s) filled with valid OrtValue pointer(s) allocated by onnxruntime. NOTE: it is customer's duty to finally release the output array and each of its member, regardless of whether the member (OrtValue*) is allocated by onnxruntime or preallocated by the customer.
[in]run_async_callbackCallback function on model run completion
[in]user_dataUser data that pass back to run_async_callback

◆ RunOptionsAddActiveLoraAdapter()

OrtStatus * OrtApi::RunOptionsAddActiveLoraAdapter ( OrtRunOptions options,
const OrtLoraAdapter adapter 
)

Add the Lora Adapter to the list of active adapters.

The function adds the Lora Adapter to the list of active adapters. The Lora Adapter must be created with OrtApi::CreateLoraAdapter or FromArray. The Lora Adapter will be used by the session to run the model. The instance of the OrtRunOptions can then be used to customize the Run() calls. More than one OrtLoraAdapter can be active at the same time. Lora Parameters that belong to different Lora adapters that will be active at the same time must not overlap. This setting does not affect RunWithBinding.

Parameters
[in]optionsOrtRunOptions instance
[in]adapterOrtLoraAdapter instance

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RunOptionsGetRunLogSeverityLevel()

OrtStatus * OrtApi::RunOptionsGetRunLogSeverityLevel ( const OrtRunOptions options,
int *  log_severity_level 
)

Get per-run log severity level.

See also
OrtApi::RunOptionsSetRunLogSeverityLevel
Parameters
[in]options
[out]log_severity_levelThe log severity level (refer to OrtLoggingLevel for possible values).

◆ RunOptionsGetRunLogVerbosityLevel()

OrtStatus * OrtApi::RunOptionsGetRunLogVerbosityLevel ( const OrtRunOptions options,
int *  log_verbosity_level 
)

Get per-run log verbosity level.

See also
OrtApi::RunOptionsSetRunLogVerbosityLevel
Parameters
[in]options
[out]log_verbosity_levelThe log verbosity level controls the display of ORT_LOGGING_LEVEL_VERBOSE severity log messages at a finer granularity. It only has an effect in debug builds. The higher the value, the more verbose the log output will be.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RunOptionsGetRunTag()

OrtStatus * OrtApi::RunOptionsGetRunTag ( const OrtRunOptions options,
const char **  run_tag 
)

Get per-run tag.

This is used in a per-run log identifier.

See also
OrtApi::RunOptionsSetRunTag
Parameters
[in]options
[out]run_tagThe run tag. Do not free this value, it is owned by options. It will be invalidated if the run tag changes (i.e., with OrtApi::RunOptionsSetRunTag) or options is freed.

◆ RunOptionsSetRunLogSeverityLevel()

OrtStatus * OrtApi::RunOptionsSetRunLogSeverityLevel ( OrtRunOptions options,
int  log_severity_level 
)

Set per-run log severity level.

See also
OrtApi::RunOptionsGetRunLogSeverityLevel
Parameters
[in]options
[in]log_severity_levelThe log severity level (refer to OrtLoggingLevel for possible values).

◆ RunOptionsSetRunLogVerbosityLevel()

OrtStatus * OrtApi::RunOptionsSetRunLogVerbosityLevel ( OrtRunOptions options,
int  log_verbosity_level 
)

Set per-run log verbosity level.

See also
OrtApi::RunOptionsGetRunLogVerbosityLevel
Parameters
[in]options
[in]log_verbosity_levelThe log verbosity level controls the display of ORT_LOGGING_LEVEL_VERBOSE severity log messages at a finer granularity. It only has an effect in debug builds. The higher the value, the more verbose the log output will be.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RunOptionsSetRunTag()

OrtStatus * OrtApi::RunOptionsSetRunTag ( OrtRunOptions options,
const char *  run_tag 
)

Set per-run tag.

This is used in a per-run log identifier.

See also
OrtApi::RunOptionsGetRunTag
Parameters
[in]options
[in]run_tagThe run tag.

◆ RunOptionsSetTerminate()

OrtStatus * OrtApi::RunOptionsSetTerminate ( OrtRunOptions options)

Set terminate flag.

If a currently executing session needs to be force terminated, this can be called from another thread to force it to fail with an error.

Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RunOptionsUnsetTerminate()

OrtStatus * OrtApi::RunOptionsUnsetTerminate ( OrtRunOptions options)

Clears the terminate flag.

Used so the OrtRunOptions instance can be used in a new OrtApi::Run call without it instantly terminating

Parameters
[in]options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ RunWithBinding()

OrtStatus * OrtApi::RunWithBinding ( OrtSession session,
const OrtRunOptions run_options,
const OrtIoBinding binding_ptr 
)

Run a model using Io Bindings for the inputs & outputs.

See also
OrtApi::Run
Parameters
[in]session
[in]run_options
[in]binding_ptr

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionEndProfiling()

OrtStatus * OrtApi::SessionEndProfiling ( OrtSession session,
OrtAllocator allocator,
char **  out 
)

End profiling and return filename of the profile data.

Profiling is turned on through OrtApi::EnableProfiling

Parameters
[in]session
[in]allocator
[out]outNull terminated string of the filename, allocated using allocator. Must be freed using allocator

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetInputCount()

OrtStatus * OrtApi::SessionGetInputCount ( const OrtSession session,
size_t *  out 
)

Get input count for a session.

This number must also match the number of inputs passed to OrtApi::Run

See also
OrtApi::SessionGetInputTypeInfo, OrtApi::SessionGetInputName, OrtApi::Session
Parameters
[in]session
[out]outNumber of inputs

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetInputName()

OrtStatus * OrtApi::SessionGetInputName ( const OrtSession session,
size_t  index,
OrtAllocator allocator,
char **  value 
)

Get input name.

Parameters
[in]session
[in]indexMust be between 0 (inclusive) and what OrtApi::SessionGetInputCount returns (exclusive)
[in]allocator
[out]valueSet to a null terminated UTF-8 encoded string allocated using allocator. Must be freed using allocator.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetInputTypeInfo()

OrtStatus * OrtApi::SessionGetInputTypeInfo ( const OrtSession session,
size_t  index,
OrtTypeInfo **  type_info 
)

Get input type information.

Parameters
[in]session
[in]indexMust be between 0 (inclusive) and what OrtApi::SessionGetInputCount returns (exclusive)
[out]type_infoMust be freed with OrtApi::ReleaseTypeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetModelMetadata()

OrtStatus * OrtApi::SessionGetModelMetadata ( const OrtSession session,
OrtModelMetadata **  out 
)

Get OrtModelMetadata from an OrtSession.

Parameters
[in]session
[out]outNewly created OrtModelMetadata. Must be freed using OrtApi::ReleaseModelMetadata

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetOutputCount()

OrtStatus * OrtApi::SessionGetOutputCount ( const OrtSession session,
size_t *  out 
)

Get output count for a session.

This number must also match the number of outputs returned by OrtApi::Run

See also
OrtApi::SessionGetOutputTypeInfo, OrtApi::SessionGetOutputName, OrtApi::Session
Parameters
[in]session
[out]outNumber of outputs

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetOutputName()

OrtStatus * OrtApi::SessionGetOutputName ( const OrtSession session,
size_t  index,
OrtAllocator allocator,
char **  value 
)

Get output name.

Parameters
[in]session
[in]indexMust be between 0 (inclusive) and what OrtApi::SessionGetOutputCount returns (exclusive)
[in]allocator
[out]valueSet to a null terminated UTF-8 encoded string allocated using allocator. Must be freed using allocator.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetOutputTypeInfo()

OrtStatus * OrtApi::SessionGetOutputTypeInfo ( const OrtSession session,
size_t  index,
OrtTypeInfo **  type_info 
)

Get output type information.

Parameters
[in]session
[in]indexMust be between 0 (inclusive) and what OrtApi::SessionGetOutputCount returns (exclusive)
[out]type_infoMust be freed with OrtApi::ReleaseTypeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetOverridableInitializerCount()

OrtStatus * OrtApi::SessionGetOverridableInitializerCount ( const OrtSession session,
size_t *  out 
)

Get overridable initializer count.

See also
OrtApi::SessionGetOverridableInitializerTypeInfo, OrtApi::SessionGetOverridableInitializerName
Parameters
[in]session
[in]out

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetOverridableInitializerName()

OrtStatus * OrtApi::SessionGetOverridableInitializerName ( const OrtSession session,
size_t  index,
OrtAllocator allocator,
char **  value 
)

Get overridable initializer name.

Parameters
[in]session
[in]indexMust be between 0 (inclusive) and what OrtApi::SessionGetOverridableInitializerCount returns (exclusive)
[in]allocator
[out]valueSet to a null terminated UTF-8 encoded string allocated using allocator. Must be freed using allocator.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetOverridableInitializerTypeInfo()

OrtStatus * OrtApi::SessionGetOverridableInitializerTypeInfo ( const OrtSession session,
size_t  index,
OrtTypeInfo **  type_info 
)

Get overridable initializer type information.

Parameters
[in]session
[in]indexMust be between 0 (inclusive) and what OrtApi::SessionGetOverridableInitializerCount returns (exclusive)
[out]type_infoMust be freed with OrtApi::ReleaseTypeInfo

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionGetProfilingStartTimeNs()

OrtStatus * OrtApi::SessionGetProfilingStartTimeNs ( const OrtSession session,
uint64_t *  out 
)

Return the time that profiling was started.

Note
The timer precision varies per platform. On Windows and MacOS, the precision will be ~100ns
Parameters
[in]session
[out]outnanoseconds of profiling's start time

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider ( OrtSessionOptions options,
const char *  provider_name,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

: Append execution provider to the session options.

Parameters
[in]options
[in]provider_name- provider to add.
[in]provider_options_keys- keys to configure the provider options
[in]provider_options_values- values to configure the provider options
[in]num_keys- number of keys passed in

Currently supported providers: QNN SNPE XNNPACK

Note: If an execution provider has a dedicated SessionOptionsAppendExecutionProvider_<provider name> function that should be used to add it.

QNN supported keys: "backend_path": file path to QNN backend library. "profiling_level": QNN profiling level, options: "off", "basic", "detailed". Default to off. "profiling_file_path": QNN profiling file path if ETW not enabled. "rpc_control_latency": QNN RPC control latency. "vtcm_mb": QNN VTCM size in MB. default to 0(not set). "htp_performance_mode": QNN performance mode, options: "burst", "balanced", "default", "high_performance", "high_power_saver", "low_balanced", "extreme_power_saver", "low_power_saver", "power_saver", "sustained_high_performance". Default to "default". "qnn_saver_path": File path to the QNN Saver backend library. If specified, QNN Saver will be enabled and will dump QNN API calls to disk for replay/debugging. QNN Saver produces incorrect model inference results and may alter model/EP partitioning. Use only for debugging. "qnn_context_priority": QNN context priority, options: "low", "normal", "normal_high", "high". Default to "normal". "htp_graph_finalization_optimization_mode": Set the optimization mode for graph finalization on the HTP backend. Available options:

  • "0": Default.
  • "1": Faster preparation time, less optimal graph.
  • "2": Longer preparation time, more optimal graph.
  • "3": Longest preparation time, most likely even more optimal graph. See QNN SDK documentation for specific details. "soc_model": The SoC model number. Refer to the QNN SDK documentation for valid values. Defaults to "0" (unknown). "htp_arch": The minimum HTP architecture the driver will use to select compatible QNN operators. Available options:
  • "0": Default (none).
  • "68"
  • "69"
  • "73"
  • "75" "device_id": The ID of the device to use when setting 'htp_arch'. Defaults to "0" (for single device). "enable_htp_fp16_precision": Used for float32 model for HTP backend. Enable the float32 model to be inferenced with fp16 precision. Otherwise, it will be fp32 precision.
  • "0": With fp32 precision.
  • "1": Default. With fp16 precision. "enable_htp_weight_sharing": Enable QNN weight sharing feature while compiling multiple graphs into one QNN context.
  • "0": Default. Disabled.
  • "1": Enabled. "offload_graph_io_quantization": Offload graph input quantization and graph output dequantization to another execution provider (typically CPU EP).
  • "0": Default. Disabled. QNN EP will handle quantization and dequantization of graph I/O.
  • "1": Enabled.

SNPE supported keys: "runtime": SNPE runtime engine, options: "CPU", "CPU_FLOAT32", "GPU", "GPU_FLOAT32_16_HYBRID", "GPU_FLOAT16", "DSP", "DSP_FIXED8_TF", "AIP_FIXED_TF", "AIP_FIXED8_TF". Mapping to SNPE Runtime_t definition: CPU, CPU_FLOAT32 => zdl::DlSystem::Runtime_t::CPU; GPU, GPU_FLOAT32_16_HYBRID => zdl::DlSystem::Runtime_t::GPU; GPU_FLOAT16 => zdl::DlSystem::Runtime_t::GPU_FLOAT16; DSP, DSP_FIXED8_TF => zdl::DlSystem::Runtime_t::DSP. AIP_FIXED_TF, AIP_FIXED8_TF => zdl::DlSystem::Runtime_t::AIP_FIXED_TF. "priority": execution priority, options: "low", "normal". "buffer_type": ITensor or user buffers, options: "ITENSOR", user buffer with different types - "TF8", "TF16", "UINT8", "FLOAT". "ITENSOR" – default, ITensor which is float only. "TF8" – quantized model required, "FLOAT" – for both quantized or non-quantized model "enable_init_cache": enable SNPE init caching feature, set to 1 to enabled it. Disabled by default. If SNPE is not available (due to a non Snpe enabled build or its dependencies not being installed), this function will fail.

XNNPACK supported keys: "intra_op_num_threads": number of thread-pool size to use for XNNPACK execution provider. default value is 0, which means to use the session thread-pool size.

Since
Version 1.12.

◆ SessionOptionsAppendExecutionProvider_CANN()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_CANN ( OrtSessionOptions options,
const OrtCANNProviderOptions cann_options 
)

Append CANN provider to session options.

If CANN is not available (due to a non CANN enabled build, or if CANN is not installed on the system), this function will return failure.

Parameters
[in]options
[in]cann_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.13.

◆ SessionOptionsAppendExecutionProvider_CUDA()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_CUDA ( OrtSessionOptions options,
const OrtCUDAProviderOptions cuda_options 
)

Append CUDA provider to session options.

If CUDA is not available (due to a non CUDA enabled build, or if CUDA is not installed on the system), this function will return failure.

Parameters
[in]options
[in]cuda_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider_CUDA_V2()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_CUDA_V2 ( OrtSessionOptions options,
const OrtCUDAProviderOptionsV2 cuda_options 
)

Append CUDA execution provider to the session options.

If CUDA is not available (due to a non CUDA enabled build), this function will return failure.

This is slightly different from OrtApi::SessionOptionsAppendExecutionProvider_CUDA, it takes an OrtCUDAProviderOptions which is publicly defined. This takes an opaque OrtCUDAProviderOptionsV2 which must be created with OrtApi::CreateCUDAProviderOptions.

For OrtApi::SessionOptionsAppendExecutionProvider_CUDA, the user needs to instantiate OrtCUDAProviderOptions as well as allocate/release buffers for some members of OrtCUDAProviderOptions. Here, OrtApi::CreateCUDAProviderOptions and Ortapi::ReleaseCUDAProviderOptions will do the memory management for you.

Parameters
[in]options
[in]cuda_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.11.

◆ SessionOptionsAppendExecutionProvider_Dnnl()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_Dnnl ( OrtSessionOptions options,
const OrtDnnlProviderOptions dnnl_options 
)

Append dnnl provider to session options.

If oneDNN is not available, this function will return failure.

Parameters
[in]options
[in]dnnl_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ SessionOptionsAppendExecutionProvider_MIGraphX()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_MIGraphX ( OrtSessionOptions options,
const OrtMIGraphXProviderOptions migraphx_options 
)

Append MIGraphX provider to session options.

If MIGraphX is not available (due to a non MIGraphX enabled build, or if MIGraphX is not installed on the system), this function will return failure.

Parameters
[in]options
[in]migraphx_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.11.

◆ SessionOptionsAppendExecutionProvider_OpenVINO()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_OpenVINO ( OrtSessionOptions options,
const OrtOpenVINOProviderOptions provider_options 
)

Append OpenVINO execution provider to the session options.

If OpenVINO is not available (due to a non OpenVINO enabled build, or if OpenVINO is not installed on the system), this function will fail.

Parameters
[in]options
[in]provider_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider_OpenVINO_V2()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_OpenVINO_V2 ( OrtSessionOptions options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Append OpenVINO execution provider to the session options.

If OpenVINO is not available (due to a non OpenVINO enabled build, or if OpenVINO is not installed on the system), this function will fail.

Parameters
[in]options
[in]provider_options_keys
[in]provider_options_values
[in]num_keys

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider_ROCM()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_ROCM ( OrtSessionOptions options,
const OrtROCMProviderOptions rocm_options 
)

Append ROCM execution provider to the session options.

If ROCM is not available (due to a non ROCM enabled build, or if ROCM is not installed on the system), this function will return failure.

Parameters
[in]options
[in]rocm_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider_TensorRT()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_TensorRT ( OrtSessionOptions options,
const OrtTensorRTProviderOptions tensorrt_options 
)

Append TensorRT provider to session options.

If TensorRT is not available (due to a non TensorRT enabled build, or if TensorRT is not installed on the system), this function will return failure.

Parameters
[in]options
[in]tensorrt_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider_TensorRT_V2()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_TensorRT_V2 ( OrtSessionOptions options,
const OrtTensorRTProviderOptionsV2 tensorrt_options 
)

Append TensorRT execution provider to the session options.

If TensorRT is not available (due to a non TensorRT enabled build), this function will return failure.

This is slightly different from OrtApi::SessionOptionsAppendExecutionProvider_TensorRT, it takes an OrtTensorRTProviderOptions which is publicly defined. This takes an opaque OrtTensorRTProviderOptionsV2 which must be created with OrtApi::CreateTensorRTProviderOptions.

For OrtApi::SessionOptionsAppendExecutionProvider_TensorRT, the user needs to instantiate OrtTensorRTProviderOptions as well as allocate/release buffers for some members of OrtTensorRTProviderOptions. Here, OrtApi::CreateTensorRTProviderOptions and Ortapi::ReleaseTensorRTProviderOptions will do the memory management for you.

Parameters
[in]options
[in]tensorrt_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsAppendExecutionProvider_VitisAI()

OrtStatus * OrtApi::SessionOptionsAppendExecutionProvider_VitisAI ( OrtSessionOptions options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Append VitisAI provider to session options.

If VitisAI is not available (due to a non VitisAI enabled build, or if VitisAI is not installed on the system), this function will return failure.

Parameters
[in]options
[in]provider_options_keys
[in]provider_options_values
[in]num_keys

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsSetCustomCreateThreadFn()

OrtStatus * OrtApi::SessionOptionsSetCustomCreateThreadFn ( OrtSessionOptions options,
OrtCustomCreateThreadFn  ort_custom_create_thread_fn 
)

Set custom thread creation function.

Parameters
[in]optionsSession options
[in]ort_custom_create_thread_fnCustom thread creation function

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsSetCustomJoinThreadFn()

OrtStatus * OrtApi::SessionOptionsSetCustomJoinThreadFn ( OrtSessionOptions options,
OrtCustomJoinThreadFn  ort_custom_join_thread_fn 
)

Set custom thread join function.

Parameters
[in]optionsSession options
[in]ort_custom_join_thread_fnCustom join thread function, must not be nullptr when ort_custom_create_thread_fn is set

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SessionOptionsSetCustomThreadCreationOptions()

OrtStatus * OrtApi::SessionOptionsSetCustomThreadCreationOptions ( OrtSessionOptions options,
void *  ort_custom_thread_creation_options 
)

Set creation options for custom thread.

Parameters
[in]optionsSession options
[in]ort_custom_thread_creation_optionsCustom thread creation options (can be nullptr)

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetCurrentGpuDeviceId()

OrtStatus * OrtApi::SetCurrentGpuDeviceId ( int  device_id)

Set current GPU device ID.

Set the current device id of the GPU execution provider (CUDA/tensorrt/rocm). The device id should be less than the total number of devices available. This is only useful when multiple-GPUs are installed and it is required to restrict execution to a single GPU.

Parameters
[in]device_id

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetDeterministicCompute()

OrtStatus * OrtApi::SetDeterministicCompute ( OrtSessionOptions options,
bool  value 
)

Set whether to use deterministic compute.

Default is false. If set to true, this will enable deterministic compute for GPU kernels where possible. Note that this most likely will have a performance cost.

Parameters
[in]options
[in]value
Since
Version 1.17.

◆ SetDimensions()

OrtStatus * OrtApi::SetDimensions ( OrtTensorTypeAndShapeInfo info,
const int64_t *  dim_values,
size_t  dim_count 
)

Set shape information in OrtTensorTypeAndShapeInfo.

Parameters
[in]info
[in]dim_valuesArray with dim_count elements. Can contain negative values.
[in]dim_countNumber of elements in dim_values

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetEpDynamicOptions()

OrtStatus * OrtApi::SetEpDynamicOptions ( OrtSession sess,
const char *const *  keys,
const char *const *  values,
size_t  kv_len 
)

Set DynamicOptions for EPs (Execution Providers)

Valid options can be found in include\onnxruntime\core\session\onnxruntime_session_options_config_keys.h Look for kOrtEpDynamicOptions

Parameters
[in]sessOrtSession
[in]keysArray of null terminated UTF8 encoded strings of EP dynamic option keys
[in]valuesArray of null terminated UTF8 encoded string of EP dynamic option values
[in]kv_lenNumber of elements in the keys and values arrays

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalCustomCreateThreadFn()

OrtStatus * OrtApi::SetGlobalCustomCreateThreadFn ( OrtThreadingOptions tp_options,
OrtCustomCreateThreadFn  ort_custom_create_thread_fn 
)

Set custom thread creation function for global thread pools.

Parameters
[in,out]tp_options
[in]ort_custom_create_thread_fnCustom thread creation function

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalCustomJoinThreadFn()

OrtStatus * OrtApi::SetGlobalCustomJoinThreadFn ( OrtThreadingOptions tp_options,
OrtCustomJoinThreadFn  ort_custom_join_thread_fn 
)

Set custom thread join function for global thread pools.

Parameters
[in,out]tp_options
[in]ort_custom_join_thread_fnCustom thread join function, must not be nullptr when global ort_custom_create_thread_fn is set

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalCustomThreadCreationOptions()

OrtStatus * OrtApi::SetGlobalCustomThreadCreationOptions ( OrtThreadingOptions tp_options,
void *  ort_custom_thread_creation_options 
)

Set custom thread creation options for global thread pools.

Parameters
[in,out]tp_options
[in]ort_custom_thread_creation_optionsCustom thread creation options (can be nullptr)

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalDenormalAsZero()

OrtStatus * OrtApi::SetGlobalDenormalAsZero ( OrtThreadingOptions tp_options)

Set threading flush-to-zero and denormal-as-zero.

Sets global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools. Flush-to-zero and denormal-as-zero are applied to threads in both intra and inter global thread pool.

Note
This option is not needed if the models used have no denormals. Having no denormals is recommended as this option may hurt model accuracy.
Parameters
[in]tp_options

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalInterOpNumThreads()

OrtStatus * OrtApi::SetGlobalInterOpNumThreads ( OrtThreadingOptions tp_options,
int  inter_op_num_threads 
)

Set global inter-op thread count.

This configures the global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools

Parameters
[in]tp_options
[in]inter_op_num_threadsNumber of threads, special values:
0 = Use default thread count
1 = The invoking thread will be used; no threads will be created in the thread pool.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalIntraOpNumThreads()

OrtStatus * OrtApi::SetGlobalIntraOpNumThreads ( OrtThreadingOptions tp_options,
int  intra_op_num_threads 
)

Set global intra-op thread count.

This configures the global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools

Parameters
[in]tp_options
[in]intra_op_num_threadsNumber of threads, special values:
0 = Use default thread count
1 = The invoking thread will be used; no threads will be created in the thread pool.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetGlobalIntraOpThreadAffinity()

OrtStatus * OrtApi::SetGlobalIntraOpThreadAffinity ( OrtThreadingOptions tp_options,
const char *  affinity_string 
)

◆ SetGlobalSpinControl()

OrtStatus * OrtApi::SetGlobalSpinControl ( OrtThreadingOptions tp_options,
int  allow_spinning 
)

Set global spin control options.

This will configure the global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools. Allow spinning of thread pools when their queues are empty. This will set the value for both inter_op and intra_op threadpools.

Parameters
[in]tp_options
[in]allow_spinningValid values are 0 or 1.
0 = It won't spin (recommended if CPU usage is high)
1 = Threadpool will spin to wait for queue to become non-empty

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetInterOpNumThreads()

OrtStatus * OrtApi::SetInterOpNumThreads ( OrtSessionOptions options,
int  inter_op_num_threads 
)

Sets the number of threads used to parallelize the execution of the graph.

If nodes can be run in parallel, this sets the maximum number of threads to use to run them in parallel.

Note
If sequential execution is enabled this value is ignored, it acts as if it was set to 1.
Parameters
[in]options
[in]inter_op_num_threadsNumber of threads to use
A value of 0 will use the default number of threads

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetIntraOpNumThreads()

OrtStatus * OrtApi::SetIntraOpNumThreads ( OrtSessionOptions options,
int  intra_op_num_threads 
)

Sets the number of threads used to parallelize the execution within nodes.

When running a single node operation, ex. add, this sets the maximum number of threads to use.

Note
If built with OpenMP, this has no effect on the number of threads used. In this case use the OpenMP env variables to configure the number of intra op num threads.
Parameters
[in]options
[in]intra_op_num_threadsNumber of threads to use
A value of 0 will use the default number of threads

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetLanguageProjection()

OrtStatus * OrtApi::SetLanguageProjection ( const OrtEnv ort_env,
OrtLanguageProjection  projection 
)

Set language projection.

Set the language projection for collecting telemetry data when Env is created.

The default is ORT_PROJECTION_C, which means it will classify the language not in the list to C also.

Parameters
[in]ort_env
[in]projection

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetOptimizedModelFilePath()

OrtStatus * OrtApi::SetOptimizedModelFilePath ( OrtSessionOptions options,
const char *  optimized_model_filepath 
)

Set filepath to save optimized model after graph level transformations.

Parameters
[in]options
[in]optimized_model_filepath

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetSessionExecutionMode()

OrtStatus * OrtApi::SetSessionExecutionMode ( OrtSessionOptions options,
ExecutionMode  execution_mode 
)

Set execution mode.

Controls whether you want to execute operators in your graph sequentially or in parallel. Usually when the model has many branches, setting this option to ExecutionMode.ORT_PARALLEL will give you better performance. See [docs/ONNX_Runtime_Perf_Tuning.md] for more details.

Parameters
[in]options
[in]execution_mode

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetSessionGraphOptimizationLevel()

OrtStatus * OrtApi::SetSessionGraphOptimizationLevel ( OrtSessionOptions options,
GraphOptimizationLevel  graph_optimization_level 
)

Set the optimization level to apply when loading a graph.

Please see https://onnxruntime.ai/docs/performance/model-optimizations/graph-optimizations.html for an in-depth explanation

Parameters
[in,out]optionsThe session options object
[in]graph_optimization_levelThe optimization level

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetSessionLogId()

OrtStatus * OrtApi::SetSessionLogId ( OrtSessionOptions options,
const char *  logid 
)

Set session log id.

Parameters
[in]options
[in]logidThe log identifier.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetSessionLogSeverityLevel()

OrtStatus * OrtApi::SetSessionLogSeverityLevel ( OrtSessionOptions options,
int  session_log_severity_level 
)

Set session log severity level.

Parameters
[in]options
[in]session_log_severity_levelThe log severity level (refer to OrtLoggingLevel for possible values).

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetSessionLogVerbosityLevel()

OrtStatus * OrtApi::SetSessionLogVerbosityLevel ( OrtSessionOptions options,
int  session_log_verbosity_level 
)

Set session log verbosity level.

Applies to session load, initialization, etc

Parameters
[in]options
[in]session_log_verbosity_levelThe log verbosity level controls the display of ORT_LOGGING_LEVEL_VERBOSE severity log messages at a finer granularity. It only has an effect in debug builds. The higher the value, the more verbose the log output will be.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetSymbolicDimensions()

OrtStatus * OrtApi::SetSymbolicDimensions ( OrtTensorTypeAndShapeInfo info,
const char *  dim_params[],
size_t  dim_params_length 
)

Set symbolic shape to type shape info

Parameters
[in]infoType shape info
[in]dim_paramsSymbolic strings
[in]dim_params_lengthNumber of strings
Since
Version 1.17.

◆ SetTensorElementType()

OrtStatus * OrtApi::SetTensorElementType ( OrtTensorTypeAndShapeInfo info,
enum ONNXTensorElementDataType  type 
)

Set element type in OrtTensorTypeAndShapeInfo.

Parameters
[in]info
[in]type

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SetUserLoggingFunction()

OrtStatus * OrtApi::SetUserLoggingFunction ( OrtSessionOptions options,
OrtLoggingFunction  user_logging_function,
void *  user_logging_param 
)

Set user logging function.

By default the logger created by the CreateEnv* functions is used to create the session logger as well. This function allows a user to override this default session logger with a logger of their own choosing. This way the user doesn't have to create a separate environment with a custom logger. This addresses the problem when the user already created an env but now wants to use a different logger for a specific session (for debugging or other reasons).

Parameters
[in]options
[in]user_logging_functionA pointer to a logging function.
[in]user_logging_paramA pointer to arbitrary data passed as the OrtLoggingFunction param parameter to user_logging_function. This parameter is optional.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.17.

◆ ShapeInferContext_GetAttribute()

OrtStatus * OrtApi::ShapeInferContext_GetAttribute ( const OrtShapeInferContext context,
const char *  attr_name,
const OrtOpAttr **  attr 
)

Get attribute from OrtShapeInferContext. Note that OrtShapeInferContext is a per-node context, one could only read attribute from current node.

Parameters
[in]context
[in]attr_nameName of the attribute
[out]attrHandle of the attribute fetched
Since
Version 1.17.

◆ ShapeInferContext_GetInputCount()

OrtStatus * OrtApi::ShapeInferContext_GetInputCount ( const OrtShapeInferContext context,
size_t *  out 
)

Get number of input from OrtShapeInferContext

Parameters
[in]context
[out]outThe number of inputs
Since
Version 1.17.

◆ ShapeInferContext_GetInputTypeShape()

OrtStatus * OrtApi::ShapeInferContext_GetInputTypeShape ( const OrtShapeInferContext context,
size_t  index,
OrtTensorTypeAndShapeInfo **  info 
)

Get type and shape info of an input

Parameters
[in]context
[in]indexThe index of the input
[out]infoType shape info of the input
Since
Version 1.17.

◆ ShapeInferContext_SetOutputTypeShape()

OrtStatus * OrtApi::ShapeInferContext_SetOutputTypeShape ( const OrtShapeInferContext context,
size_t  index,
const OrtTensorTypeAndShapeInfo info 
)

Set type and shape info of an output

Parameters
[in]context
[in]indexThe index of the output
[out]infoType shape info of the output
Since
Version 1.17.

◆ SynchronizeBoundInputs()

OrtStatus * OrtApi::SynchronizeBoundInputs ( OrtIoBinding binding_ptr)

Synchronize bound inputs. The call may be necessary for some providers, such as cuda, in case the system that allocated bound memory operated on a different stream. However, the operation is provider specific and could be a no-op.

Parameters
[in,out]binding_ptr

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ SynchronizeBoundOutputs()

OrtStatus * OrtApi::SynchronizeBoundOutputs ( OrtIoBinding binding_ptr)

Synchronize bound outputs. The call may be necessary for some providers, such as cuda, in case the system that allocated bound memory operated on a different stream. However, the operation is provider specific and could be a no-op.

Parameters
[in,out]binding_ptr

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ TensorAt()

OrtStatus * OrtApi::TensorAt ( OrtValue value,
const int64_t *  location_values,
size_t  location_values_count,
void **  out 
)

Direct memory access to a specified tensor element.

For example, given a tensor with shape of [3,224,224], a pointer to the element at location [2,150,128] can be retrieved

This function only works for numeric type tensors (No strings, etc). This is a no-copy method whose returned pointer is valid until the passed in OrtValue is free'd.

Parameters
[in]value
[in]location_valuesPointer to an array of index values that specify an element's location relative to its shape
[in]location_values_countNumber of elements in location_values. Must match the number of elements in the tensor's shape.
[out]outSet to a pointer to the element specified

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ UnregisterAllocator()

OrtStatus * OrtApi::UnregisterAllocator ( OrtEnv env,
const OrtMemoryInfo mem_info 
)

Unregister a custom allocator.

It is an error if you provide an OrtMemoryInfo not corresponding to any registered allocators for sharing.

Parameters
[in]env
[in]mem_info

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ UpdateCANNProviderOptions()

OrtStatus * OrtApi::UpdateCANNProviderOptions ( OrtCANNProviderOptions cann_options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Set options in a CANN Execution Provider.

Parameters
[in]cann_options
[in]provider_options_keysArray of UTF-8 null-terminated string for provider options keys
[in]provider_options_valuesArray of UTF-8 null-terminated string for provider options values
[in]num_keysNumber of elements in the provider_option_keys and provider_options_values arrays

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.13.

◆ UpdateCUDAProviderOptions()

OrtStatus * OrtApi::UpdateCUDAProviderOptions ( OrtCUDAProviderOptionsV2 cuda_options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Set options in a CUDA Execution Provider.

Please refer to https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#configuration-options to know the available keys and values. Key should be in null terminated string format of the member of OrtCUDAProviderOptionsV2 and value should be its related range. Recreates the options and only sets the supplied values.

For example, key="device_id" and value="0"

Parameters
[in]cuda_options
[in]provider_options_keysArray of UTF-8 null-terminated string for provider options keys
[in]provider_options_valuesArray of UTF-8 null-terminated string for provider options values
[in]num_keysNumber of elements in the provider_option_keys and provider_options_values arrays

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.11.

◆ UpdateCUDAProviderOptionsWithValue()

OrtStatus * OrtApi::UpdateCUDAProviderOptionsWithValue ( OrtCUDAProviderOptionsV2 cuda_options,
const char *  key,
void *  value 
)

Update CUDA EP provider option where its data type is pointer, for example 'user_compute_stream'. If the data type of the provider option can be represented by string please use UpdateCUDAProviderOptions.

Note: It's caller's responsibility to properly manage the lifetime of the instance pointed by this pointer.

Parameters
cuda_options- OrtCUDAProviderOptionsV2 instance
key- Name of the provider option
value- A pointer to the instance that will be assigned to this provider option
Since
Version 1.16.

◆ UpdateDnnlProviderOptions()

OrtStatus * OrtApi::UpdateDnnlProviderOptions ( OrtDnnlProviderOptions dnnl_options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Set options in a oneDNN Execution Provider.

Key should be in null terminated string format of the member of OrtDnnlProviderOptions and value should be its related range.

For example, key="use_arena" and value="1"

Parameters
[in]dnnl_options
[in]provider_options_keysArray of UTF-8 null-terminated string for provider options keys
[in]provider_options_valuesArray of UTF-8 null-terminated string for provider options values
[in]num_keysNumber of elements in the provider_option_keys and provider_options_values arrays

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.15.

◆ UpdateEnvWithCustomLogLevel()

OrtStatus * OrtApi::UpdateEnvWithCustomLogLevel ( OrtEnv ort_env,
OrtLoggingLevel  log_severity_level 
)

◆ UpdateROCMProviderOptions()

OrtStatus * OrtApi::UpdateROCMProviderOptions ( OrtROCMProviderOptions rocm_options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Set options in a ROCm Execution Provider.

Please refer to https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html to know the available keys and values. Key should be in null terminated string format of the member of OrtROCMProviderOptions and value should be its related range.

For example, key="device_id" and value="0"

Parameters
[in]rocm_options
[in]provider_options_keysArray of UTF-8 null-terminated string for provider options keys
[in]provider_options_valuesArray of UTF-8 null-terminated string for provider options values
[in]num_keysNumber of elements in the provider_option_keys and provider_options_values arrays

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since
Version 1.16.

◆ UpdateTensorRTProviderOptions()

OrtStatus * OrtApi::UpdateTensorRTProviderOptions ( OrtTensorRTProviderOptionsV2 tensorrt_options,
const char *const *  provider_options_keys,
const char *const *  provider_options_values,
size_t  num_keys 
)

Set options in a TensorRT Execution Provider.

Please refer to https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#cc to know the available keys and values. Key should be in null terminated string format of the member of OrtTensorRTProviderOptionsV2 and value should be its related range. Recreates the options and only sets the supplied values.

For example, key="trt_max_workspace_size" and value="2147483648"

Parameters
[in]tensorrt_options
[in]provider_options_keysArray of UTF-8 null-terminated string for provider options keys
[in]provider_options_valuesArray of UTF-8 null-terminated string for provider options values
[in]num_keysNumber of elements in the provider_option_keys and provider_options_values arrays

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ UpdateTensorRTProviderOptionsWithValue()

OrtStatus * OrtApi::UpdateTensorRTProviderOptionsWithValue ( OrtTensorRTProviderOptionsV2 tensorrt_options,
const char *  key,
void *  value 
)

Update TensorRT EP provider option where its data type is pointer, for example 'user_compute_stream'. If the data type of the provider option can be represented by string please use UpdateTensorRTProviderOptions.

Note: It's caller's responsibility to properly manage the lifetime of the instance pointed by this pointer.

Parameters
tensorrt_options- OrtTensorRTProviderOptionsV2 instance
key- Name of the provider option
value- A pointer to the instance that will be assigned to this provider option
Since
Version 1.16.

◆ UseBlockSparseIndices()

OrtStatus * OrtApi::UseBlockSparseIndices ( OrtValue ort_value,
const int64_t *  indices_shape,
size_t  indices_shape_len,
int32_t *  indices_data 
)

The assigns BlockSparse format indices to the SparseTensor that was created by OrtApi::CreateSparseTensorWithValuesAsOrtValue above. It also sets OrtSparseFormat to ORT_SPARSE_BLOCK_SPARSE. This will not allocate any additional memory for data. The life span of indices_data buffer must eclipse the lifespan of this OrtValue.

Parameters
[in,out]ort_valueOrtValue instance constructed with OrtApi::CreateSparseTensorWithValuesAsOrtValue
[in]indices_shapepointer to indices shape. Use {0} for fully sparse tensors
[in]indices_shape_lenlength of the indices shape
[in,out]indices_datapointer to user pre-allocated buffer or nullptr for fully sparse tensors.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ UseCooIndices()

OrtStatus * OrtApi::UseCooIndices ( OrtValue ort_value,
int64_t *  indices_data,
size_t  indices_num 
)

This assigns Coo format indices to the SparseTensor that was created by OrtApi::CreateSparseTensorWithValuesAsOrtValue above. It also sets OrtSparseFormat to ORT_SPARSE_COO. This will not allocate any additional memory for data. The life span of indices_data buffer should eclipse the life span of this OrtValue.

Parameters
[in,out]ort_valueOrtValue instance constructed with OrtApi::CreateSparseTensorWithValuesAsOrtValue
[in,out]indices_datapointer to a user pre-allocated buffer or nullptr for fully sparse tensors.
[in]indices_numnumber of COO indices. Should either be 0 for fully sparse tensors, be equal to the number of nnz values specified to OrtApi::CreateSparseTensorWithValuesAsOrtValue for 1-D {nnz} indices or be twice as number of nnz values for a 2-D indices {nnz, 2}

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

◆ UseCsrIndices()

OrtStatus * OrtApi::UseCsrIndices ( OrtValue ort_value,
int64_t *  inner_data,
size_t  inner_num,
int64_t *  outer_data,
size_t  outer_num 
)

The assigns CSR format indices to the SparseTensor that was created by OrtApi::CreateSparseTensorWithValuesAsOrtValue above. It also sets OrtSparseFormat to ORT_SPARSE_CSRC. This will not allocate any additional memory for data. The life spans of inner_data and outer_data buffers should eclipse the life span of this OrtValue.

Parameters
[in,out]ort_valueOrtValue instance constructed with OrtApi::CreateSparseTensorWithValuesAsOrtValue
[in,out]inner_datapointer to a user pre-allocated buffer or nullptr for fully sparse tensors.
[in]inner_numnumber of inner CSR indices. Should either be 0 for fully sparse tensors or be equal to the number of nnz values specified to OrtApi::CreateSparseTensorWithValuesAsOrtValue.
[in,out]outer_datapointer to user pre-allocated buffer or nullptr for fully sparse tensors.
[in]outer_numnumber of CSR outer indices. Should either be 0 for fully sparse tensors or equal to rows + 1 of the dense shape.

Returns
If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Member Data Documentation

◆ ClearBoundInputs

void( * OrtApi::ClearBoundInputs) (OrtIoBinding *binding_ptr) __attribute__((nonnull))

Clears any previously set Inputs for an OrtIoBinding.

◆ ClearBoundOutputs

void( * OrtApi::ClearBoundOutputs) (OrtIoBinding *binding_ptr) __attribute__((nonnull))

Clears any previously set Outputs for an OrtIoBinding.

◆ CreateStatus

OrtStatus *( * OrtApi::CreateStatus) (OrtErrorCode code, const char *msg) __attribute__((nonnull))

Create an OrtStatus from a null terminated string.

Parameters
[in]code
[in]msgA null-terminated string. Its contents will be copied.
Returns
A new OrtStatus object, must be destroyed with OrtApi::ReleaseStatus

◆ GetBuildInfoString

const char *( * OrtApi::GetBuildInfoString) (void)

Returns a null terminated string of the build info including git info and cxx flags.

Returns
UTF-8 encoded version string. Do not deallocate the returned buffer.
Since
Version 1.15.

◆ GetErrorCode

OrtErrorCode( * OrtApi::GetErrorCode) (const OrtStatus *status) __attribute__((nonnull))

Get OrtErrorCode from OrtStatus.

Parameters
[in]status
Returns
OrtErrorCode that status was created with

◆ GetErrorMessage

const char *( * OrtApi::GetErrorMessage) (const OrtStatus *status) __attribute__((nonnull))

Get error string from OrtStatus.

Parameters
[in]status
Returns
The error message inside the status. Do not free the returned value.

◆ GetTrainingApi

const OrtTrainingApi *( * OrtApi::GetTrainingApi) (uint32_t version)

Gets the Training C Api struct.

Call this function to access the OrtTrainingApi structure that holds pointers to functions that enable training with onnxruntime.

Note
A NULL pointer will be returned and no error message will be printed if the training api is not supported with this build. A NULL pointer will be returned and an error message will be printed if the provided version is unsupported, for example when using a runtime older than the version created with this header file.
Parameters
[in]versionMust be ORT_API_VERSION
Returns
The OrtTrainingApi struct for the version requested.
Since
Version 1.13

◆ MemoryInfoGetDeviceType

void( * OrtApi::MemoryInfoGetDeviceType) (const OrtMemoryInfo *ptr, OrtMemoryInfoDeviceType *out)

◆ ReleaseCANNProviderOptions

void( * OrtApi::ReleaseCANNProviderOptions) (OrtCANNProviderOptions *input)

Release an OrtCANNProviderOptions.

Parameters
[in]inputThe pointer of OrtCANNProviderOptions which will been deleted
Since
Version 1.13.

◆ ReleaseCUDAProviderOptions

void( * OrtApi::ReleaseCUDAProviderOptions) (OrtCUDAProviderOptionsV2 *input)

Release an OrtCUDAProviderOptionsV2.

Note
This is an exception in the naming convention of other Release* functions, as the name of the method does not have the V2 suffix, but the type does
Since
Version 1.11.

◆ ReleaseDnnlProviderOptions

void( * OrtApi::ReleaseDnnlProviderOptions) (OrtDnnlProviderOptions *input)

Release an OrtDnnlProviderOptions.

Since
Version 1.15.

◆ ReleaseROCMProviderOptions

void( * OrtApi::ReleaseROCMProviderOptions) (OrtROCMProviderOptions *input)

Release an OrtROCMProviderOptions.

Note
This is an exception in the naming convention of other Release* functions, as the name of the method does not have the V2 suffix, but the type does
Since
Version 1.16.

◆ ReleaseTensorRTProviderOptions

void( * OrtApi::ReleaseTensorRTProviderOptions) (OrtTensorRTProviderOptionsV2 *input)

Release an OrtTensorRTProviderOptionsV2.

Note
This is an exception in the naming convention of other Release* functions, as the name of the method does not have the V2 suffix, but the type does