The OrtCompileApi struct provides functions to compile ONNX models. More...

#include <onnxruntime_c_api.h>

Public Member Functions
OrtModelCompilationOptions
void	ReleaseModelCompilationOptions (OrtModelCompilationOptions *input)

OrtStatus *	CreateModelCompilationOptionsFromSessionOptions (const OrtEnv env, const OrtSessionOptions session_options, OrtModelCompilationOptions **out)
	Creates an OrtModelCompilationOptions object from an existing OrtSessionOptions object.

OrtStatus *	ModelCompilationOptions_SetInputModelPath (OrtModelCompilationOptions model_compile_options, const char input_model_path)
	Sets the file path to the input ONNX model to compile.

OrtStatus *	ModelCompilationOptions_SetInputModelFromBuffer (OrtModelCompilationOptions model_compile_options, const void input_model_data, size_t input_model_data_size)
	Sets the buffer that stores the bytes of the loaded ONNX model to compile.

OrtStatus *	ModelCompilationOptions_SetOutputModelPath (OrtModelCompilationOptions model_compile_options, const char output_model_path)
	Sets the file path for the output ONNX model generated by CompileModel.

OrtStatus *	ModelCompilationOptions_SetOutputModelExternalInitializersFile (OrtModelCompilationOptions model_compile_options, const char external_initializers_file_path, size_t external_initializers_size_threshold)
	Optionally sets the file that should store external initializers for the compiled ONNX model. If not set, initializers are stored within the model.

OrtStatus *	ModelCompilationOptions_SetOutputModelBuffer (OrtModelCompilationOptions model_compile_options, OrtAllocator allocator, void *output_model_buffer_ptr, size_t output_model_buffer_size_ptr)
	Configures model compilation to store the output compiled ONNX model in a buffer.

OrtStatus *	ModelCompilationOptions_SetEpContextEmbedMode (OrtModelCompilationOptions *model_compile_options, bool embed_ep_context_in_model)
	Enables or disables the embedding of EPContext binary data into the `ep_cache_context` attribute of EPContext nodes. Defaults to false.

OrtStatus *	CompileModel (const OrtEnv env, const OrtModelCompilationOptions model_options)
	Compiles an input ONNX model with the given compilation options.

Detailed Description

The OrtCompileApi struct provides functions to compile ONNX models.

ORT Compile API

Execution providers that support compilation fuse a subgraph into an EPContext node that wraps a provider-specific binary representation of the subgraph. For more details about the EPContext design, refer to: EPContext design document.

Example (error handling not shown): OrtStatus* status = NULL; OrtCompileApi* compile_api = ort_api->GetCompileApi(); OrtModelCompilationOptions* compile_options = NULL;

status = compile_api->CreateModelCompilationOptionsFromSessionOptions(env, session_options, &compile_options); status = compile_api->ModelCompilationOptions_SetInputModelPath(compile_options, ORT_TSTR("model.onnx")); status = compile_api->ModelCompilationOptions_SetOutputModelPath(compile_options, ORT_TSTR("model.compiled.onnx")); status = compile_api->CompileModel(env, compile_options); compile_api->ReleaseModelCompilationOptions(compile_options);

Since: Version 1.22.

Member Function Documentation

◆ CompileModel()

OrtStatus * OrtCompileApi::CompileModel	(	const OrtEnv *	env,
		const OrtModelCompilationOptions *	model_options
	)

Compiles an input ONNX model with the given compilation options.

Parameters

[in]	env	OrtEnv object.
[in]	model_options	The compilation options that defines compilation options for a model.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ CreateModelCompilationOptionsFromSessionOptions()

OrtStatus * OrtCompileApi::CreateModelCompilationOptionsFromSessionOptions	(	const OrtEnv *	env,
		const OrtSessionOptions *	session_options,
		OrtModelCompilationOptions **	out
	)

Creates an OrtModelCompilationOptions object from an existing OrtSessionOptions object.

An OrtModelCompilationOptions object contains the settings used to generate a compiled ONNX model. The OrtSessionOptions object has the execution providers with which the model will be compiled.

ReleaseOrtModelCompilationsOptions must be called to free the OrtModelCompilationOptions after calling CompileModel.

Parameters

[in]	env	OrtEnv object.
[in]	session_options	The OrtSessionOptions instance from which to create the OrtModelCompilationOptions.
[out]	out	The created OrtModelCompilationOptions instance. Returns If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ModelCompilationOptions_SetEpContextEmbedMode()

OrtStatus * OrtCompileApi::ModelCompilationOptions_SetEpContextEmbedMode	(	OrtModelCompilationOptions *	model_compile_options,
		bool	embed_ep_context_in_model
	)

Enables or disables the embedding of EPContext binary data into the ep_cache_context attribute of EPContext nodes. Defaults to false.

If enabled, the ep_cache_context attribute of EPContext nodes will store the context binary data, which may include weights for compiled subgraphs.

If disabled, the ep_cache_context attribute of EPContext nodes will contain the path to the file containing the context binary data. The path is set by the execution provider creating the EPContext node.

More details relate to EPContext design refers to: EPContext design document.

Parameters

[in]	model_compile_options	The OrtModelCompilationOptions instance.
[in]	embed_ep_context_in_model	True to embed EPContext binary data into the EPContext node `ep_cache_context` attributes.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ModelCompilationOptions_SetInputModelFromBuffer()

OrtStatus * OrtCompileApi::ModelCompilationOptions_SetInputModelFromBuffer	(	OrtModelCompilationOptions *	model_compile_options,
		const void *	input_model_data,
		size_t	input_model_data_size
	)

Sets the buffer that stores the bytes of the loaded ONNX model to compile.

The input model's location (e.g., file path or memory buffer) must be set with either ModelCompilationOptions_SetInputModelPath or ModelCompilationOptions_SetInputModelFromBuffer.

Parameters

[in]	model_compile_options	The OrtModelCompilationOptions instance.
[in]	input_model_data	Buffer containing the loaded ONNX model bytes.
[in]	input_model_data_size	The number of bytes in the `input_model_data` buffer.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ModelCompilationOptions_SetInputModelPath()

OrtStatus * OrtCompileApi::ModelCompilationOptions_SetInputModelPath	(	OrtModelCompilationOptions *	model_compile_options,
		const char *	input_model_path
	)

Sets the file path to the input ONNX model to compile.

The input model's location (e.g., file path or memory buffer) must be set with either ModelCompilationOptions_SetInputModelPath or ModelCompilationOptions_SetInputModelFromBuffer.

Parameters

[in]	model_compile_options	The OrtModelCompilationOptions instance.
[in]	input_model_path	Null terminated string of the path (wchar on Windows, char otherwise).

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ModelCompilationOptions_SetOutputModelBuffer()

OrtStatus * OrtCompileApi::ModelCompilationOptions_SetOutputModelBuffer	(	OrtModelCompilationOptions *	model_compile_options,
		OrtAllocator *	allocator,
		void **	output_model_buffer_ptr,
		size_t *	output_model_buffer_size_ptr
	)

Configures model compilation to store the output compiled ONNX model in a buffer.

The caller passes an OrtAllocator that ONNX Runtime uses to allocate memory for the buffer.

The output model's location (e.g., file path or memory buffer) can be set with either ModelCompilationOptions_SetOutputModelPath or ModelCompilationOptions_SetOutputModelBuffer.

If the output model's location is not set, ONNX Runtime will generate an output file with a path based on the input model's file path. Examples: /Path/my_model.onnx -> /Path/my_model_ctx.onnx /Path/my_model -> /Path/my_model_ctx.onnx

Parameters

[in]	model_compile_options	The OrtModelCompilationOptions instance.
[in]	allocator	The allocator used to allocate the buffer for the compiled model.
[out]	output_model_buffer_ptr	Pointer to the buffer that stores the compiled model.
[out]	output_model_buffer_size_ptr	Pointer set to the size of output model in bytes.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ModelCompilationOptions_SetOutputModelExternalInitializersFile()

OrtStatus * OrtCompileApi::ModelCompilationOptions_SetOutputModelExternalInitializersFile	(	OrtModelCompilationOptions *	model_compile_options,
		const char *	external_initializers_file_path,
		size_t	external_initializers_size_threshold
	)

Optionally sets the file that should store external initializers for the compiled ONNX model. If not set, initializers are stored within the model.

Only initializers for nodes that were not compiled are stored in the external initializers file. Compiled nodes contain their initializer data within the ep_cache_context attribute of EPContext nodes. Refer to ModelCompilationOptions_SetEpContextEmbedMode.

Parameters

[in]	model_compile_options	The OrtModelCompilationOptions instance.
[in]	external_initializers_file_path	Null terminated string of the path to the file.
[in]	external_initializers_size_threshold	Initializers larger than this threshold are stored in the file.

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ModelCompilationOptions_SetOutputModelPath()

OrtStatus * OrtCompileApi::ModelCompilationOptions_SetOutputModelPath	(	OrtModelCompilationOptions *	model_compile_options,
		const char *	output_model_path
	)

Sets the file path for the output ONNX model generated by CompileModel.

The output model's location (e.g., file path or memory buffer) can be set with either ModelCompilationOptions_SetOutputModelPath or ModelCompilationOptions_SetOutputModelBuffer.

If the output model's location is not set, ONNX Runtime will generate an output file with a path based on the input model's file path. Examples: /Path/my_model.onnx -> /Path/my_model_ctx.onnx /Path/my_model -> /Path/my_model_ctx.onnx

Parameters

[in]	model_compile_options	The OrtModelCompilationOptions instance.
[in]	output_model_path	Null terminated string of the path (wchar on Windows, char otherwise).

Returns: If no error, nullptr will be returned. If there is an error, a pointer to an OrtStatus that contains error details will be returned. Use OrtApi::ReleaseStatus to free this pointer.

Since: Version 1.22.

◆ ReleaseModelCompilationOptions()

void OrtCompileApi::ReleaseModelCompilationOptions ( OrtModelCompilationOptions * input )

Public Member Functions

Detailed Description

Member Function Documentation

◆ CompileModel()

◆ CreateModelCompilationOptionsFromSessionOptions()

◆ ModelCompilationOptions_SetEpContextEmbedMode()

◆ ModelCompilationOptions_SetInputModelFromBuffer()

◆ ModelCompilationOptions_SetInputModelPath()

◆ ModelCompilationOptions_SetOutputModelBuffer()

◆ ModelCompilationOptions_SetOutputModelExternalInitializersFile()

◆ ModelCompilationOptions_SetOutputModelPath()

◆ ReleaseModelCompilationOptions()