Build the generate() API from source

Pre-requisites

  • cmake
  • .NET6 (if building C#)

Clone the onnxruntime-genai repo 

git clone https://github.com/microsoft/onnxruntime-genai
cd onnxruntime-genai

Build the generate() API

This step assumes that you are in the root of the onnxruntime-genai repo.

All of the build commands below have a --config argument, which takes the following options:

  • Release builds release binaries
  • Debug build binaries with debug symbols
  • RelWithDebInfo builds release binaries with debug info

Build Python API

Windows CPU build 

python build.py --config Release

Windows DirectML build 

python build.py --use_dml --config Release

Linux build 

python build.py --config Release

Linux CUDA build 

python build.py --use_cuda --config Release

Mac build 

python build.py --config Release

Build Java API 

python build.py --build_java --config Release

Build for Android

If building on Windows, install ninja.

pip install ninja

Run the build script.

python build.py --build_java --android --android_home <path to your Android SDK> --android_ndk_path <path to your NDK installation> --android_abi  [armeabi-v7a|arm64-v8a|x86|x86_64] --config Release

Install the library into your application

Install Python wheel 

cd build/wheel
pip install *.whl

Install NuGet

Coming soon

Install JAR

Copy build/Windows/Release/src/java/build/libs/*.jar into your application.

Install AAR

Copy build/Android/Release/src/java/build/android/outputs/aar/onnxruntime-genai-release.aar into your application.

Install C/C++ header file and library

Windows

Use the header in src\ort_genai.h and the libraries in build\Windows\Release

Linux

Use the header in src/ort_genai.h and the libraries in build/Linux/Release