Install ONNX Runtime GenAI

Python package release candidates

pip install numpy
pip install onnxruntime-genai --pre --index-url=

Append -cuda for the library that is optimized for CUDA environments

pip install onnxruntime-genai-cuda --pre --index-url=

Nuget package release candidates

To install the NuGet release candidates, add a new package source in Visual Studio, go to Project -> Manage NuGet Packages.

  1. Click on the Settings cog icon

  2. Click the + button to add a new package source

    • Change the Name to onnxruntime-genai
    • Change the Source to
  3. Check the Include prerelease button

  4. Add the Microsoft.ML.OnnxRuntimeGenAI package

  5. Add the Microsoft.ML.OnnxRuntime package

To run with CUDA, use the following packages instead:

  • Microsoft.ML.OnnxRuntimeGenAI.Cuda
  • Microsoft.ML.OnnxRuntime.Gpu