ONNX Runtime IoT Deployment on Raspberry Pi

Learn how to perform image classification on the edge using ONNX Runtime and a Raspberry Pi, taking input from the device’s camera and sending the classification results to the terminal.

If you have not configured a Raspberry Pi before, check out the Raspberry Pi documentation to get your device setup.

There are many benefits and use cases for deploying models to edge devices. Check out the list on our IoT tutorial home page.

Image of Raspberry Pi and camera.

Contents

Prerequisites

Download the source code, ML model and install the packages

Once you have imaged the Raspberry Pi and configured it for use its time to connect and download the source code to your device.

  • Connect to your Raspberry Pi device

In this tutorial we are using VNC Viewer to remote in. If you are going to use VNC Viewer be sure to follow these setup steps to establish a connection. Once VNC is enabled on Raspberry Pi and you have downloaded the VNC Viewer app on your computer, then you can remote into the device.

Image of VNC Viewer

  • Download the source to your Raspberry Pi. The source code includes everything you need to run inference including a mobilenet ONNX model from the model zoo and imagenet_classes.txt classes.

      git clone https://github.com/cassiebreviu/onnxruntime-raspberrypi.git

  • Navigate to the onnxruntime-raspberrypi download location and install the package from the requirements.txt with the following command.

      cd onnxruntime-raspberrypi
  pip install -r requirements.txt

    In this tutorial we are using the Raspberry Pi Camera Module. We want to test the camera with the cameratest.py script provided. If you have issues getting the camera to work run sudo apt update sudo apt upgrade to update the board and firmware.

  • Configure and test the camera by running the below command. This will create a image capture called test.jpg in the current directory and open a live video stream of the camera output. Hit ESC to cancel out of the live video output.

      python cameratest.py

  • The cameratest.py script is below for reference:

      import numpy as np
  import cv2

  # Create test image using opencv.
  cap = cv2.VideoCapture(0)
  cap.set(3,640) # set Width
  cap.set(4,480) # set Height

  ret, frame = cap.read()
  frame = cv2.flip(frame, -1) # Flip camera vertically
  cv2.imwrite('test.jpg', frame)
    
  # Start live video feed until `ESC` is pressed to quit.
  while(True):
      ret, frame = cap.read()
      frame = cv2.flip(frame, -1) # Flip camera vertically
      gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        
      cv2.imshow('frame', frame)
      cv2.imshow('gray', gray)
        
      k = cv2.waitKey(30) & 0xff
      if k == 27: # press 'ESC' to quit
          break

  cap.release()
  cv2.destroyAllWindows()

    Run inference on the Raspberry Pi with the inference_mobilenet.py script

Now that we have validated that the camera is connected and working on the Raspberry Pi, its time to inference the ONNX model provided in the source. The model is a MobileNet model that performs image classification on 1000 classes.

  • Run the inference script with the below command. 
      python inference_mobilenet.py
  • Terminal output: Image of VNC Viewer

  • The image that was inferenced on the Raspberry Pi: Image of VNC Viewer

  • The inference_mobilenet.py script below for reference:

      # Import the packages.
  from PIL import Image
  import numpy as np
  import onnxruntime
  import torch
  import cv2

  def preprocess_image(image_path, height, width, channels=3):
      image = Image.open(image_path)
      image = image.resize((width, height), Image.LANCZOS)
      image_data = np.asarray(image).astype(np.float32)
      image_data = image_data.transpose([2, 0, 1]) # transpose to CHW
      mean = np.array([0.079, 0.05, 0]) + 0.406
      std = np.array([0.005, 0, 0.001]) + 0.224
      for channel in range(image_data.shape[0]):
          image_data[channel, :, :] = (image_data[channel, :, :] / 255 - mean[channel]) / std[channel]
      image_data = np.expand_dims(image_data, 0)
      return image_data

  def softmax(x):
      """Compute softmax values for each sets of scores in x."""
      e_x = np.exp(x - np.max(x))
      return e_x / e_x.sum()

  def run_sample(session, image_file, categories):
      output = session.run([], {'input':preprocess_image(image_file, 224, 224)})[0]
      output = output.flatten()
      output = softmax(output) # this is optional
      top5_catid = np.argsort(-output)[:5]
      for catid in top5_catid:
          print(categories[catid], output[catid])
      # Write the result to a file.
      with open("result.txt", "w") as f:
          for catid in top5_catid:
              f.write(categories[catid] + " " + str(output[catid]) + " \r")

  # Create main function to run inference.
  if __name__ == "__main__":
      # Read the categories from the classes file.
      with open("imagenet_classes.txt", "r") as f:
          categories = [s.strip() for s in f.readlines()]
        
      # Create Inference Session
      session = onnxruntime.InferenceSession("mobilenet_v2_float.onnx")

      # Get image from the camera.
      cap = cv2.VideoCapture(0)
      cap.set(3,640) # set Width
      cap.set(4,480) # set Height

      ret, frame = cap.read()
      frame = cv2.flip(frame, -1) # Flip camera vertically
      cv2.imwrite('capture.jpg', frame)
      cap.release()
      cv2.destroyAllWindows()

      # Run inference
      run_sample(session, 'capture.jpg', categories)

Conclusion

Now that we have successfully run inference on the Raspberry Pi, we can use the same code to run inference on any device that supports ONNX Runtime. We can also use the same code to run inference on the Raspberry Pi with a different model. Check out the other models in the ONNX Model Zoo.

More examples