Package ai.onnxruntime

Class OrtEnvironment.ThreadingOptions

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    Enclosing class:
    OrtEnvironment
    public static final class OrtEnvironment.ThreadingOptions
extends java.lang.Object
implements java.lang.AutoCloseable
    Controls the global thread pools in the environment. Only used if the session is constructed using an options with OrtSession.SessionOptions.disablePerSessionThreads() set.

    • Constructor Summary

      Constructors 
      Constructor Description
      ThreadingOptions()
      Create an empty threading options.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Closes the threading options.
      void setGlobalDenormalAsZero()
      When this is set it causes intra-op and inter-op thread pools to flush denormal values to zero.
      void setGlobalInterOpNumThreads​(int numThreads)
      Sets the number of threads available for inter-op parallelism (i.e.
      void setGlobalIntraOpNumThreads​(int numThreads)
      Sets the number of threads available for intra-op parallelism (i.e.
      void setGlobalSpinControl​(boolean allowSpinning)
      Allows spinning of thread pools when their queues are empty.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • ThreadingOptions

        public ThreadingOptions()
        Create an empty threading options.

    • Method Detail

      • close

        public void close()
        Closes the threading options.
        Specified by:
        close in interface java.lang.AutoCloseable

      • setGlobalInterOpNumThreads

        public void setGlobalInterOpNumThreads​(int numThreads)
                                throws OrtException
        Sets the number of threads available for inter-op parallelism (i.e. running multiple ops in parallel).

        Setting it to 0 will allow ORT to choose the number of threads, setting it to 1 will cause the main thread to be used (i.e., no thread pools will be used).

        Parameters:
        numThreads - The number of threads.
        Throws:
        OrtException - If there was an error in native code.

      • setGlobalIntraOpNumThreads

        public void setGlobalIntraOpNumThreads​(int numThreads)
                                throws OrtException
        Sets the number of threads available for intra-op parallelism (i.e. within a single op).

        Setting it to 0 will allow ORT to choose the number of threads, setting it to 1 will cause the main thread to be used (i.e., no thread pools will be used).

        Parameters:
        numThreads - The number of threads.
        Throws:
        OrtException - If there was an error in native code.

      • setGlobalSpinControl

        public void setGlobalSpinControl​(boolean allowSpinning)
                          throws OrtException
        Allows spinning of thread pools when their queues are empty. This call sets the value for both inter-op and intra-op thread pools.

        If the CPU usage is very high then do not enable this.

        Parameters:
        allowSpinning - If true allow the thread pools to spin.
        Throws:
        OrtException - If there was an error in native code.

      • setGlobalDenormalAsZero

        public void setGlobalDenormalAsZero()
                             throws OrtException
        When this is set it causes intra-op and inter-op thread pools to flush denormal values to zero.
        Throws:
        OrtException - If there was an error in native code.