enable_opencl_throttling should be a boolean

[Mat2001mat]

I'm having troubles with GPU Inference and not sure what I'm missing.

I'm creating the inference like this.

var device = "GPU"
options.AppendExecutionProvider_OpenVINO(device);
inferenceSession = new InferenceSession(modelPath, options);

When I run inference, I get this:

[ERROR] [OpenVINO-EP] enable_opencl_throttling should be a boolean.

Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:RuntimeException] /opt/intel/onnxruntime/onnxruntime/core/providers/openvino/openvino_provider_factory.cc:42 bool onnxruntime::openvino_ep::ParseBooleanOption(const onnxruntime::ProviderOptions&, std::string) [ERROR] [OpenVINO-EP] enable_opencl_throttling should be a boolean.

I'm not sure how I can set this option. I've tried different options with options.AppendExecutionProvider();, but am having no luck. Has anyone seen this before?

[thoron]

You might be able to work around this issue by explicitly setting the config value to something before adding the EP:

  var device = "GPU";
+ options.AddSessionConfigEntry("enable_opencl_throttling", "true"); // Or false - https://onnxruntime.ai/docs/execution-providers/OpenVINO-ExecutionProvider.html#summary-of-options
  options.AppendExecutionProvider_OpenVINO(device);
  inferenceSession = new InferenceSession(modelPath, options);

[Mat2001mat]

Appreciate the help, but unfortunately it made no difference.

[thoron]

I did some digging and found that the AppendExecutionProvider docs does not reflect actual usage and OpenVINO can be specified: https://onnxruntime.ai/docs/api/csharp/api/Microsoft.ML.OnnxRuntime.SessionOptions.html#Microsoft_ML_OnnxRuntime_SessionOptions_AppendExecutionProvider_System_String_System_Collections_Generic_Dictionary_System_String_System_String__

            var device = "GPU";
            var sessionOptions = new SessionOptions();
            sessionOptions.AppendExecutionProvider("OpenVINO",
                new Dictionary<string, string>
                {
                    { "enable_opencl_throttling", "false" },
                    { "device_type",  device }
                });

Perhaps this will help you (and hopefully others)!

[Mat2001mat]

You absolute legend, thank you! That's solved the problem.