NPU with Core 7 ultra throwing an error

[phildrage]

I have a core 7 ultra, I've set the following -

I've changed line 12 in constants.py to DEVICE = environ.get("DEVICE", "npu")
model to LCM-openVINO
LCM OpenVINO model - rupeshs/hyper-sd-sdxl-1-step-openvino-int8

but when I try to create an image i see no NPU usage, and it throws the following errors. Can anyone help?

env) PS C:\Users\phils\fastsdcpu> .\start-webui.bat
Starting fastsdcpu...
Python command check :OK
Python version: 3.10.11
Press any key to continue . . .
FastSD CPU - v1.0.0 beta 120
Running on Windows platform
OS: Windows-10-10.0.26100-SP0
Processor: Intel64 Family 6 Model 170 Stepping 4, GenuineIntel
Using device : NPU
Found 8 LCM models in config/lcm-models.txt
Found 7 stable diffusion models in config/stable-diffusion-models.txt
Found 4 LCM-LoRA models in config/lcm-lora-models.txt
Found 9 OpenVINO LCM models in config/openvino-lcm-models.txt
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_11m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_384 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_512 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
Starting web UI mode
No lora models found, please add lora models to C:\Users\phils\fastsdcpu\lora_models directory
C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\components\dropdown.py:226: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True.
warnings.warn(
C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\components\base.py:194: UserWarning: show_label has no effect when container is False.
warnings.warn("show_label has no effect when container is False.")
INFO: Could not find files for the given pattern(s).

• Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Torch datatype : torch.float32
Reshape and compile
{'clip_skip': 1,
'controlnet': None,
'diffusion_task': 'text_to_image',
'dirs': {'controlnet': 'C:\Users\phils\fastsdcpu\controlnet_models',
'lora': 'C:\Users\phils\fastsdcpu\lora_models'},
'gguf_model': {'clip_path': None,
'diffusion_path': None,
'gguf_models': 'C:\Users\phils\fastsdcpu\models\gguf',
't5xxl_path': None,
'vae_path': None},
'guidance_scale': 1.0,
'image_height': 512,
'image_width': 512,
'inference_steps': 1,
'init_image': None,
'lcm_lora': {'base_model_id': 'Lykon/dreamshaper-8',
'lcm_lora_id': 'latent-consistency/lcm-lora-sdv1-5'},
'lcm_model_id': 'stabilityai/sd-turbo',
'lora': {'enabled': False,
'fuse': True,
'models_dir': 'C:\Users\phils\fastsdcpu\lora_models',
'path': '',
'weight': 0.5},
'negative_prompt': '',
'number_of_images': 1,
'openvino_lcm_model_id': 'rupeshs/hyper-sd-sdxl-1-step-openvino-int8',
'prompt': 'a cat\n',
'rebuild_pipeline': False,
'seed': 123123,
'strength': 0.6,
'token_merging': 0.0,
'use_gguf_model': False,
'use_lcm_lora': False,
'use_offline_model': False,
'use_openvino': True,
'use_safety_checker': False,
'use_seed': False,
'use_tiny_auto_encoder': False}
***** Init Text to image (OpenVINO) - rupeshs/hyper-sd-sdxl-1-step-openvino-int8 *****
Fetching 27 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 27/27 [00:00<?, ?it/s]
The config attributes {'interpolation_type': 'linear', 'skip_prk_steps': True, 'use_karras_sigmas': False} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
Compiling the vae_decoder to NPU ...
Traceback (most recent call last):
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\queueing.py", line 624, in process_events
response = await route_utils.call_process_api(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\route_utils.py", line 323, in call_process_api
output = await app.get_blocks().process_api(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\blocks.py", line 2015, in process_api
result = await self.call_function(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\blocks.py", line 1562, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\anyio\to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\anyio_backends_asyncio.py", line 2461, in run_sync_in_worker_thread
return await future
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\anyio_backends_asyncio.py", line 962, in run
result = context.run(func, *args)
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\utils.py", line 865, in wrapper
response = f(*args, **kwargs)
File "C:\Users\phils\fastsdcpu\src\frontend\webui\text_to_image_ui.py", line 54, in generate_text_to_image
images = future.result()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\concurrent\futures_base.py", line 458, in result
return self.__get_result()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\concurrent\futures_base.py", line 403, in __get_result
raise self._exception
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\phils\fastsdcpu\src\context.py", line 55, in generate_text_to_image
self.lcm_text_to_image.init(
File "C:\Users\phils\fastsdcpu\src\backend\lcm_text_to_image.py", line 243, in init
self.pipeline = get_ov_text_to_image_pipeline(
File "C:\Users\phils\fastsdcpu\src\backend\openvino\pipelines.py", line 40, in get_ov_text_to_image_pipeline
pipeline = OVStableDiffusionXLPipeline.from_pretrained(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\modeling_base.py", line 420, in from_pretrained
return from_pretrained_method(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 336, in _from_pretrained
return cls(
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 917, in init
super().init(*args, **kwargs)
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 165, in init
self.compile()
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 585, in compile
self.vae_decoder._compile()
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 721, in _compile
super()._compile()
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 634, in _compile
self.request = core.compile_model(self.model, self._device, self.ov_config)
File "C:\Users\phils\fastsdcpu\env\lib\site-packages\openvino\runtime\ie_api.py", line 543, in compile_model
super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:107:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:697:
Exception from src/plugins/intel_npu/src/plugin/src/compiled_model.cpp:62:
Exception from src/core/src/partial_shape.cpp:266:
to_shape was called on a dynamic shape.

[phildrage]

I've tried changing the model to rupeshs/sd15-lcm-square-openvino-int8 and now get the following output, with no image created

Starting fastsdcpu...
Python command check :OK
Python version: 3.10.11
Press any key to continue . . .
FastSD CPU - v1.0.0 beta 120
Running on Windows platform
OS: Windows-10-10.0.26100-SP0
Processor: Intel64 Family 6 Model 170 Stepping 4, GenuineIntel
Using device : NPU
Found 8 LCM models in config/lcm-models.txt
Found 7 stable diffusion models in config/stable-diffusion-models.txt
Found 4 LCM-LoRA models in config/lcm-lora-models.txt
Found 9 OpenVINO LCM models in config/openvino-lcm-models.txt
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_5m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_11m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_224 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_384 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
C:\Users\phils\fastsdcpu\env\lib\site-packages\controlnet_aux\segment_anything\modeling\tiny_vit_sam.py:654: UserWarning: Overwriting tiny_vit_21m_512 in registry with controlnet_aux.segment_anything.modeling.tiny_vit_sam.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
return register_model(fn_wrapper)
Starting web UI mode
No lora models found, please add lora models to C:\Users\phils\fastsdcpu\lora_models directory
C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\components\dropdown.py:226: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True.
warnings.warn(
C:\Users\phils\fastsdcpu\env\lib\site-packages\gradio\components\base.py:194: UserWarning: show_label has no effect when container is False.
warnings.warn("show_label has no effect when container is False.")
INFO: Could not find files for the given pattern(s).

• Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Torch datatype : torch.float32
Reshape and compile
{'clip_skip': 1,
'controlnet': None,
'diffusion_task': 'text_to_image',
'dirs': {'controlnet': 'C:\Users\phils\fastsdcpu\controlnet_models',
'lora': 'C:\Users\phils\fastsdcpu\lora_models'},
'gguf_model': {'clip_path': None,
'diffusion_path': None,
'gguf_models': 'C:\Users\phils\fastsdcpu\models\gguf',
't5xxl_path': None,
'vae_path': None},
'guidance_scale': 1.0,
'image_height': 512,
'image_width': 512,
'inference_steps': 5,
'init_image': None,
'lcm_lora': {'base_model_id': 'Lykon/dreamshaper-8',
'lcm_lora_id': 'latent-consistency/lcm-lora-sdv1-5'},
'lcm_model_id': 'stabilityai/sd-turbo',
'lora': {'enabled': False,
'fuse': True,
'models_dir': 'C:\Users\phils\fastsdcpu\lora_models',
'path': '',
'weight': 0.5},
'negative_prompt': '',
'number_of_images': 1,
'openvino_lcm_model_id': 'rupeshs/sd15-lcm-square-openvino-int8',
'prompt': 'a cat\n',
'rebuild_pipeline': False,
'seed': 123123,
'strength': 0.6,
'token_merging': 0.0,
'use_gguf_model': False,
'use_lcm_lora': False,
'use_offline_model': False,
'use_openvino': True,
'use_safety_checker': False,
'use_seed': False,
'use_tiny_auto_encoder': False}
***** Init Text to image (OpenVINO) - rupeshs/sd15-lcm-square-openvino-int8 *****
Loading Heterogeneous Compute pipeline
Fetching 17 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 17/17 [00:00<?, ?it/s]
Compiling the text_encoder to NPU ...
Compiling the unet to NPU ...
Compiling the vae_decoder to NPU ...
Compiling the vae_encoder to NPU ...
Press any key to continue . . .
PS C:\Users\phils\fastsdcpu>

[rupeshs]

@phildrage NPU does not suppport dynamic shape.
Currently FastSD support this model for NPU rupeshs/sd15-lcm-square-openvino-int8.
More details : https://github.com/rupeshs/fastsdcpu?tab=readme-ov-file#intel-ai-pc-support---openvino-cpu-gpu-npu