Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

- **Exception Message:** cuDNN Frontend error: CUDNN_BACKEND_OPERATION: cudnnFinalize Failed cudnn_status: CUDNN_STATUS_BAD_PARAM #145

Open
AS1ING opened this issue Oct 28, 2024 · 9 comments

Comments

@AS1ING
Copy link

AS1ING commented Oct 28, 2024

XLabs-AI/flux-ip-adapter-v2

ComfyUI Error Report

Error Details

  • Node Type: XlabsSampler
  • Exception Type: RuntimeError
  • Exception Message: cuDNN Frontend error: CUDNN_BACKEND_OPERATION: cudnnFinalize Failed cudnn_status: CUDNN_STATUS_BAD_PARAM

Stack Trace

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\execution.py", line 323, in execute
    output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\execution.py", line 198, in get_output_data
    return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\execution.py", line 169, in _map_node_over_list
    process_inputs(input_dict, i)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\execution.py", line 158, in process_inputs
    results.append(getattr(obj, func)(**inputs))

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\nodes.py", line 411, in sampling
    x = denoise(

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\sampling.py", line 193, in denoise
    pred = model_forward(

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\sampling.py", line 51, in model_forward
    img, txt = block(img=img, txt=txt, vec=vec, pe=pe)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\venv\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\venv\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\xflux\src\flux\modules\layers.py", line 297, in forward
    return self.processor(self, img, txt, vec, pe)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\venv\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\venv\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\layers.py", line 388, in forward
    img = self.shift_ip(img_q, attn, img)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\layers.py", line 313, in shift_ip
    x += torch.mean(block(img_qkv, attn), dim=0, keepdim=True)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\venv\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\venv\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)

  File "F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\custom_nodes\x-flux-comfyui\layers.py", line 265, in forward
    ip_attention = F.scaled_dot_product_attention(

System Information

  • ComfyUI Version: v0.2.4-12-g669d9e4
  • Arguments: F:\StabilityMatrix-win-x64\Data\Packages\ComfyUI Test\main.py --preview-method auto --auto-launch --fast --reserve-vram 0.8
  • OS: nt
  • Python Version: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
  • Embedded Python: false
  • PyTorch Version: 2.5.0+cu124

Devices

  • Name: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
    • Type: cuda
    • VRAM Total: 25756696576
    • VRAM Free: 2406419424
    • Torch VRAM Total: 20434649088
    • Torch VRAM Free: 34708448
@caslix
Copy link

caslix commented Oct 30, 2024

Hello! I have the same problem.

@rgfx
Copy link

rgfx commented Oct 30, 2024

This error occurs on the new workspace that was given. If I used the old one, and switch the modal to the new one it works.

@christming
Copy link

This error occurs on the new workspace that was given. If I used the old one, and switch the modal to the new one it works.

Hello~ how do you fix this bug? Can you give some more details here? I meet the same error with use the new XLabs-AI/flux-ip-adapter-v2

@xlibfly
Copy link

xlibfly commented Nov 1, 2024

I have the same problem.

@lior007
Copy link

lior007 commented Nov 2, 2024

same

@Orenji-Tangerine
Copy link

I have the same problem.

@xieyao2
Copy link

xieyao2 commented Nov 5, 2024

same problem.rgfx how did u fix it?

@xieyao2
Copy link

xieyao2 commented Nov 5, 2024

XLabs-AI/flux-ip-adapter-v2
14700kf
4060ti 16gb
ram 64gb
torch2.5.0 + cuda12.4+xformers0.0.28.post2

@xieyao2
Copy link

xieyao2 commented Nov 6, 2024

when i use one image,the workflow is ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants