Skip to content

[CI] pytorch-rocm-llms / run-model failed on halo (windows) #655

Description

@github-actions

This issue was opened automatically by the Test Playbooks workflow after the test run-model failed on the main branch.

Failure scope

Hardware / OS to use to reproduce

Run the failing test on a machine that matches the runner labels above (OS = windows, device = halo). The repo's self-hosted runners already advertise these labels; if you reproduce locally, use the same OS family and the same AMD device class.

How to dispatch the same test from CI

Re-run only the failing playbook on the same matrix entry by triggering the workflow with the playbook id:

gh workflow run test-playbooks.yml --repo amd/playbooks -f playbook_id=pytorch-rocm-llms

The workflow's matrix narrows down to this (device, platform) combination automatically based on the playbook's tested_platforms.

How to run just this test locally

python .github/scripts/run_playbook_tests.py --playbook pytorch-rocm-llms --platform windows --device halo

The runner extracts test blocks from playbooks/*/pytorch-rocm-llms/README.md (the failing block starts around line 394).

Failing test (verbatim from the README)

  • Setup: pytorch-env\Scripts\activate
  • Timeout: 600s
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "openai/gpt-oss-20b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

Result

  • Exit code: 1

stderr (last lines)

`torch_dtype` is deprecated! Use `dtype` instead!
MXFP4 quantization requires Triton and kernels installed: CUDA requires Triton >= 3.4.0, XPU requires Triton >= 3.5.0, we will default to dequantizing the model to bf16

Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "C:\actions-runner\_work\playbooks\playbooks\test-results\pytorch-rocm-llms\test_run-model.py", line 6, in <module>
    model = AutoModelForCausalLM.from_pretrained(
        model_name,
        torch_dtype=torch.bfloat16,
        device_map="auto"
    )
  File "C:\actions-runner\_work\playbooks\playbooks\playbooks\core\pytorch-rocm-llms\assets\pytorch-env\Lib\site-packages\transformers\models\auto\auto_factory.py", line 604, in from_pretrained
    return model_class.from_pretrained(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\actions-runner\_work\playbooks\playbooks\playbooks\core\pytorch-rocm-llms\assets\pytorch-env\Lib\site-packages\transformers\modeling_utils.py", line 277, in _wrapper
    return func(*args, **kwargs)
  File "C:\actions-runner\_work\playbooks\playbooks\playbooks\core\pytorch-rocm-llms\assets\pytorch-env\Lib\site-packages\transformers\modeling_utils.py", line 5048, in from_pretrained
    ) = cls._load_pretrained_model(
        ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        model,
        ^^^^^^
    ...<12 lines>...
        weights_only=weights_only,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "C:\actions-runner\_work\playbooks\playbooks\playbooks\core\pytorch-rocm-llms\assets\pytorch-env\Lib\site-packages\transformers\modeling_utils.py", line 5468, in _load_pretrained_model
    _error_msgs, disk_offload_index = load_shard_file(args)
                                      ~~~~~~~~~~~~~~~^^^^^^
  File "C:\actions-runner\_work\playbooks\playbooks\playbooks\core\pytorch-rocm-llms\assets\pytorch-env\Lib\site-packages\transformers\modeling_utils.py", line 831, in load_shard_file
    state_dict = load_state_dict(
        shard_file, is_quantized=is_quantized, map_location=map_location, weights_only=weights_only
    )
  File "C:\actions-runner\_work\playbooks\playbooks\playbooks\core\pytorch-rocm-llms\assets\pytorch-env\Lib\site-packages\transformers\modeling_utils.py", line 484, in load_state_dict
    with safe_open(checkpoint_file, framework="pt") as f:
         ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: Insufficient system resources exist to complete the requested service. (os error 1450)


This issue is opened and deduplicated by .github/scripts/create_failure_issues.py. Close it once the failure is fixed; subsequent failures with the same scope will reopen a fresh issue.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions