Skip to content

Commit e11bcf9

Browse files
committed
fix(qwen3_moe): pass kwargs to self_attn
This is needed to avoid `.item()` calls in `_flash_attention_forward`.
1 parent 10627c1 commit e11bcf9

File tree

2 files changed

+2
-0
lines changed

2 files changed

+2
-0
lines changed

src/transformers/models/qwen3_moe/modeling_qwen3_moe.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -355,6 +355,7 @@ def forward(
355355
use_cache=use_cache,
356356
cache_position=cache_position,
357357
position_embeddings=position_embeddings,
358+
**kwargs,
358359
)
359360
hidden_states = residual + hidden_states
360361

src/transformers/models/qwen3_moe/modular_qwen3_moe.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,7 @@ def forward(
189189
use_cache=use_cache,
190190
cache_position=cache_position,
191191
position_embeddings=position_embeddings,
192+
**kwargs,
192193
)
193194
hidden_states = residual + hidden_states
194195

0 commit comments

Comments
 (0)