Possible webgpu bug in AutoModelForSequenceClassification

[brianpetro]

System Info

Windows
Electron/Chromium

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I'm using the example for re-ranking, almost verbatim, as seen #632 (comment)

Weirdly, when device=webgpu, I see the same output for every inputted document (screenshot below).

Not setting device seems to work, but CPU performance makes it unusable even with the smallest quantized models (can't have the computer freeze for 10+ seconds for every ranking).

It might be notable that I'm successfully using webgpu in a feature-extraction pipeline in the same environment.

I'm a big fan of the transformers.js project and I'm looking forward to getting this working. I see a lot of opportunity for implementing the ranking models for various features. Thanks for your work on this valuable open-source contribution! 🌴

Reproduction

Using device=webgpu

When device is unset

[brianpetro]

In case it helps, here is a link to the actual code https://github.com/brianpetro/jsbrains/blob/main/smart-rank-model/adapters/transformers.js

And also, a link to the embedding code that uses webgpu and works https://github.com/brianpetro/jsbrains/blob/main/smart-embed-model/adapters/transformers.js