Playwright wasm tests by k9 · Pull Request #9114 · gfx-rs/wgpu

k9 · 2026-02-25T18:51:01Z

Connections
#8709

Description
Try using playwright to run wasm tests in browser. Just runs in chrome/webgl for now.

The approach in this PR is fairly complex, but has the advantage of using the nextest CLI on the front end.

The tests are run using cargo xtask test-wasm. The xtask code:

copies some html/js from tests/wasm/web to tests/wasm/dist (basically a build step)
builds ".wasm" targets, runs wasm-bindgen then copies them into the tests/wasm/dist folder
starts a simple node express http server. This is used to serve static files for the tests, and also for communication between the task running on native and playwright.
the xtask asks the node server for a gpu report using an http request. The node server opens a browser using playwright, gets info about the adapter, and returns it. This report gets saved similar to a native gpu report.
kicks off nextest with TEST_WASM=true env var. The causes a version of the tests to run natively, where instead of running an actual test, a request is made to the node server to run the browser test in playwright. This is done using wasm_manager.rs
- wasm manager filters and names the tests using the saved gpu report.
- for a given test, wasm manager makes an http request to the node server. Node runs the test using playwright, and returns the result. Since each test runs in a new page load, this should isolate them from each other.

An alternative approach to the architecture above would be to just loop over the tests in the browser, skipping nextest, and sending logs back to the shell. This is what I started with in 33ea7ef. Downside is that it doesn't allow the same CLI args as nextest, and would require something like an iframe to recover from bad tests.

Testing
Changed ci.yml to call a new xtask task instead of wasm-pack

k9 · 2026-02-25T18:52:20Z

tests/src/wasm.rs

+use wasm_bindgen::prelude::wasm_bindgen;
+use wgpu::Backends;
+
+// @todo avoid copying this from wgpu-info


I copied this list since I'm not sure where it should live if it's also imported here.

On latest trunk this should now be solved! You can call wgpu::TextureFormat::exhaust and from the exhaust::Exhaust trait, which should get you the list of all formats.

k9 · 2026-02-25T18:53:47Z

tests/src/wasm.rs

+}
+
+#[wasm_bindgen]
+pub async fn gpu_report() -> String {


Attempt here is to create a one-adapter "gpu report" for a given browser. I wasn't sure this was valid, but if it is, the advantage is it gives accurate info on the CLI about which tests are skipped.

Yeah this is fine! Not for this PR, but it's a real shame we can't re-use the code between this and wgpu-info.

k9 · 2026-02-25T18:56:33Z

tests/wasm/web/index.js

+  let url = new URL(window.location.href);
+  let name = url.searchParams.get("name");
+  let wasm = url.searchParams.get("wasm");
+  const { default: init, run_test, gpu_report } = await import(`./${wasm}.js`);


This is a bit hacky to load a JS file based on URL param, I'd try to do something better in a real PR.

k9 · 2026-02-25T19:11:11Z

xtask/src/test_wasm.rs

+
+    let cargo_args = flatten_args(args, passthrough_args);
+
+    let bins = [


Best-case this list wouldn't be needed. But I couldn't think of a general way to loop over the workspace and build the right tests.

~~Also, now wondering why the example tests work at all, given that I mistakenly list wgpu-test twice...~~
OK fixed example tests. Now a couple tests are failing, but I think there are specific reasons for those.

Yeah the failing tests seem to be because they're checking for not(target_arch=wasm32), but since wasm_manager is actually enumerating the tests on the native system, that check doesn't work. So once a general approach is decided, I can update that check for the failing tests.

Alright, that sounds good.

k9 · 2026-02-25T19:12:34Z

xtask/src/test_wasm.rs

+            .cmd("cargo")
+            .args(build_args)
+            .args(["--message-format=json", "-q"])
+            .output()?;


Idea here is to figure out where the wasm binary actually is, since it has a random hash appended to it. I felt like there should be an obvious way to do this, but couldn't find one.

cargo nextest list -v will output the binary name, PWD, and list of tests attributed to that binary, which should prvent the need to double build (as it will do the build for you)

cwfitzgerald · 2026-02-27T03:49:04Z

Thanks for putting this up! I'm going to self-assign this so I don't lose it

cwfitzgerald

Sorry it took so long to look at this, I wanted to make sure I got the time to give this a proper look and to fully understand it.

I think this approach is great! There's a few gotchas that I've noted, and the failures that you've noted, but I think we should go forward with this approach, and we can refine the experience and infrastructure as we go.

Thank you for taking the time to look into this!

cwfitzgerald · 2026-03-27T19:10:44Z

Cargo.toml

-
-# Speed up image comparison even in debug builds
-[profile.dev.package."nv-flip-sys"]
-opt-level = 3


What's this about?

cwfitzgerald · 2026-03-27T19:51:08Z

tests/wasm/runner/package.json

+  "scripts": {
+    "test": "echo \"Error: no test specified\" && exit 1"
+  },


nit: leftover default init script

cwfitzgerald · 2026-03-27T20:04:52Z

xtask/src/test_wasm.rs

+            .cmd("cargo")
+            .args(build_args)
+            .args(["--message-format=json", "-q"])
+            .output()?;


cargo nextest list -v will output the binary name, PWD, and list of tests attributed to that binary, which should prvent the need to double build (as it will do the build for you)

cwfitzgerald · 2026-03-27T20:05:22Z

xtask/src/test_wasm.rs

+
+    let cargo_args = flatten_args(args, passthrough_args);
+
+    let bins = [


Alright, that sounds good.

cwfitzgerald · 2026-03-27T20:06:29Z

tests/Cargo.toml

 cargo_metadata.workspace = true
 env_logger.workspace = true
 parking_lot = { workspace = true, features = ["deadlock_detection"] }
+ureq.workspace = true


nit: as we don't need tls, it is possible that we turn it off to prevent taking in deps on the TLS stack?

cwfitzgerald · 2026-03-27T20:08:00Z

tests/src/wasm.rs

+use wasm_bindgen::prelude::wasm_bindgen;
+use wgpu::Backends;
+
+// @todo avoid copying this from wgpu-info


On latest trunk this should now be solved! You can call wgpu::TextureFormat::exhaust and from the exhaust::Exhaust trait, which should get you the list of all formats.

cwfitzgerald · 2026-03-27T20:08:30Z

tests/src/lib.rs

        #[cfg(not(target_arch = "wasm32"))]
        fn main() -> $crate::native::MainResult {
-            $crate::native::main($tests)
+            if std::env::var("TEST_WASM") == Ok("true".to_string()) {


Suggested change

if std::env::var("TEST_WASM") == Ok("true".to_string()) {

if std::env::var("TEST_WASM").as_deref() == Ok("true") {

super nit

cwfitzgerald · 2026-03-27T20:10:31Z

tests/src/wasm.rs

+}
+
+#[wasm_bindgen]
+pub async fn gpu_report() -> String {


Yeah this is fine! Not for this PR, but it's a real shame we can't re-use the code between this and wgpu-info.

cwfitzgerald · 2026-03-27T20:14:34Z

xtask/src/test_wasm.rs

+    } else {
+        shell
+            .cmd("cargo")
+            .args(["nextest", "run", "-P", "wasm"])


Suggested change

.args(["nextest", "run", "-P", "wasm"])

.args(["nextest", "run", "--profile", "wasm"])

Nit, lets use the long name for flags, as it's clearer what they mean

k9 added 4 commits February 25, 2026 10:38

Try running tests in playwright - basic version

33ea7ef

Try using express server to coordinate tests

5585286

Work on cli args and error handling

c34ee01

Fix CLI doc url

0370617

k9 commented Feb 25, 2026

View reviewed changes

k9 added 3 commits February 25, 2026 14:55

Fail test if not found in browser, and fix wasm examples build

f22c40a

Try running multiple tests at once

4591cb2

Fix clippy default issue

fac431a

cwfitzgerald self-assigned this Feb 27, 2026

cwfitzgerald requested changes Mar 27, 2026

View reviewed changes


		let cargo_args = flatten_args(args, passthrough_args);

		let bins = [

	if std::env::var("TEST_WASM") == Ok("true".to_string()) {
	if std::env::var("TEST_WASM").as_deref() == Ok("true") {

	.args(["nextest", "run", "-P", "wasm"])
	.args(["nextest", "run", "--profile", "wasm"])

Conversation

k9 commented Feb 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k9 Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cwfitzgerald commented Feb 27, 2026

Uh oh!

cwfitzgerald left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

k9 Feb 25, 2026 •

edited

Loading