chore(visual-regression): add script to update ground truths #29204
Conversation
…ual-regression-script
…ual-regression-script
|
Also adding Liam and Brandy as reviewers since they've worked on the visual regression changes. |
There was a problem hiding this comment.
I am running into the following error when running this branch locally:
$ npm run test.e2e.script
> @ionic/[email protected] test.e2e.script
> sh src/utils/test/playwright/e2e-script.sh
Do you want to update your local ground truths? (y/n)
y
Enter the base branch name (default: main):
Updating the base branch...
From https://github.com/ionic-team/ionic-framework
* branch main -> FETCH_HEAD
Successfully rebased and updated refs/heads/visual-regression-script.
Building core...
....
Enter the component or path you want to test (e.g. chip, src/components/chip). Or leave empty to test all components:
chip
Updating local ground truths...
fatal: invalid reference: origin/
Running e2e tests...…ual-regression-script
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
…ual-regression-script
|
@brandyscarney ready for review |
|
|
||
| // Ask user if they want to open the Playwright report. | ||
| const shouldOpenReport = await confirm({ | ||
| message: 'Do you want to open the Playwright report?', |
There was a problem hiding this comment.
The command doesn't end when you say "Yes" here unless you do a cmd + C which prints an error:
| // User chose to open the Playwright report. | ||
| if (shouldOpenReport) { | ||
| await execAsync('npx playwright show-report').catch(() => { | ||
| console.error('Error: Failed to open the Playwright report or user closed the report'); |
There was a problem hiding this comment.
This error doesn't seem to print when I close the report
| else { | ||
| // Update the local ground truths for all components. | ||
| await execAsync(`git checkout origin/${baseBranch} -- src/components/*/test/*/*.e2e.ts-snapshots/*-linux.png`).catch((error) => { | ||
| updateGroundTruth.stop('Failed to update local ground truths'); |
There was a problem hiding this comment.
Trying to update the ground truths from a branch that doesn't have some screenshots causes errors:
I'm not sure if this is expected or something we want to catch.
| } | ||
|
|
||
| // User chose to open the Playwright report. | ||
| if (shouldOpenReport) { |
There was a problem hiding this comment.
Should we print the npx playwright show-report command if they say "No" in case they want to view it later?
Issue number: N/A
What is the current behavior?
Devs would have to manually generate the ground truths from their desired base branch. This causes a dev to checkout the base branch and pull the latest screenshots. They would then return to their working branch and start the E2E tests.
What is the new behavior?
A script has been created to automate this process using Docker as mentioned in the design doc:
Does this introduce a breaking change?
Other information
How to test:
npm run test.e2e.script