Skip to content

Migrate from karma to vitest to run unit tests #6988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Conversation

acelaya
Copy link
Contributor

@acelaya acelaya commented Apr 17, 2025

Depends on hypothesis/frontend-build#722
Depends on #6989
Depends on hypothesis/frontend-testing#76

This PR is the equivalent to hypothesis/frontend-shared#1947, but for client repository.

TODO

  • Fix remaining tests
  • Investigate some suspicious output printed after starting tests. Frontend-shared has a similar output in one test. Let's investigate this separately
    stderr | build/scripts/tests.bundle.js > ImageTextLayer > debounces text layer size updates
    null
    stdout | build/scripts/tests.bundle.js > NavigationObserver > only fires an event if the path or query params change
    Potential memory leak detected; be sure to call restore() to clean up your sandbox. To suppress this warning, modify the leakThreshold property of your sandbox.
    Sourcemap for "/home/alejandro/Development/hypothesis/client/build/styles/annotator.css" points to missing source files
    
  • Do not dump a summary of every successful test in the output (Add SummaryReporter for vitest frontend-testing#76)
  • Investigate reduction in code coverage

@acelaya acelaya force-pushed the vitest-poc branch 2 times, most recently from 8d782a4 to 41637dd Compare April 18, 2025 10:43
@acelaya acelaya changed the base branch from main to test-improvements April 18, 2025 10:48
headless: true,
screenshotFailures: false,
instances: [{ browser: 'chromium' }],
viewport: { width: 1024, height: 768 },
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bunch of tests that rely on the screen resolution, like the ones around side-by-side mode.

Karma sets bigger screen resolutions, but vitest uses a mobile screen resolution by default. I reckon we probably want to have more control over this and makes sense to set it here.

@acelaya acelaya force-pushed the vitest-poc branch 2 times, most recently from dfeefbb to 97d5a42 Compare April 18, 2025 10:52
yarn.lock Outdated
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many of the changes here are caused by running yarn dedupe, which was required to fix an incompatibility between different istanbul versions.

@acelaya acelaya force-pushed the test-improvements branch from 80c9e79 to 1a2dae8 Compare April 21, 2025 08:03
@acelaya acelaya force-pushed the test-improvements branch from 1a2dae8 to 8ee6ac9 Compare April 21, 2025 13:00
@acelaya acelaya force-pushed the test-improvements branch from 8ee6ac9 to b7a0747 Compare April 22, 2025 07:45
Base automatically changed from test-improvements to main April 22, 2025 07:47
@acelaya acelaya force-pushed the vitest-poc branch 4 times, most recently from b8b174d to 3813020 Compare April 23, 2025 12:24
@acelaya acelaya changed the title Add support to run tests with vitest Migrate from karma to vitest to run unit tests Apr 23, 2025
@acelaya acelaya force-pushed the vitest-poc branch 2 times, most recently from d701d1f to 643249c Compare April 23, 2025 12:27
Copy link

codecov bot commented Apr 23, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.39%. Comparing base (f9592db) to head (7aaa261).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6988      +/-   ##
==========================================
- Coverage   99.50%   99.39%   -0.12%     
==========================================
  Files         278      276       -2     
  Lines       11054    11010      -44     
  Branches     2662     2653       -9     
==========================================
- Hits        10999    10943      -56     
- Misses         55       67      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@acelaya acelaya force-pushed the vitest-poc branch 2 times, most recently from 9bae1af to 5b52a31 Compare April 25, 2025 08:30
@acelaya
Copy link
Contributor Author

acelaya commented Apr 25, 2025

There's a 1% decrease in code coverage, I'm not completely sure why 🤔

@acelaya acelaya marked this pull request as ready for review April 25, 2025 08:32
@acelaya acelaya requested a review from robertknight April 25, 2025 08:32
@acelaya
Copy link
Contributor Author

acelaya commented Apr 25, 2025

I'm investigating some eslint errors. I need to understand why this is not being reported in other projects 🤔

@acelaya
Copy link
Contributor Author

acelaya commented Apr 25, 2025

I'm investigating some eslint errors. I need to understand why this is not being reported in other projects 🤔

No other project is complaining about this because this is the only one using beforeAll and afterAll.

@robertknight
Copy link
Member

There's a 1% decrease in code coverage, I'm not completely sure why 🤔

It looks to me like the main reason is that some source files (index.{ts, tsx}) were not included before, but now they are. Did you find other reasons?

@acelaya
Copy link
Contributor Author

acelaya commented Apr 25, 2025

There's a 1% decrease in code coverage, I'm not completely sure why 🤔

It looks to me like the main reason is that some source files (index.{ts, tsx}) were not included before, but now they are. Did you find other reasons?

Yeah, that sounds reasonable, but I can't find where were we excluding those files before 🤔

'**/node_modules/**',
'**/test/**/*.js',
'**/test-util/**',
'src/**/index.{ts,tsx}',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excluding module index files from coverage, because they were not being considered for code coverage when using karma.

@acelaya
Copy link
Contributor Author

acelaya commented Apr 28, 2025

After excluding index files from code coverage, it is still dropping 0.12%.

I'm not sure if it's worth going module by module checking what's going on. I would probably merge this as is, and we can focus on improving the code coverage separately.

@acelaya
Copy link
Contributor Author

acelaya commented Apr 28, 2025

After excluding index files from code coverage, it is still dropping 0.12%.

I'm not sure if it's worth going module by module checking what's going on. I would probably merge this as is, and we can focus on improving the code coverage separately.

I just realized Codecov has an "Indirect changes" tab that indicates what coverage changes have happened in files that were not changed in the PR: https://app.codecov.io/gh/hypothesis/client/pull/6988/indirect-changes

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants