-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Epic: Testing improvements #22503
Labels
Comments
Because this issue has not had activity for over 150 days, we're automatically closing it for house-keeping purposes. Still require assistance? Please, create a new issue with up-to date details. |
msft-fluent-ui-bot
added
the
Resolution: Soft Close
Soft closing inactive issues over a certain period
label
Sep 11, 2022
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Labels
Overview
We have various layers of testing in Fluent UI, but a lot of the tools are out of date, and there are ways we could make our tests easier to write, faster to run, and generally more effective. These issues reduce development velocity and quality/quantity of tests.
This project's goal is to "clean house," making changes that will facilitate writing great tests to help take our library's quality to the next level as we continue to build out more converged components.
General areas:
Getting up to date: Upgrade Jest, Cypress, and other testing-related deps to the latest versions (and help unblock upgrades of other deps such as React)
Screener: Make our visual tests faster and more reliable, update outdated Storybook usage, and investigate whether a different visual testing framework would serve us better in the long run
Migrate away from Enzyme: Enzyme is effectively dead, was delaying our repo's internal React 17 upgrade, and will entirely block an in-repo React 18 upgrade. We need to work on switching to Testing Library and Cypress, which provide better-supported, less buggy, and more realistic testing.
Improve conformance tests: Conformance tests enable us to test both current and future components, and maintain API/behavior coherency as we build more components. The
@fluentui/react-conformance
package had ambitious goals, but its current structure is holding us back, so it needs to be rewritten (and then we need to add more tests).Documentation: We need up-to-date documentation to help people write great tests.
Address "paper cuts": Eliminate or improve small things that are detracting from the developer experience (with respect to testing)
Future ideas: Some additional types of tests we'd like to add at some point in the future
Getting up to date
Our testing-related dependencies had gotten pretty badly out of date, and new versions continue to be released.
✅ Done
Medium/long-term
@testing-library/user-event
to 14 (bug fixes and API changes) - Upgrade to user-event v14 #22260Screener (vr-tests)
✅ Done
For various reasons (outlined in #21070), our screener runs were becoming less reliable as we added more v9 tests. Also, having v8 and v9 in the same package was slowing down builds (due to needing to build both + run tests for both when either changed).
Medium/long-term
The storybooks in our vr-tests packages are using an outdated story format, which will need to be fixed before we can migrate to storybook 7 (once it comes out). #21779
We also haven't been impressed with various aspects of screener, so longer-term we'd like to investigate other solutions (possibly chromatic if we can find an interaction testing solution). #22369
Migrate away from Enzyme
Enzyme is no longer in active development (no official React 17 adapter after 1.5 years) and won't work with React 18 without a substantial rewrite of the adapter code/helpers. It's also just kind of difficult to work with and buggy.
A better solution is to use a combination of React Testing Library and Cypress.
Prior to this work, all the tests for v8 and v0 used Enzyme, and so did the react-conformance and a11y-testing packages shared between versions. v9 did not use Enzyme aside from conformance/a11y.
✅ Done
Goal: do enough to unblock migration to React 17 (#20145). This involves a hybrid approach using the "community" adapter
@wojtekmaj/enzyme-adapter-react-17
and porting any tests that don't work to either testing-library or cypress.We also communicated to the Redmond team that all new v8 tests should be written in testing-library (and strongly encouraged that if you have to fix an enzyme test, go ahead and port it). Ideally this would be enforced by tooling, but this is more challenging since a lint rule can't easily be applied to new code only.
Medium/long-term
Completely get rid of Enzyme by porting all tests (v8 and v0) to either testing-library or cypress. As mentioned above, this will be necessary before we can adopt React 18 in-repo. #21663
An alternative might be finding a way to run only the Enzyme tests against React 17, but that would be challenging to set up.
Conformance tests
Component conformance tests are important because they enable us to test both current and future components, and help us ensure that we have a coherent API surface and behaviors as we continue to build more components.
Short-term
@fluentui/react-conformance
was written with the goal of being reusable and extensible, but it's not a very good design. It also has the issue of including tests in the core package that are actually specific to v8/v9/v0 (which can cause dependency graph issues and is generally unhelpful), and it introduced enzyme+node types into the global namespace anywhere it's imported.Medium/long-term
Documentation
Our testing documentation was very out of date and didn't cover current recommended tools/patterns.
✅ The main testing page has been revamped and new pages added with specific info about Jest, Cypress, and Screener.
For the future, it would be good to continue expanding those docs with more information about best practices, "gotchas," etc.
Address "paper cuts"
There are various things creating small slowdowns or inconveniences which detract from the developer experience.
When we encounter one of these things, we should consider "is this thing still providing value? (if not, how hard is it to remove?) or "is there an easy way to make this a little better?"
✅ Short-term
Deferred/ideas
resetIds()
in a globalafterEach
Future ideas
Longer-term, what are some ways we can take our tests to the next level?
The text was updated successfully, but these errors were encountered: