core: convert to ES modules

[connorjclark]

ref #12689

This PR converts all of lighthouse-core/ to ES modules.

build/

• __dirname and __filename are gone, replaced with import.meta.url. The replace rollup bundle now drops the former and replaces the latter.
• A couple places in core use module.createRequire to get a require to use require.resolve, for getting a path to something inside a package. These resources end up being inlined by the inline-fs plugin, and require.resolve disappears. This is fortunate, because createRequire can never work in bundled environments. With the help of terser the call to createRequire is dropped entirely. Just in case it gets called somehow, createRequire is shimmed with a specific error message. ex: lighthouse-core/lib/axe.js
• build/plugins/inline-fs.js supports __dirname... there is no ESM equivalent, so I went with moduleDir which is the variable named used in runner.js.
• The bundledModules special variable in config-helpers.js is now a map of promises, and each "dynamic module" is imported there.

lighthouse-core/config

• requireWrapper - as before, this function first checks bundledModules for an exact match. It now falls back to import with the provided path (or with .js appended if no extension is present). This change is because the old behavior of require trying various extensions itself is not what ES import does. It looks only for what is given (sans any Node-runtime level aliases like pkg.exports...). Once a module is found, if it has a default export that is what's returned. To support named exports in the future, the last thing this function does now is look for an export that has a meta property, and returns that one.
• Some refactoring of mock usage happened in these tests: lighthouse-core/test/runner-test.js, lighthouse-core/test/gather/gather-runner-test.js. Also a couple tests were .skiped for further investigation (and marked with a TODO(esmodules) comment)

UIStrings

• Pretty much all the changes in audits/ and gatherers are due to __filename being replaced with import.meta.url. To make this a simple change, createIcuMessageFn replaces the file:// prefix found on import.meta.url.

Misc

• td.replace replaced with td.replaceEsm
• Minimal API surface lighthouse-core/index.cjs added so CJS users can continue to require lighthouse with minimal effort. What cannot be supported any longer are importing Audit/Gatherer subclasses for custom audits/gatherers.
• A preference for named exports was applied, and where the imported variable name was a mismatch, such variables were renamed to match the original declaration. Some exports remain default exports, such as computed artifacts, audits, and gatherers, mostly in an attempt to keep the PR smaller but also in anticipation of potential disagreement on that direction.
• Disabled pubads temporarily because it nows needs to be ES modules in order to import our Audit base class.

stuff for later

• Re-enable pubads
• Remove lighthouse-core/util-commonjs.cjs
• import/order lint pass
• Convert computed artifacts to named exports
• Convert audits to named exports
• Convert gatherers to named exports

[connorjclark]

Once I put a breakpoint down in lh-error.js module it became obvious that testdouble is somehow causing the module cache to get cleared between evaluation of a test file's static imports–and subsequent dynamic imports.

This is a problem in files (like runner-test.js) which statically import a class later relied on to have a singleton value (LighthouseError, symbols) and assert (usually indirectly) that they are the same expected instance in a test. Ex:

• runner-test.js static imports LighthouseError
• (for some reason, a dynamic import instantly after the static import blocks will not be equal to the static import)
• a before in runner-test.js dynamic imports runner.js, which itself static imports LighthouseError
• runner.js's static import of LighthouseError suffers from the same error an immedite dynamic import did (makes sense, it's the same cache) - it is new
• so then the test in runner-test.js that throws a LighthouseError fails an instanceof check in runner.js
• If that is hard to follow ... just set a breakpoint in lh-error.js module scope and it becomes clearer.

minimal repro:

import assert from 'assert';

import * as td from 'testdouble';

import {LighthouseError} from '../lib/lh-error.js';

td.replaceEsm('../gather/gatherers/stacks.js', undefined);

assert( (await import('../lib/lh-error.js')).LighthouseError === LighthouseError );

One obvious fix is to also dynamic import LighthouseError in runner-test.js before. But I'm gonna see if there's a better fix available.

[connorjclark]

better repro:

import assert from 'assert';

import 'quibble';

import {LighthouseError} from '../lib/lh-error.js';

assert( (await import('../lib/lh-error.js')).LighthouseError === LighthouseError );

import 'quibble'; has a sideeffect 👀 but only if testdouble loader is being used.

testdouble/testdouble.js#492

[connorjclark]

Worked all day to produce this code, which fixes the unexpected "re-import" of non-mocked modules (the cause of multiple LighthouseError classes existing).

node_modules/quibble/lib/quibble.mjs

Just placing it here in case I have total amnesia tomorrow about what the fix was. Will do a proper confirmation by adding to quibble tomorrow, confirming tests don't break, and seeing if it really fixes the "minimal repro" I shared above.

---

Related, but evidently not needed to fix the mock issues we've seen, is that I experimented with wrapping test files that use ES module mocks with a separate test file, to avoid the issues with mocking deps of modules imports by the test file. (That's why we have the ugly dynamic import hacks). The result is a much cleaner separation of mocks and the tests:

https://github.com/GoogleChrome/lighthouse/blob/mock-wrapper-test-file/lighthouse-core/test/runner-test.js
https://github.com/GoogleChrome/lighthouse/blob/mock-wrapper-test-file/lighthouse-core/test/runner-test.impl.js

After this PR lands, I'll open a PR converting just runner-test.js to this new test format to facilitate discussion.

[adamraine]

Overall LGTM

[adamraine]

Was this ever figured out?

[connorjclark]

This comment isn't new btw