Provide built-in implementations of module linker, initializeImportMeta, importModuleDynamically

[justinfagnani]

Is your feature request related to a problem? Please describe.

When using vm.SourceTextModule we have to implement a linker, initializeImportMeta, and importModuleDynamically, even if we just want the default behavior of built-in module support. There are many subtle ways to get that wrong.

Describe the solution you'd like

Default implementations that do basically what built-in module support does:

• A linker function that:
  • Reads files from disk, within a specified root.
  • Uses Node module resolution as implemented for modules
  • Uses a specified module cache
  • initializeImportMeta that sets url property
  • By configuration, allows access to built-in modules.

Describe alternatives you've considered

Userland libraries can take the first cuts at easy-to-use vm.Modules, but I think something for common use cases should likely be included.

[Jamesernator]

This is slightly more complicated as both --experimental-specifier-resolution and loaders can arbitrarily change the resolution of specifiers.

Most likely you want to use the same resolution as you are loaded as, for example suppose you're a tester that loads tests files and you're loaded under a loader that allows for running typescript e.g. node --loader=./with-typescript-loader.js ./runTestSuite.ts you'd want to load any tests using the same resolution scheme.

However some tools will still probably need access to the default resolution especially if they're dealing with things in node_modules/ (e.g. rollup).

I'm not sure what this might look like, maybe something that takes a loader and produces a linker, initialImportMeta and cache e.g.:

import { createESMLoaderHooks } from 'module';

const loaderHooks = createESMLoaderHooks({ loader: 'current' });
const nodeLoaderHooks = createESMLoaderHooks({ loader: 'default' });
const nodeWithExtensionResolutionHooks = createESMLoaderHooks({ loader: 'node' });
const customLoaderHooks = createESMLoaderHooks({ loader: new URL('./my-custom-loader.js', import.meta.url) });

[devsnek]

I'm kind of curious, would workers/processes be better suited to this use case?

If we did want to expose this machinery in the way you ask, I think it would need to wait a while, until we finish working out all the specifics of our module system.

[AlbertMarashi]

How does one even resolve a built-in-module currently? Is it possible to read source code of a built-in node module

[marcj]

That would be great. Jest currently does all of this manually which seems unbelievable error prone. I'm currently also switching to ESM and trying to rewrite my benchmark runner from new vm.Script to modules using ts-node/esm, which is quite a work since I have to deal now with all the loader/linker stuff manually (that was not necessary with vm.Script), where I have almost no clue how to implement correctly.

[frank-dspeed]

only my 5cent for test cases where you need the vm context and you simply need input output you can simply spawn "node" and use stdout and stdin for the IPC.

this allows all kind of easy string manipulation to inject stuff this is at last how i did run my esm tests the last years.

[axkibe]

As I understand providing a variant of the dynamic import that allows to change the "url" (module_wrap in module implemenation) the import is supposed to be coming from would solve this.

Sadly simply setting it doesn't work:

import.meta.url = 'file:///path/some/where/src/x.js';
await import( 'apackagefromthere' );

modulewrap in node.js overrides the file url the import is coming from.

[axkibe]

Found it, this way you can load modules from another path.

import { createRequire } from 'node:module';
const require = createRequire('file:///a/path/somewhere/script.mjs');
await import(require.resolve("apackage"));

Wasn't obvious in any way tough.

[bnoordhuis]

This issue has been open for over three years with no movement so I'm inclined to close it. I'm not even sure if the ask is still needed at this point. I'll close it in a few days unless someone chimes in.

I will say that OP's requested features are way too concrete/specific/inflexible to make a good fit for the standard library.

[axkibe]

Agree, at least until the last point "access to builtin function" IMO the 3 lines I posted do exactly this. I'm stil for making a more intuative API somewhere tough, rather than creating a require just to use it to resolve the import of a different file path. Like module.setMeta() or something that instead of createRequire(). According to the docs import.meta should be writable, so it ideally should be possible to change the file url there too, but I understand this may possible be harder to accomplish the way node.js builds the hooks (and thus overrides the current meta again)

[MadLittleMods]

Related to #43899