Use of ESM *within* the Node.js Core Lib

[jasnell]

A simple question with a not so simple answer: Right now all of Node.js core lib is implemented as CommonJS. It's not loaded using the same mechanisms however, we pre-compile most (if not all of it), and statically link it into the Node.js binary... so here is the key question: Is there a reasonable path forward to migrating Node.js' internal lib to ESM, either incrementally or all at once? What would be the challenges for doing so?

[bmeck]

Does snapshotting even work with ESM in V8?

[jasnell]

And that's exactly one of the questions that would need to be asked ;-) .... @nodejs/v8 @joyeecheung ... any input?

[devsnek]

It does, but I would wait until 9.0 is landed to make use of it. Prior to that the support is spotty due to missing test coverage.

[camillobruni]

There is – rather basic – coverage for this in V8: https://source.chromium.org/chromium/chromium/src/+/master:v8/test/cctest/test-serialize.cc;l=3358?q=SnapshotCreatorAddData&sq=

[targos]

The first question I ask myself is: would there be any advantage for us from having core modules written in ESM?

[jasnell]

This is another one of the really good questions that needs to be answered. I would also flip it around and ask what are the critical disadvantages?

[targos]

Not necessarily critical, but here's what I can think of:

• Churn because we would have to change all requires and exports.
• Technical challenge to get access to objects like primordials and internalBinding that are currently passed as arguments to the function (maybe we can use import.meta).
• No top-level return anymore (not sure if we use that anywhere).
• Can we transition gradually or would we have to change all of the files in lib at once?

[Trott]

We'd lose deferred loading too I think, no?

[joyeecheung]

No top-level return anymore (not sure if we use that anywhere).

We do in the bootstrap script so that the C++ gets what's prepared in JS land though it can be replaced with a export-like object that gets passed around (or maybe just export them)

[mhdawson]

It would possibly also affect the ability to monkey patch? Causing issues for APMs etc.

[targos]

Note that technically, with #37587 (specifically this commit), part of core will effectively be ESM.

[joyeecheung]

The core libs are always going to be loaded with a special loader, right? So to be precise it's going to be a special version of ESM just like it's now a special version of CommonJS, and I think most of the obstacles can be solved with some special internal logic that won't affect the userland ;)