Node.js ecosystem compatibility breaks w/ ES Modules

[mikeal]

I'll be attending this meeting as an invited expert, primarily to cover this topic.

The Node.js Foundation CTC along with Bradley Meck and many others have been diving deep into what it will take to implement ES Modules in Node.js. The current state of the spec, if unmodified, leaves Node.js in a position where implementing it "as is" would cause compatibility breaks with the existing module system that the Node.js project is not willing to make.

Without spec changes Node.js will be forced to implement the new ES Modules support in a way that is not entirely compatible with the spec as well as rely on out of spec behaviors like .jsm file extensions.

I'll be attending to try to represent the project consensus thus far and we are also making sure that several of the other member companies are sending their Node.js contributors as well.

[littledan]

Do you have a link to more details about what you're going to talk about? Sometimes we link to slides or explainer docs on the agenda. This helps bring the discussion further along, in my experience.

[mikeal]

I know that @bmeck has a flow graph somewhere. There's also an active EPS here nodejs/node-eps#39

One of the difficult things with this has been the interlinking of various small issues so it's hard to find a single endpoint that explains everything.

[littledan]

Thanks for the reference. If you could write up something that pulls it all together, I bet we can make more progress on addressing the issues in TC39. Either way, I merged the agenda item.

[bmeck]

@littledan most of it is summed up in ways via the PR referenced

The flow chart mentioned is also sub linked in the PR: https://docs.google.com/drawings/d/1YVdM-VIfSZifia12xxZ6-lx_3lA7rwLX8aRg_96IlWA/edit

The discussion will most likely center around what was the expected plan of node implementing a synchronous loader, being able to share modules with the browser, and the roadblocks of achieving such a goal. Not all of these need be seen as singular items, rather the amount of roadblocking increases with every difference / breakage to existing expectation on how using ES Modules can interact with other module systems. Highlights below:

1. problems supporting named imports from existing/transpiled modules
   • problems delegating variable bindings to value properties
     • getter/setter and implication of variable access
     • problems deferring to Reflect.Module via a Proxy
   • problems porting Node's core to ES Modules
     • problems implementing a second core that is based upon ES Modules
2. synchronous loading problems
   • recursive HostResolveImportedModule implications
     • circular dependency errors
   • implication of top level await
   • implication of browser stack unwinding for all loading operations
3. semantic differences/problems of module cache/instrumentation
   • permanent errors/idempotency of HostResolveImportedModule
     • problems of having 2 different cache mechanisms and semantics
   • mocking, spying, and instrumentation
4. differing behavior from browsers
   • specifier parsing as URLs

[bmeck]

The only clear path to avoid this is that PR rewrite linked above, but that causes some pretty massive breakage related to those points above that breaks most existing expectations in modules in some way once ES Modules start being adopted. [hence the talks of not being to spec]