Is `require` with ESM planned? #308

tjcrowder · 2019-04-06T16:07:40Z

I'm embarrassed to say that I can't quite tell from the Plan for New Modules Implementation.

Suppose I have a project where I'm using the CommonJS default, and I install a module with "type": "module" in its package.json from npm. Is the current plan that I'll be able to use require to load exports from that ESM module? (Or similarly, if I just try to require an .mjs file...)

(To be clear: This is purely a question, not a veiled suggestion or criticism. Here's an unveiled comment, though: Thank you for the updated ESM stuff!)

It doesn't work with the v12 nightlies, but hey, it's nightlies, stuff is in flux. :-)

My test setup, in case I'm just doing it wrong:

index.js:

const { foo } = require("foo");

foo("Hi");

node_modules/foo/index.js:

export function foo(...args) {
    console.log("foo:", ...args);
}

node_modules/foo/package.json:

{
  "type": "module",
  "name": "foo",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT"
}

Command:

node12 --experimental-modules index.js

Result:

(node:20699) ExperimentalWarning: The ESM module loader is experimental.
/home/blah/blah/node_modules/foo/index.js:1
export function foo(...args) {
^^^^^^

SyntaxError: Unexpected token export
    at Module._compile (internal/modules/cjs/loader.js:768:23)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:835:10)
    at Module.load (internal/modules/cjs/loader.js:693:32)
    at Function.Module._load (internal/modules/cjs/loader.js:620:12)
    at Module.require (internal/modules/cjs/loader.js:731:19)
    at require (internal/modules/cjs/helpers.js:14:16)
    at Object. (/home/tjc/temp/esmcheck/index.js:1:17)
    at Module._compile (internal/modules/cjs/loader.js:824:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:835:10)
    at Module.load (internal/modules/cjs/loader.js:693:32)

The text was updated successfully, but these errors were encountered:

MylesBorins · 2019-04-06T16:25:34Z

There are currently no plans on the road map to support requiring esm Some members of the team are exploring the possibility but I am personally skeptical of the ability to do so The biggest challenge is moving from and async loader to a sync loader and back or sync to async to sync. Something we have referred to as zebra striping. It introduces all sorts of problems including deadlock and cache duplication.

…

On Sat, Apr 6, 2019, 12:07 PM T.J. Crowder ***@***.***> wrote: I'm embarrassed to say that I can't quite tell from the Plan for New Modules Implementation <https://github.com/nodejs/modules/blob/master/doc/plan-for-new-modules-implementation.md> . Suppose I have a project where I'm using the CommonJS default, and I install a module with "type": "module" in its package.json from npm. Is the current plan that I'll be able to use require to load exports from that ESM module? (Or similarly, if I just try to require an .mjs file...) (To be clear: This is purely a question, not a veiled suggestion or criticism. Here's an unveiled comment, though: *Thank you* for the updated ESM stuff!) It doesn't work with the v12 nightlies, but hey, it's nightlies, stuff is in flux. :-) My test setup, in case I'm just doing it wrong: index.js: const { foo } = require("foo"); foo("Hi"); node_modules/foo/index.js: export function foo(...args) { console.log("foo:", ...args); } node_modules/foo/package.json: { "type": "module", "name": "foo", "version": "1.0.0", "main": "index.js", "license": "MIT" } Command: node12 --experimental-modules index.js Result: (node:20699) ExperimentalWarning: The ESM module loader is experimental. /home/blah/blah/node_modules/foo/index.js:1 export function foo(...args) { ^^^^^^ SyntaxError: Unexpected token export at Module._compile (internal/modules/cjs/loader.js:768:23) at Object.Module._extensions..js (internal/modules/cjs/loader.js:835:10) at Module.load (internal/modules/cjs/loader.js:693:32) at Function.Module._load (internal/modules/cjs/loader.js:620:12) at Module.require (internal/modules/cjs/loader.js:731:19) at require (internal/modules/cjs/helpers.js:14:16) at Object. (/home/tjc/temp/esmcheck/index.js:1:17) at Module._compile (internal/modules/cjs/loader.js:824:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:835:10) at Module.load (internal/modules/cjs/loader.js:693:32) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#308>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecV5TL4vgOurOA-5m8zAN8rpCRKXxfks5veMZNgaJpZM4cgRwj> .

devsnek · 2019-04-06T16:26:53Z

we do have a design for allowing require(esm) but it's still in the early stages.

as a side note, i would not advise mixing cjs and esm in the same extension within a package

robpalme · 2019-04-06T16:37:35Z

Even if require(esm) was provided today it becomes hazardous to use over time as async-to-eval modules get introduced. For example a JS module using top-level await or a WebAssembly module. This can happen in dependencies you don't control, so it's not easy to defend against this hazard.

For this reason I'm also skeptical that this feature could be provided in a safe way that guaranteed future compatibility.

The safe way for CJS to import ESM is dynamic import().

ljharb · 2019-04-06T16:38:51Z

There also remains the possibility of dual modules, so you’d be able to require or import them.

tjcrowder · 2019-04-06T16:43:45Z

Wow, never expected to get such quick and complete answers. Thank you all!!

tjcrowder · 2019-04-06T16:45:42Z

@devsnek -

as a side note, i would not advise mixing cjs and esm in the same extension within a package

LOL no. That would be...bad...

weswigham · 2019-04-08T21:29:30Z

Even if require(esm) was provided today it becomes hazardous to use over time as async-to-eval modules get introduced. For example a JS module using top-level await or a WebAssembly module. This can happen in dependencies you don't control, so it's not easy to defend against this hazard.

This is baseless fearmongering, IMO - the synchronicity (or lack thereof) of the execution of a module has no bearing on the synchronicity of the resolution of the module graph containing it; which is all that matters for pulling on the correct namespace objects. Introducing async evaluation does introduce the possibility of witnessing namespaces whose members do not yet have final values, but this is already possible in CJS today, eg with

setTimeout(() => module.exports = {x: 12}, 100);

nodejs/node#49450 has a proposal allowing require(esm) and this is a partial implementation (a proof of concept that integrating a require(esm) is certainly possible), with full support of "zebra striping" as it is called. Cache duplication isn't an issue when there's only one canonical loader for each cache entry (unlike a dual impl model where "cache duplication" is replaced with "implementation duplication" which is even worse), and deadlocking isn't a problem provided the code within node itself that's doing the syncification doesn't introduce async dependencies that can deadlock (user code cannot produce a deadlock, pending the final design of loaders with might be within the syncificed pipeline and thus may need some extra (already desired) context separation).

The fears around it are overblown, IMO.

robpalme · 2019-04-08T22:24:01Z

@weswigham Thanks for highlighting the proposal. Let's continue main discussion there.

I look forwards to learning how you can eliminate the hazard 😉

This implements the ability to use require on .mjs files, loaded via the esm loader, using the same tradeoffs that top level await makes in esm itself. What this means: If possible, all execution and evaluation is done synchronously, via immediately unwrapping the execution's component promises. This means that any and all existing code should have no observable change in behavior, as there exist no asynchronous modules as of yet. The catch is that once a module which requires asynchronous execution is used, it must yield to the event loop to perform that execution, which, in turn, can allow other code to execute before the continuation after the async action, which is observable to callers of the now asynchronous module. If this matters to your callers, this means making your module execution asynchronous could be considered a breaking change to your library, however in practice, it will not matter for most callers. Moreover, as the ecosystem exists today, there are zero asynchronously executing modules, and so until there are, there are no downsides to this approach at all, as no execution is changed from what one would expect today (excepting, ofc, that it's no longer an error to require("./foo.mjs"). Ref: nodejs/modules#308 Ref: https://github.com/nodejs/modules/issues/299 Ref: nodejs/modules#454

stevenvachon · 2021-02-27T01:38:06Z

Any update on this?

NullDev · 2021-10-29T16:23:27Z

This would be really nice. Many dependencies are dropping require for ESM. So once we update, we'd have to change all the CommonJS imports to ESM.

ljharb · 2021-10-29T16:29:36Z

… or, you could drop those dependencies and find replacements that maintain compatibility.

NullDev · 2021-10-29T16:33:33Z

@ljharb So, rewriting the whole code instead of changing the imports? 😄
Why not just add the ability to load ESM with commonjs.

I mean, while at it why not just replace all the require()'s to import()'s...

let { default: fetch } = await import("node-fetch");

Not really a solution either ^^

ljharb · 2021-10-29T19:29:10Z

@NullDev no, rewriting the code is what you'd need to do to keep using a dep in CJS or transpiled ESM that dropped support for CJS. Changing the imports is the easy path - iow, using a different dependency. TLA isn't available outside of native ESM, so your solution wouldn't work.

tjcrowder closed this as completed Apr 6, 2019

jkrems mentioned this issue Dec 10, 2019

Open issues with require(esm) #454

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Is `require` with ESM planned? #308

Is `require` with ESM planned? #308

tjcrowder commented Apr 6, 2019

MylesBorins commented Apr 6, 2019 via email

devsnek commented Apr 6, 2019

robpalme commented Apr 6, 2019

ljharb commented Apr 6, 2019

tjcrowder commented Apr 6, 2019

tjcrowder commented Apr 6, 2019

weswigham commented Apr 8, 2019

robpalme commented Apr 8, 2019

stevenvachon commented Feb 27, 2021

NullDev commented Oct 29, 2021

ljharb commented Oct 29, 2021

NullDev commented Oct 29, 2021 •

edited

Loading

ljharb commented Oct 29, 2021

Is require with ESM planned? #308

Is require with ESM planned? #308

Comments

tjcrowder commented Apr 6, 2019

MylesBorins commented Apr 6, 2019 via email

devsnek commented Apr 6, 2019

robpalme commented Apr 6, 2019

ljharb commented Apr 6, 2019

tjcrowder commented Apr 6, 2019

tjcrowder commented Apr 6, 2019

weswigham commented Apr 8, 2019

robpalme commented Apr 8, 2019

stevenvachon commented Feb 27, 2021

NullDev commented Oct 29, 2021

ljharb commented Oct 29, 2021

NullDev commented Oct 29, 2021 • edited Loading

ljharb commented Oct 29, 2021

Is `require` with ESM planned? #308

Is `require` with ESM planned? #308

NullDev commented Oct 29, 2021 •

edited

Loading