-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WebWorker support? #312
Comments
Oh, interesting. That's pretty cool. I didn't know about that feature of Parcel. I assume this also works for the However, the cache busting part won't work yet. None of the entry points for esbuild are cache-busted right now. I'm working on enabling this as part of general code splitting support (see #16) but it's still a work in progress. Introducing cache busting in entry point names is also a breaking change and I've been waiting to do this until the next batch of breaking changes. Once that's in, I think it should be relatively straightforward to make something like this work. |
This sort of feature is sorely needed in Rollup and can be achieved in Webpack with something like worker-plugin. Having this built into esbuild as an opt-in would be absolutely fantastic! I think deferring to plugins would likely not do it 'right' as these would need to parse an AST, figure out bindings, etc.. (or just RegEx and yolo). |
There are other web apis that would benefit from this form of non |
I think this would be a really nice feature. As a stopgap, I was thinking this could be implemented as an esbuild plugin. This would impose some rather unfortunate syntax, but in a pinch, something like this would work: import workerUrl from "workerUrl(./worker.js)";
const worker = new Worker(workerUrl) import path from "path";
import { build } from "esbuild";
let workerLoader = (plugin) => {
plugin.setName("worker-loader");
plugin.addResolver({ filter: /^workerUrl\((.+)\)/ }, (args) => {
return { path: args.path, namespace: "workerUrl" };
});
plugin.addLoader(
{ filter: /^workerUrl\((.+)\)/, namespace: "workerUrl" },
async (args) => {
let match = /^workerUrl\((.+)\)/.exec(args.path),
workerPath = match[1];
let outfile = path.join("dist", path.basename(workerPath));
try {
// bundle worker entry in a sub-process
await build({
entryPoints: [workerPath],
outfile,
minify: true,
bundle: true,
});
// return the bundled path
return { contents: `export default ${JSON.stringify(workerPath)};` };
} catch (e) {
// ...
}
}
);
}; |
Besides path resolution within |
As of version 5.x webpack can bundle web workers out of the box with the following syntax: new Worker( new URL( "./worker", import.meta.url ), { type: "module" } )
This bundler looks great and I'd really look forward to seeing this feature added! It's been too hard to use WebWorkers generically for too long... |
Update: I created a minimal demonstration for how this plugin would work at https://github.com/endreymarcell/esbuild-plugin-webworker
|
Webpack does not inline it either, it's emitted as a separate file too. |
@RReverser oh OK, thanks for the correction. I updated the original comment to not mislead anyone. |
Side note: it's possible to (partially) minify worker code with the inline-worker technique: let worker = new Worker(URL.createObjectURL(new Blob([
(function(){
// ...worker code goes here
}).toString().slice(11,-1) ], { type: "text/javascript" })
)); so as a workaround, workers can be stored in usual js files and included as follows: // worker.js
export default URL.createObjectURL(new Blob([(function(){
// ...worker code
}).toString().slice(11,-1)], {type:'text/javascript'})) // main.js
import workerUrl from './worker.js'
let worker = new Worker(workerUrl) |
The
This is fragile and can easily break with esbuild. For example, setting the language target to
This is my current plan to support this feature. This approach is sufficiently general and doesn't need to know anything about web workers. That would make this issue a duplicate of #795. It also looks like Parcel might drop support for the syntax that was originally proposed in this thread: parcel-bundler/parcel#5430 (comment). So that's another count against the original approach. |
Note that while there is some overlap, those are not really duplicates, because general asset handling and Worker handling need to differ. In particular, with general asset handling Worker JS file would be treated as raw binary, and wouldn't get minified, the imports wouldn't resolve and so on - which is not what user normally wants. For this reason other bundlers have separate implementation paths for these two features. |
I was assuming that the |
Fair enough. That's not how it's handled in any of those bundlers AFAIK, but seems to be a sensible route too (probably also needs to include |
This would also support the version without While I would love for this to all be ES Modules, neither Firefox nor Safari have any support for ES Module workers, which would make Firefox issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1247687 |
Not necessarily, the point of |
I guess that makes sense, assuming it always bundles down into "old school" Workers. I'd expect the flag to be kept in the output though, as the choice can have pretty big effects on other things, for example Basically, an author should be able to always pick "old school" workers for cross-browser support, but also optionally use ES Module workers for Chrome (which has supported this for a long time). |
Usually it has to be removed precisely because it has effect on other things. E.g. if bundler supports code splitting, then chunk loading usually has to be transformed down to |
Oh right. Thanks! I updated my implementation at https://github.com/endreymarcell/esbuild-plugin-webworker to use pluginData instead of the hack. |
for me, ESM works as a work build target for the most part when combined with 'bundle' except, I need to remove all I submitted a PR to microsoft/vscode#123739 |
I managed to resolve this after following the docs at link. |
It seems Firefox and Safari are newly making progress on module workers. Given the hours I've wasted on CJS workarounds, I'd love for But I more importantly for me, I would like to note that that it would be most useful if this was not specifically tied to the implicit/explicit global new CustomWorkerConstructor( new URL( "./worker", import.meta.url ), { type: "module" } ) (It's possible to assign Handling this at the |
Another update to this issue is that Emscripten on C++ side and wasm-bindgen-rayon on Rust side both now emit |
It appears that Chrome has very recently implemented the sync |
…/debugging the dev page. Uses the workaround from evanw/esbuild#312 (comment) , since evanw/esbuild#2508 has not landed.
…/debugging the dev page. Uses the workaround from evanw/esbuild#312 (comment) , since evanw/esbuild#2508 has not landed.
It would be nice if the esbuild uses the |
IMO it'd be cool if something like the following got implemented, which as far as I know is pretty much the cleanest web worker abstraction possible: import {foo, bar} from './foo?worker';
await foo ( 123 ); Where every exported function is an async function. Now the type checker doesn't need to be aware that those functions will be executed in web worker, your code calling those functions doesn't need to know that either (unless in some cases if you are passing on transferrable objects), and neither the functions themselves really need to be changed, other than being marked as async. Something like this can be implemented as a plugin, but something more tightly integrated with the bundler would work better. |
I tried the various approach of this thread - thanks for all the great ideas - but the only solution that worked out for me once I embedded the lib that contains the worker within my apps was loaded the worker as base64 as displayed in the gist https://gist.github.com/manzt/689e4937f5ae998c56af72efc9217ef0 of @manzt |
Release notes: - Switch purely to module workers. - As of [Firefox 114](https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/114), all modern JavaScript browsers and runtimes with worker support are now able to instantiate module workers. This is important for performance, as it significantly improves worker loading for `cubing.js` (requiring ⅓ as much code as before): #214 - Due to the heavy complexity and maintenance burden of alternatives to module workers, `cubing.js` is dropping support for "classic" workers and there is no longer a "catch-all" fallback when worker instantiation fails. In removing this fallback, have carefully made sure `cubing.js` scrambles will continue work out of the box with all modern browsers, as well as the main runtimes that support web worker (`node` and `deno`). We have also tested compatibility against major bundlers. However, if you are passing `cubing.js` through your own choice of bundler, then it must correctly bundle the worker "entry file" using one of the following: - [`import.meta.resolve(…)`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta/resolve) — this is the only "officially supported" way that we plan to support indefinitely, and the only one that will work without showing a warning in the JavaScript console. - One of two variants of using `new URL(…, import.meta.url)` as an alternative to `import.meta.resolve(…)`. - Using this workaround for `esbuild`: evanw/esbuild#312 (comment)
Release notes: - Switch purely to module workers. - As of [Firefox 114](https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/114), all modern JavaScript browsers and runtimes with worker support are now able to instantiate module workers. This is important for performance, as it significantly improves worker loading for `cubing.js` (requiring ⅓ as much code as before): #214 - Due to the heavy complexity and maintenance burden of alternatives to module workers, `cubing.js` is dropping support for "classic" workers and there is no longer a "catch-all" fallback when worker instantiation fails. In removing this fallback, have carefully made sure `cubing.js` scrambles will continue work out of the box with all modern browsers, as well as the main runtimes that support web worker (`node` and `deno`). We have also tested compatibility against major bundlers. However, if you are passing `cubing.js` through your own choice of bundler, then it must correctly bundle the worker "entry file" using one of the following: - [`import.meta.resolve(…)`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/import.meta/resolve) — this is the only "officially supported" way that we plan to support indefinitely, and the only one that will work without showing a warning in the JavaScript console. - One of two variants of using `new URL(…, import.meta.url)` as an alternative to `import.meta.resolve(…)`. - Using this workaround for `esbuild`: evanw/esbuild#312 (comment)
@evanw |
I do not advocate this approach, but if you are desperate, new-url-v0.18.13 , has I have included a devcontainer in the branch to make it simple to get going. If you don't know how to get a container going in vscode, um, read tutorial. Fire up the container and vscode should prompt you to install any additional go stuff you need. The follow assumes you are building for linux-x64 platform - change if need be. make clean
make platform-linux-x64 new binary will be at ./npm/@esbuild/linux-x64/bin/esbuild. With ESBUILD_BINARY_PATH you can use this new binary with the esbuild from npm. The versions of npm esbuild and this new binary HAVE TO match, new-url-v0.18.13 is currently pinned to If you copy ./npm/@esbuild/linux-x64/bin/esbuild out of container, and into your project at bin/esbuild - just add note that I use on a project with only js. typescript is untested. |
Glad to see that the code from last year can still be adapted! 🤓 It looks like you're trying to support bare package names in firien@196a279 |
For example: `bun run src/bin/scramble.ts -- 333` Thanks to oven-sh/bun#3645 and oven-sh/bun#3669, we can use `bun` directly on our source code (except the WASM parts). This requires a few changes: - Move around the source code to account for the fact that `esbuild` does not have understand relative `new URL(…, import.meta.url)` or `import.meta.resolve(…)` references yet: evanw/esbuild#312 (comment) - This has the unfortunate side effect that some files have to move to the source code root. This isn't *bad* per se, but it breaks some assumptions while still relying on some other assumptions. I hope we can move the code back some time soon. - Avoid using the trampoline workaround when we seem to be in a browser environment. - Avoid assuming that the output of `await import.meta.resolve(…)` can be passed to `new URL(…)` (`bun` returns a bath without the `file:` protocol).
We can't quite do this for the `esm` build, due to evanw/esbuild#312
I would prefer to avoid hardcoding any paths except those that are published as CDN entry points, but `esbuild` still doesn't support `new URL(…, import.meta.url)` (evanw/esbuild#312) or `import.meta.resolve(…)` (evanw/esbuild#2866).
…irefox. Users of the CDN can't prevent the worker failure messages, so they are just noise (and a potential slowdown). This hardcodes the entry file path where the instantiator (which reliably ends up in a chunk for us) expects it: https://github.com/cubing/cubing.js/blob/4afc6d727549aadc1e82abf52d531565953e7a9a/src/cubing/search/worker-workarounds/index.ts#L11 I would prefer to avoid hardcoding any paths except those that are published as CDN entry points, but `esbuild` still doesn't support `new URL(…, import.meta.url)` (evanw/esbuild#312) or `import.meta.resolve(…)` (evanw/esbuild#2866).
any news on this? |
I'm late to this thread, having faced this issue with an audio worklet. I eventually resorted to the following (rather primitive) workaround: In esbuild settings: entryPoints: [
'./src/www/components/audioRecorder/index.mts',
'./src/www/components/audioRecorder/audioRecorderWorklet.mts'
] In the audio worklet export function getWorkletUrl() {
return import.meta.url;
}
if (globalThis.AudioWorkletProcessor) {
const AudioRecorderWorklet = class extends AudioWorkletProcessor {
// ... audio worklet class
}
registerProcessor('audio-recorder-worklet', AudioRecorderWorklet);
} In the main file import { getWorkletUrl } from './audioRecorderWorklet.mjs'
//...
await audioContext.audioWorklet.addModule(getWorkletUrl()); This works, but the esbuild-generated import {
getWorkletUrl
} from "../../chunk-GRFBZUB4.mjs";
import "../../chunk-4OBMG7SZ.mjs";
export {
getWorkletUrl
}; And the URL returned by I hope someone (including my future self) may find this useful, but I'm still looking for a better way of doing this. PS. Huge thank to Evan for creating and maintaining ESBuild. It's an immensely useful tool, which keeps on delivering for cases where similar tools often fall short. |
This is what I found when looking how to do it: evanw/esbuild#2508 evanw/esbuild#312
I saw this issue was still open and thought esbuild didn't support the |
@fasiha, I believe #2439 is for allowing esbuild to preserve the comment, so it can make it to webpack. (https://github.com/privatenumber/esbuild-loader) |
Any updates? |
This issue can also be worked around with explicitly passing some kind of build id to esbuild, for example: # can be replaced with anything, build id, random hash, commit id, date
BUILD_ID = $(date +"%s%N")
esbuild \
--asset-names="./[name]_$(BUILD_ID)" \
--entry-names="./[name]_$(BUILD_ID)" \
--bundle \
--define:__BUILD_ID=\"$(BUILD_ID)\" \
--define:__STATIC_FILES_PATH="http://mycdn" \ # replace with your path to static files
controller_mycontroller.ts \
worker_myworker.ts It will produce a worker file in the output directory named exactly like new Worker(`${__STATIC_FILES_PATH}/worker_myworker_${__BUILD_ID}.js`) |
Hi I just discovered esbuild, coming from Parcel. One thing I like about parcel is that if I instaniate a WebWorker with the string literal filename, like
Parcel will recognize the
Worker
constructor and create another bundle starting at./myWorker.js
. It also handles cache busting for the filename. So the bundle would be called./myWorker8a68r8912q.js
or something and that string would be updated in the code above.Does ESBuild do something like that? If not, where would I look to implement that?
The text was updated successfully, but these errors were encountered: