Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove usage of a WebWorker/worker thread #2498

Merged
merged 41 commits into from
Jul 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
231a57f
Remove liveness pings
tomaka Jul 11, 2022
5a75d38
Remove "current task" system
tomaka Jul 11, 2022
ad9d586
Rename instance.ts -> raw-instance.ts
tomaka Jul 11, 2022
5acb494
Turn worker.ts into a simple object
tomaka Jul 11, 2022
063679b
Remove the utilities that deal with the worker
tomaka Jul 11, 2022
3782fc1
Provide the configuration when starting and not asychronously
tomaka Jul 11, 2022
4fe54bd
Use a Promise for state rather than an array
tomaka Jul 11, 2022
01dfd45
Use queueOperation instead of handleMessage
tomaka Jul 11, 2022
34117ef
Turn generic handleMessage into individual functions
tomaka Jul 11, 2022
5ca6ba0
Make worker.addChain return a Promise
tomaka Jul 11, 2022
e4ff8bd
addChain is now properly async
tomaka Jul 11, 2022
b478da1
Make removeChain, request, and databaseContent always synchronous
tomaka Jul 11, 2022
511c6c0
Track the active chains within worker.ts
tomaka Jul 11, 2022
025d6ab
Fix state wasn't being set properly
tomaka Jul 11, 2022
f240fb4
Make databaseContent return a Promise
tomaka Jul 11, 2022
c61a4a6
Provide logCallback in config and finish removing messages
tomaka Jul 11, 2022
26d3c45
Remove mention of the worker
tomaka Jul 11, 2022
ba4addc
Remove unused node:worker_threads import
tomaka Jul 11, 2022
97a3944
Finish removing the ToWorker messages
tomaka Jul 11, 2022
48a2698
Get rid of messages module
tomaka Jul 11, 2022
2a9130d
Add some console.asserts
tomaka Jul 11, 2022
a590f73
No longer track chains in client.ts
tomaka Jul 11, 2022
338bd00
Move the number handling of databaseContent to worker.ts
tomaka Jul 11, 2022
3aeba77
Rename worker to instance
tomaka Jul 11, 2022
b31d463
Pass onProcExit rather than throwing directly
tomaka Jul 12, 2022
dea8db8
Add onPanic to smoldot bindings
tomaka Jul 12, 2022
2476fa4
Clarify panic handling situation
tomaka Jul 12, 2022
77ab5a7
Finally properly handle crashes
tomaka Jul 12, 2022
750f848
Properly format instance.ts
tomaka Jul 12, 2022
f1ed3e3
Remove obsolete code
tomaka Jul 12, 2022
8126cf6
Implement clean shutdown of the instance
tomaka Jul 12, 2022
15af6cd
Fix removeChain and databaseContent still callable
tomaka Jul 12, 2022
ebe5764
Document start_shutdown
tomaka Jul 12, 2022
9532b5f
Clarify alreadyDestroyedError
tomaka Jul 12, 2022
c9ed8ac
Merge branch 'main' into rm-worker
tomaka Jul 12, 2022
aa3ca94
CHANGELOG
tomaka Jul 12, 2022
13bc166
Fix startShutdown handling of errors
tomaka Jul 12, 2022
9dce352
Spellcheck
tomaka Jul 12, 2022
4e22435
Rename
tomaka Jul 12, 2022
fd00321
More rename
tomaka Jul 12, 2022
00f0e1e
Merge branch 'main' into rm-worker
mergify[bot] Jul 13, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions bin/wasm-node/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### Changed

- No WebWorker/worker thread is spawned anymore by the JavaScript code. The WebAssembly virtual machine that runs smoldot is now directly instantiated by the `start` function. This should fix compatibility issues with various JavaScript bundlers. ([#2498](https://github.com/paritytech/smoldot/pull/2498))

## 0.6.23 - 2022-07-11

### Fixed
Expand Down
19 changes: 0 additions & 19 deletions bin/wasm-node/javascript/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,22 +73,3 @@ chains must be passed as parameter to `addChain` as well. In situations where th
specifications passed to `addChain` are not trusted, it is important for security reasons to not
establish a parachain-relay-chain link between two chains that aren't part of the same "trust
sandbox".

# About the worker

The code in this package uses a web worker (in browsers) or a worker thread (on NodeJS). The
line of JavaScript that creates the worker is of the following form:

``` js
new Worker(new URL('./worker.js', import.meta.url), { type: "module" });
```

This format is compatible [with Webpack 5](https://webpack.js.org/guides/web-workers/), meaning
that Webpack will be able to resolve the imports in `worker.js` and adjust this little snippet.

This format also works in NodeJS without any issue.

However, at the time of writing of this comment, this format doesn't work with Parcel (both 1 and
2) due to various bugs.

As a general warning, be aware of the fact that this line might cause issues if you use a bundler.
3 changes: 1 addition & 2 deletions bin/wasm-node/javascript/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@
"test": "node prepare.js --debug && rimraf ./dist && tsc && ava --timeout=2m --concurrency 2 --no-worker-threads"
},
"browser": {
"./dist/compat/index.js": "./dist/compat/index-browser-overwrite.js",
"./dist/worker/spawn.js": "./dist/worker/spawn-browser-overwrite.js"
"./dist/compat/index.js": "./dist/compat/index-browser-overwrite.js"
},
"dependencies": {
"buffer": "^6.0.1",
Expand Down
10 changes: 5 additions & 5 deletions bin/wasm-node/javascript/prepare.js
Original file line number Diff line number Diff line change
Expand Up @@ -69,13 +69,13 @@ child_process.execSync(
{ 'stdio': 'inherit' }
);

// The code below will write a variable number of files to the `src/worker/autogen` directory.
// The code below will write a variable number of files to the `src/instance/autogen` directory.
// Start by clearing all existing files from this directory in case there are some left from past
// builds.
const filesToRemove = fs.readdirSync('./src/worker/autogen');
const filesToRemove = fs.readdirSync('./src/instance/autogen');
for (const file of filesToRemove) {
if (!file.startsWith('.')) // Don't want to remove the `.gitignore` or `.npmignore` or similar
fs.unlinkSync(path.join("./src/worker/autogen", file));
fs.unlinkSync(path.join("./src/instance/autogen", file));
}

// We then do an optimization pass on the Wasm file, using `wasm-opt`.
Expand Down Expand Up @@ -119,13 +119,13 @@ try {
const chunk = base64Data.slice(0, 1024 * 1024);
// We could simply export the chunk instead of a function that returns the chunk, but that
// would cause TypeScript to generate a definitions file containing a copy of the entire chunk.
fs.writeFileSync('./src/worker/autogen/wasm' + fileNum + '.ts', 'export default function(): string { return "' + chunk + '"; }');
fs.writeFileSync('./src/instance/autogen/wasm' + fileNum + '.ts', 'export default function(): string { return "' + chunk + '"; }');
imports += 'import { default as wasm' + fileNum + ' } from \'./wasm' + fileNum + '.js\';\n';
chunksSum += ' + wasm' + fileNum + '()';
fileNum += 1;
base64Data = base64Data.slice(1024 * 1024);
}
fs.writeFileSync('./src/worker/autogen/wasm.ts', imports + 'export default ' + chunksSum + ';');
fs.writeFileSync('./src/instance/autogen/wasm.ts', imports + 'export default ' + chunksSum + ';');

} finally {
fs.rmSync(tmpDir, { recursive: true });
Expand Down
Loading