Denoland/x Publishing

[harrysolovay]

I propose that––upon merging #2426––we release the Smoldot JS lib to the Denoland/x registry.

Assuming we want to do so, we should consider two changes:

1. Committing assets: Deno users point directly at JS/TS source files. When these files point to / depend on gitignored assets, their execution will fail. For instance, the src/worker/instance.ts file points to src/worker/autogen/wasm.js (which is gitignored). We'll need to commit files such as this.
2. The release process: Denoland/x relies on GH tags + releases as a source of publish truth. Whenever a release is created, a webhook (specified in the repo settings) notifies Denoland/x of the new tag-specified version. A detailed description can be found here.

[tomaka]

These are annoying constraints.

1. The autogen/wasm.js file is actually re-generated whenever you run npm run ... or npm publish. But the file generated by npm publish isn't the same as the one generated by npm start for example. The file we publish has additional optimizations enabled that very significantly increase the build time.

2. I intentionally don't use tags in this repo because it contains 3 different projects (the library, the full node, and the light client) that each have a different version number.

[tomaka]

The easiest solution, I suppose, is to put the "release versions" of the smoldot light client in a separate branch that isn't main.
For example, whenever there's a version bump, a bot builds the file, creates a commit, tags it, and pushes it, but the commit isn't on the main branch.
And tags would be named light-x.y.z instead of just x.y.z

[harrysolovay]

We could also (A) create a deno_smoldot repo & (B) configure this repo with...

• a GH user access token, scoped to have the ability to trigger a workflow in the deno_smoldot repo.
• a workflow that (upon merged release-titled PR) triggers the workflow in deno_smoldot

The other repo's workflow would tag in accordance with this repo's light client version and create the release, which would trigger the publish to denoland/x.

[tomaka]

I've published a version manually, to test: https://deno.land/x/[email protected]

Trying to use the published version leads to:

error: Remote modules are not allowed to import local modules. Consider using a dynamic import instead.
  Importing: node:crypto
    at https://deno.land/x/[email protected]/compat/index.js:20:32

[harrysolovay]

Unfortunately, I believe you'll need to create a Deno-specific build (with the import and usage swapped out for that of Deno's crypto utils).

A more radical solution may be to make this repo's JS Deno-first and then utilize DNT to create Node/Browser-specific versions (see dnt, a tool for generating Node/Browser packages from Deno projects).

[tomaka]

(see dnt, a tool for generating Node/Browser packages from Deno projects)

I'm reluctant to use tools that might be abandoned in the future. At this point, NodeJS/npm are technologies that we can assume will not disappear from one day to the next. When it comes to Deno and especially dnt I think that the risk is still high.

[harrysolovay]

Dnt is the flagship packaging tool of Deno (which btw, just received $18.5+ funding from Sequoia). The Deno core team consists of the NodeJS creators and most expert members of the TypeScript community.

TL;DR: definitely not going to disappear. Very low risk.

[tomaka]

Todo list:

• Remove the worker from the JS bindings #2488
• Make users inject some kind of connectivity object into the JS API #2489
• Consider replacing buffer JavaScript dependency with manually written JavaScript #732

Afterwards, we should be able to publish.

[tomaka]

Another problem which I didn't think of is the pako dependency, that we use to unzip the Wasm runtime.

cc #1802

[harrysolovay]

I'd recommend replacing with deno_lz4.

[tomaka]

The problem isn't finding a replacement, but the fact that my strategy of getting rid of all dependencies has an obstacle.

I might just copy-paste https://github.com/dankogai/js-deflate/blob/master/rawinflate.js within the source tree and call it a day.

EDIT: can't do that because it's GPLv2 and we're v3

[tomaka]

I've published v0.6.26 manually and everything seems to work.

#2538 will make the publication automatic.