Use V8 Snapshots

[leafac]

This protects the source code and may lead to a marginal performance improvement.

#4 (comment)

[pdcastro]

Is this related to what pkg calls V8 bytecode? -

https://www.npmjs.com/package/pkg
Bytecode (reproducibility)
By default, your source code is precompiled to v8 bytecode before being written to the output file. To disable this feature, pass --no-bytecode to pkg.
Why would you want to do this?
If you need a reproducible build process where your executable hashes (e.g. md5, sha1, sha256, etc.) are the same value between builds. Because compiling bytecode is not deterministic (see here or here) it results in executables with differing hashed values. Disabling bytecode compilation allows a given input to always have the same output.

If so, it would be good to have a way of disabling it too, for the reproducibility benefit in opensource projects where protecting the code is not a concern.

[leafac]

Is this related to what pkg calls V8 bytecode?

Yes, that’s exactly it.

And thanks for the information; I didn’t know that compiling bytecode is nondeterministic and that it could be an issue. When we get to this, I’ll make it optional. As it stands, caxa builds should be deterministic and produce exactly the same binary every time (though I haven’t checked).

[rightaway]

@leafac Looking forward to this so that switching from pkg is possible. Does this issue need nodejs/node#35711 to be added first?

[leafac]

@rightaway I suppose you’re right. Support for snapshots in Node.js would make adding support for it in caxa much more natural and straightforward.

[rightaway]

@leafac Is there any way to get protected source code before this issue is solved?

[leafac]

As far as I know currently the best you can do is obfuscate the JavaScript.

[leafac]

Exciting news: https://nodejs.org/en/blog/announcements/v18-release-announce/#build-time-user-land-snapshot-experimental

In summary:

• Starting with Node.js v18, you can compile Node.js from scratch and include your application in the executable. This means that the source code for your application won’t be readable by people who have the executable. But compiling Node.js from scratch can take a long time.

• The current way you compile Node.js from scratch with your application is a bit clunky, but the Node.js team is working on improving it. Right now you need two entrypoints, one that puts your application on the global namespace, and one that actually starts off the process. Note that this second entry point would still be readable by people who have your binary, but it should boil down to something like MyApplication.start(). And this awkwardness should be addressed soon anyway.

• The Node.js team is also working on letting you produce the snapshot of your application without requiring you to compile Node.js from scratch. Things will be much more straightforward and I think that when this comes out is the right point for us to start adding support for it in caxa.

• None of this subsumes or obsoletes anything in caxa, because as far as I understand the Node.js team doesn’t have the intention of including assets (images, fonts, and so forth) in the snapshot. And I don’t really know what the story with native extensions looks like either. Maybe you’d have to provide your .node files containing native extensions separately. And that’s where caxa helps you out.

• In general, this whole development in Node.js helps caxa users in two ways: Source-code protection, and potentially faster startup times, particularly on Windows which struggles a bit with a ton of small files like most JavaScript projects tend to be.

[luckyyyyy]

Exciting news: https://nodejs.org/en/blog/announcements/v18-release-announce/#build-time-user-land-snapshot-experimental

In summary:

• Starting with Node.js v18, you can compile Node.js from scratch and include your application in the executable. This means that the source code for your application won’t be readable by people who have the executable. But compiling Node.js from scratch can take a long time.
• The current way you compile Node.js from scratch with your application is a bit clunky, but the Node.js team is working on improving it. Right now you need two entrypoints, one that puts your application on the global namespace, and one that actually starts off the process. Note that this second entry point would still be readable by people who have your binary, but it should boil down to something like MyApplication.start(). And this awkwardness should be addressed soon anyway.
• The Node.js team is also working on letting you produce the snapshot of your application without requiring you to compile Node.js from scratch. Things will be much more straightforward and I think that when this comes out is the right point for us to start adding support for it in caxa.
• None of this subsumes or obsoletes anything in caxa, because as far as I understand the Node.js team doesn’t have the intention of including assets (images, fonts, and so forth) in the snapshot. And I don’t really know what the story with native extensions looks like either. Maybe you’d have to provide your .node files containing native extensions separately. And that’s where caxa helps you out.
• In general, this whole development in Node.js helps caxa users in two ways: Source-code protection, and potentially faster startup times, particularly on Windows which struggles a bit with a ton of small files like most JavaScript projects tend to be.

When will support be available? what else do we need to do

[leafac]

When will support be available?

I don’t think that the Node.js have a specific deadline for delivering the features we’d need. And we don’t have a deadline for exactly how long it’ll take us to do the necessary adaptations in caxa.

what else do we need to do

You contribution can certainly help speed things up, so thanks for offering. For the time being, you may head to the Node.js project and follow along with nodejs/node#42617 & nodejs/node#38905 Offer your help and the Node.js team will direct you from there.

Once the Node.js features are out, you may come back here and we’ll discuss how to best integrate with caxa. We’ll be happy to have your help.

[rightaway]

@leafac Are the node dependencies ready or is there more they need to do? Any updates with node 19 out and 20 a few months away?

[rightaway]

Once the Node.js features are out, you may come back here and we’ll discuss how to best integrate with caxa.

@leafac Because those issues are closed and node 20 is out with https://nodejs.org/api/single-executable-applications.html could we discuss how to integrate?

[leafac]

Hi y’all,

Thanks for using caxa and for the conversation here.

I’ve been thinking about the broad strategy employed by caxa and concluded that there is a better way to solve the problem. It doesn’t address V8 Snapshots, and I consider them outside the scope of this new project, but if y’all learn more about the topic, I’m interested in what you have to say—and I’m sure other users will be interested as well, so please share.

It’s a different enough approach that I think it deserves a new name, and it’s part of a bigger toolset that I’m building, which I call Radically Straightforward · Package.

I’m deprecating caxa and archiving this repository. I invite you to continue the conversation in Radically Straightforward’s issues.

Best.

[robertsLando]

@leafac Could I ask you why you never add commit messages on your repos?