devDependencies are part of the binary

[renkei]

@leafac
What a cool project! Thank you very much for your work on this. Once ARM support is ready it is a very interesting alternative to pkg although the source code is not hidden and must be evaluated first.

After some tests with my project on Linux I found out that all of my devDependencies are part of the resulting binary (file size is ca. 42 MB, ca. 160 MB after extraction). In the README you write:

You don’t need to npm prune --production and npm dedupe, because caxa will do that for you from within the build directory. (Otherwise, if you tried to npm prune --production you’d uninstall caxa, which should probably be in devDependencies.)

but this seems not to be the case for me. My current workaround:

1. Install caxa globally with npm install -g caxa
2. Delete the node_modules folder of my project, it seems to be unused by caxa
3. Delete the devDependencies section in my package.json
4. Run caxa as recommended, this seems to cause the installation of all remaining dependencies from package.json in the build directory of caxa.

Now, I get a working binary with file size of ca. 15 MB (ca. 39 MB after extraction). The result is impressive. Is there something wrong with this approach to avoid the devDependencies?

[papb]

This looks like a bug in caxa. Caxa should install the dev dependencies but prune them right before packaging.

[renkei]

After some more tests, I've found out that this is only true on Linux, but not on Windows.

On Windows, my workaround does not work, because deleting the node_modules folder causes a binary without any dependencies and the app cannot start. So, on Windows, the node_modules folder must be kept and yes, npm prune seems to be executed on Windows.

Anyway, when I compare the extracted node_modules folder (ca. 80MB) from the binary with a node_modules folder (ca. 39 MB) that I've manullay created with npm install --production I still see a significant difference in size. Maybe it is an option not to reuse the node_modules folder and to "optimize" it with npm prune but to create a new one with npm install --production before creating the executable. What do you think?

Please forget the previous section, the size difference is because of the node.exe in node_modules/.bin. So, to prune the existing node_modules folder seems to be as efficient in size as to create a new one with npm install --production.

[fcastilloec]

I can confirm that this is definitely happening on Linux.

@renkei my workaround fixes the problem but it makes the whole process slower and can become a problem if you're source code is very big (not including node_modules). So, I first run a script that prepares the files for caxa, here's a sample:

output_dir='build'
include_list="
src/
package.json
package-lock.json"

mkdir -p $output_dir
echo "$include_list" | rsync -arc --files-from=- . "$output_dir"
cd $output_dir
npm ci --production
cd -
npx caxa -d $output_dir -c "{{caxa}}/node_modules/.bin/node" "{{caxa}}/src/index.js" -o output

You can play with rsync to sync whatever you need, I just find it easy to have all my source code in one directory. This also has the added benefit of excluding any other folders and/or files you might not need (i.e. .github/, .git/, .eslintrc.json, etc).
The key part is running npm ci --production which only installs what you need. It will not install caxa but we are using caxa from the main project directory (or you can install it globally but I haven't tried this), this is why we run cd - to go back to your main project directory.
You can also copy node_modules into $output_dir and run npm prune instead of npm ci --production, both will have the same outcome, I guess it's a matter of personal preference.

You can even run this as an npm script, which has the added benefits of having access to npm_package_ variables from the script without having to explicitly pass them. For example, you can use $npm_package_name inside the script as the -o option for caxa to produce the output based on the name of your project.

You can see that it's slower because you have to copy files and run npm ci --production. At least, with rsync, you'll copy only changed files.

[btarg]

This is happening to me on Windows. Caxa itself, along with other stuff specified in devDepenencies is being packaged into the final binary.


npm prune --production doesn't seem to be making a massive difference.

[fcastilloec]

@leafac After further testing, I've realized that npm dedupe reinstalls all dependencies including all devDependencies, undoing what npm prune --production did.
A simple fix to this issue would be to just remove this line https://github.com/leafac/caxa/blob/master/src/index.ts#L31. Or at the very least run prune after dedupe and not before.

PR #8 could also fix this issue, but I think that PR is targeted at introducing a new feature rather than fixing this issue. It just happens to fix this issue as a side effect.

[leafac]

Hi all,

First, thanks for the bug report and for all the investigative work. You really made it easy for me to fix the issue 😃

This has been fixed as part of v2.0.0. There are actually two solutions:

1. We aren’t running npm prune --production and then npm dedupe (which undoes the pruning). Instead, we’re doing only npm dedupe --production, which happens to prune as well!
2. If you want to opt out of this entirely, you may use the --no-dedupe option. You may combine this with the new --prepare-command option to have complete control over what runs on the build directory before packaging.

Let me know how it works for you.

[leafac]

Oh, and by the way, v2.0.0 also includes support for Linux ARM! @renkei, I hope you like this 😃