Why are bevy executable so large?

[morr0ne]

After tinkering around with bevy for a bit I decided to run the this example.
After building in release mode with several optimizations: link-time optimization (lto) enable, codegen units set to 1, default linker (#1057), striping the binary
I noticed a resulting binary of 14.1 Mb.
Even after setting abort to panic and optimizing for size, I get a 10.1 Mb binary, which while being a big improvement is still pretty big for such a simple game

Running cargo-bloat doesn't help has 90%+ of the methods are discarded as too small

My questions then obviously are: Why is the executable so large? Can anything be done to fix it? What improvements can I make and what has to be improved on bevy?

I'm not a rust nor a game developing expert so forgive me if I'm asking something incredibly stupid

My config:
i5-8520u
940mx with priopetary nvidia drivers
Manjaro kde 20.2

[cart]

Earlier in the project I'm pretty sure I had size-optimized binaries that were just slightly over 1 MB. At some point we regressed. I'm guessing it was probably this pr: #740, as we start linking to some pretty large non-rust binaries. It would be interesting to see if checking out the commit right before that pr cuts size-optimized binary size.

[cart]

And if it does, the good news is that the approach there is a temporary measure. As soon as the pure-rust Naga shader compiler is ready we'll drop bevy-glsl-to-spirv.

[morr0ne]

Thanks for the fast response, I'll try compiling the same example before that commit

[morr0ne]

After reverting to this commit which seems to be the one just before #740 using the same optimizations I get a 9.1 Mb and a 5.3 Mb respectively. That's a huge difference for just one change. I'm definitely looking forward to naga but in the meantime is there no solution? That change was made to support android so it possible it could be disabled for other platforms? Also, Is it possible to entirely avoid a shader compiler and compile everything at compile time? Wouldn't that improve both size and speed? I'm no Vulkan expert but I roughly know how shaders work, would it be possible even if just some shaders are converted to spir-v at compile time?

[cart]

That change was made to support android so it possible it could be disabled for other platforms?

We also support the shaderc compiler on some platforms (ios + web). The main problem is that it requires a more complicated build system. Especially on windows, it requires non-standard dependencies like Ninja + python. I want Bevy to be as "plug and play" as possible. It shouldn't require a bunch of manual setup.

I would like to make it possible to opt-in to shaderc on any platform, but there are some cargo corner cases that make it non-trivial. We need it to be the default on some platforms, but opt-in on other platforms (which doesn't quite work, but it soon will rust-lang/cargo#7914).

Also, Is it possible to entirely avoid a shader compiler and compile everything at compile time? Wouldn't that improve both size and speed? I'm no Vulkan expert but I roughly know how shaders work, would it be possible even if just some shaders are converted to spir-v at compile time?

Yup this is absolutely a scenario I want to support. You can already do this with simple shaders (the shader type supports pre-compiled spirv), but the ShaderDef system that bevy uses to compile shader variants based on component/assets values requires a bit more thought to allow pre-compiling (as the ShaderDefs can change at compile time). You either need to brute-force compile every permutation (which will be infeasible for shaders with many variants) or you need to enumerate the variants you know you need ahead of time.

[morr0ne]

My understanding is that Naga will (hopefully) resolve all this problems.
Regarding compile shaders at compile time are there examples available? Especially about enumerating them. Would it make sense to provide an example of breakout with compile time shaders?

[cart]

Hmm adding precompiled "built in" shaders is non-trivial because of the whole ShaderDef thing mentioned above. I'm planning on making the built-in shaders have many ShaderDefs, which will produce the combinatorics explosion mentioned above. So doing that now feels a bit short-sighted.

I think my ideal solution is: cheap-to-include shader compiler (ex: Naga) plus optional pre-compiled SPIRV caching (maybe users opt in specific permutations, maybe we auto-cache any permutation used by the game).

Rather than go in a direction that conflicts with the "ideal solution" in the short term interest of smaller binaries, I'd rather find some other workaround (such as sorting out a way for you to use shaderc).

[morr0ne]

Binary size problem I feel like it will be mostly solved by switching to Naga, which right now seems the most promising pure rust shader compiler, especially considering it's made by the same team working on gfx/wgpu

I believe that the ShaderDefs implementation should be thought out more, spir-v caching and any other type of shader optimization may be of a small impact on binary size and speed for such simple projects but at a big scale it could have really big impact

Also some people might prefer shaderc over Naga anyway

The best solution is probably to give as much choice as possible to the user, by providing many ShaderDefs options and the ability to change shader compiler

After this gets resolved, would it be possible to provide shaderc on all platforms?
How would I switch to shaderc right now?

[cart]

I believe that the ShaderDefs implementation should be thought out more, spir-v caching and any other type of shader optimization may be of a small impact on binary size and speed for such simple projects but at a big scale it could have really big impact

Fully agreed. Pre-compiling shaders is an important feature.

After this gets resolved, would it be possible to provide shaderc on all platforms?

yup!

How would I switch to shaderc right now?

I would clone a local copy of bevy, make the relevant changes, and use that:

• Edit this cargo.toml to use shaderc on all platforms: https://github.com/bevyengine/bevy/blob/master/crates/bevy_render/Cargo.toml#L51
• Edit this file to use the shaderc compilation logic on all platforms:

  bevy/crates/bevy_render/src/shader/shader.rs

  Line 64 in 51650f1

  pub fn glsl_to_spirv(

[morr0ne]

Would it make sense to add this features to the roadmap as they seem pretty important or are they already there and i just missed them?

[cart]

Probably 😄

Right now the roadmap isn't a perfect reflection of everything we plan to do. Its largely a way to communicate what we're currently working on and a few "big ticket" items on the horizon.

[morr0ne]

I see, I'll dig around in the source maybe I can contribute some myself :D

Since I'm still learning my way around bevy I'll probably stick a bigger binaries for now but thanks for the really fast response, bevy is exactly what I'm looking for in a game engine, and seeing how open and active this project is compared to others is really promising, keep up the good work!

Edit: I'll leave this issue open in case someone has the same size problems or wants add something to the discussion

[CleanCut]

We need it to be the default on some platforms, but opt-in on other platforms (which doesn't quite work, but it soon will rust-lang/cargo#7914).

Maybe very soon! rust-lang/cargo#8997

[NHodgesVFX]

Now breakout is 16.9mb large when compiled in release mode on Windows. If I am reading the result of Cargo bloat correct then the majority of the size is 12. 6mb worth of text data, leaving 4.3 mb of binary data. If I compress the resulting exe I get 5.56mb for standard zip compression or 3.29mb when using 7z. Not sure if its possible but it would be nice to see the file size shrink a bit.

Cargo Bloat output from the breakout example on Bevy 0.7 and Rust 1.60
CrateSizes.txt
FunctionSizes.txt

[bjorn3]

We don't have control over libstd, naga and wgpu, which combined account for 3-4MB of text data. I don't see anything spring out as outrageously large. I do notice that bevy_render for you is 400kb bigger than for me on linux (1.3MB vs 900KB). This may be caused by the unstable-Zshare-generics=yes being used by me as part of the fast compile profile. Note that most of the size is fixed overhead of bevy itself independent of the size of your own game.

[michaelrampl]

I just managed to reduce my executable size from 35.9 to 9.7 MiB.
These are the settings I introduced for the release build

[profile.release]
opt-level = "z"
strip = "symbols"
lto = "fat"
codegen-units = 1
panic = 'abort'

[profile.release.package."*"]
opt-level = "z"
strip = "symbols"
codegen-units = 1

This is pretty impressive but I wonder If I can do anything more to reduce the size?

[flodihn]

Just an update from 2024.
I just compiled the breakout example on Windows.

In release without optimizations the binary is now 55 MB.
The debug EXE is 102 MB and the PDB file generated with it is about 800 MB.

Cargo is installing 445 packages.

[PatchMixolydic]

Another datapoint: on x86_64-unknown-linux-gnu, the breakout example in 2bd3282, built with the following optimizations:

• Cargo.toml

[profile.release]
strip = "symbols"
lto = true
codegen-units = 1
panic = "abort"

• .cargo/config.toml

[build]
rustflags = ["-C", "link-arg=-fuse-ld=lld", "-Z", "share-generics"]

• mod stepping manually removed

... comes out to 35.6 MiB (37,344,184 bytes).