Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Android Support #86

Closed
cart opened this issue Aug 5, 2020 · 44 comments
Closed

Android Support #86

cart opened this issue Aug 5, 2020 · 44 comments
Labels
A-Build-System Related to build systems or continuous integration C-Feature A new feature, making something new possible O-Android Specific to the Android mobile operating system

Comments

@cart
Copy link
Member

cart commented Aug 5, 2020

It should be possible to run Bevy Apps on Android

@karroffel karroffel added A-Build-System Related to build systems or continuous integration C-Feature A new feature, making something new possible labels Aug 12, 2020
@PrototypeNM1
Copy link
Contributor

I'm working on this issue now, can I get it assigned to me to help head off accidental parallel implementations?

@cart
Copy link
Member Author

cart commented Aug 16, 2020

absolutely! thanks for picking it up. this will be huge ❤️

@cart
Copy link
Member Author

cart commented Aug 16, 2020

haha just noticed @karroffel already beat me to it

@endragor
Copy link

@PrototypeNM1 could you please share your progress and what's left to be done? I have some time on weekends that I could dedicate to adding Android support to Bevy. Last weekend I added Android support to cpal, which was one of the major blockers. There are still some issues on android-ndk-rs side. I could look into that, but it would be nice if you broke this issue into subitems and clarify what is done and what isn't.

@PrototypeNM1
Copy link
Contributor

To my knowledge there are 4 main blockers for Android support.

  1. Lack of Android support for cpal (wip) (thanks for tacking this!)
    Once fixed, may reveal issues in it's transitive dependents Rodio or winit.
  2. cargo apk is broken for how Bevy structures Cargo.toml (PR)
    Alt. removing glob imports from Bevy's Cargo.toml fixes this.
  3. Mobile needs input support, touch and IMU
  4. bevy-glsl-to-spirv requires a rewrite or replacement.

I'm working on 4. glsl-to-spirv as written depends on a bundled glslangValidator executable. This can't work on Android without hackery. Luckily glslangValidator has a recently added a C API.

@endragor since you started investigating cpal/oboe issues, I recommend figuring out why cargo apk had issues with including multiple shared libraries, doing so is helpful both for the Rust Android ecosystem and Bevy.

@MichaelHills
Copy link
Contributor

@PrototypeNM1 FYI over at #87 I put up #324 where currently uses shaderc-rs instead of glsl-to-spirv for iOS. Feel free to give it a try.

I too have now run into the missing touch support... I might hack something in just so I can play with it, not exactly sure what the correct API is.

@enfipy
Copy link
Contributor

enfipy commented Oct 11, 2020

Hey all! I was able to run Bevy 3d example on my Android phone.

Proof and repo:

Current issues are next:

  1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.
  2. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.
  3. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.
  4. Audio, 3d models, textures, etc were not tested due to problems with assets.

This is the current status. Any feedback and advice are highly appreciated.

@cart
Copy link
Member Author

cart commented Oct 11, 2020

Fantastic! This is a big step 😄

  1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.

Yeah as we discussed elsewhere I think we'll want to solve this by adding pre-compiled shader support with the various permutations available. We'll want a system like that anyway to improve startup times in release builds. Alternatively we can try adding dynamic compilation support by (1) waiting for Naga or (2) somehow making shaderc compile on android

  1. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.

Yeah thats a bit annoying, but it seems like we can work around it by selecting formats according to the platform. Alternatively, it might be worth checking with the wgpu folks to see if its something they can fix on their end.

  1. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.

Hmm maybe we need to add them to some sort of asset manifest? I haven't worked with android for awhile, but i vaguely remember something like that.

@PrototypeNM1
Copy link
Contributor

PrototypeNM1 commented Oct 12, 2020

@enfipy Awesome! Referencing your changes for TextureFormat I got runtime shader generation working on Android.

@enfipy
Copy link
Contributor

enfipy commented Oct 18, 2020

After about 10 hours of unsuccessful tries to make @PrototypeNM1 example with bevy-glsl-to-spirv work on my Mac (and Windows) - I added shaderc-rs support for Android (but it's still shitty) and was able to make shaders work in runtime.

Status update:

  • Shaders are working on runtime but the build still needs to be enhanced (or I very hope Naga will land soon). (Screenshot_1)
  • I added a temporary solution for assets loading on Android. I plan to add a better one to Bevy as @cart will finish asset refactoring.
  • PNG images rendered successfully (Screenshot_2).
  • Text and UI rendered successfully (Screenshot_1).

Things need to be resolved:

  • Audio. This PR looks good and hopefully will enable high-performance audio on Android.
  • TextureFormat. It looks like some issues still persist with Bgra/Rgba as I couldn't build a Monkey example and faced a similar issue that was before this (but I don't think it's very serious).
  • Shaderc. As I specified above - it's not finished and tested on other platforms yet and hard to build.
  • Request Permissions. From Android version 23 and higher, Android requires applications to request permissions at runtime. I want to add jni call to this in a couple of days.
  • Touch Support. I hope this will be available soon with this Touch support implementation #696.

This is all I remembered.

Screenshots:

Screenshot_1:

Screenshot_2:

@cart
Copy link
Member Author

cart commented Oct 18, 2020

Very nice progress! I just merged the asset system changes. I'm making a few more changes to AssetIo to re-add wasm compatibility. It should also make integrating the android AssetIo backend slightly easier (as I'm boxing AssetIo)

@trezm
Copy link

trezm commented Oct 18, 2020

I'd like to just say, I'm not working on this issue, but am wildly excited to see its progress. Huge thanks to everyone working on it and know that the community is ecstatic to see this moving forward!!

@PrototypeNM1
Copy link
Contributor

@enfipy @cart Given Enfipy has more time to work on and organize this issue, I think we should transfer this issue to them.

@cart
Copy link
Member Author

cart commented Oct 26, 2020

I think this is a big enough issue / problem space that I don't think anyone needs to own it.

@memoryruins memoryruins added the O-Android Specific to the Android mobile operating system label Oct 27, 2020
@PrototypeNM1
Copy link
Contributor

PrototypeNM1 commented Oct 28, 2020

Added support for runtime spirv generation using a rewrite of glsl-to-spirv.

We're still waiting for a fix in cargo-apk's dependencies to land. The alternative of removing glob imports for bevy's workspaces is still a viable alternative if we wanted this working asap.

@mmacedoeu
Copy link

Would like to bring attention to proper implementation of winit events Event::Suspended and Event::Resumed at https://github.com/bevyengine/bevy/blob/master/crates/bevy_winit/src/lib.rs#L170 so it can proper release and reload resources

@francesca64
Copy link

Would there be any interest in using cargo-mobile to help with building, generating Android Studio projects, running on device, etc?

@cart
Copy link
Member Author

cart commented Dec 2, 2020

Medium-to-long term I might be interested in adopting higher level abstractions. Short term I'd rather keep it simple, encourage people to become familiar with the "native" mobile tooling, and maintain control over "official" templates. We will likely create an official bevy_template repo in the near future.

A minor note on the current cargo-mobile bevy template: bevy has its own answer to #[mobile_entry_point] called #[bevy_main]. I'd prefer it if you used that instead (to support future non-mobile scenarios). Although thats only available on the master branch right now.

@alice-i-cecile alice-i-cecile removed this from the Bevy 0.8 milestone Jul 18, 2022
@alice-i-cecile
Copy link
Member

Removing from the 0.8 milestone; even if we merge #4913 I don't think this should be closed for 0.8 until we have significantly more testing.

@Leinnan
Copy link
Contributor

Leinnan commented Nov 14, 2022

Is there any update on that? What is the status of Android in 0.9?

@rib
Copy link
Contributor

rib commented Nov 14, 2022

Although I haven't looked at Bevy recently we did finally get the android-activity backend merged for Winit which should help with Android support but it will also require some integration work in Bevy when it updates to Winit 0.28.

One of the other notable thing upstreamed in Winit was that all backends now consistently deliver a Resume event which helps with writing portable code that runs on Android (where handling Resume is very important) and other window systems.

I'm not sure atm how tricky it's going to be to rebase #4913 (or maybe something equivalent was done in the mean time) but something equivalent to that will be one of the main blockers still I expect.

@mockersf
Copy link
Member

Is there any update on that? What is the status of Android in 0.9?

Works on some device but without audio, doesn't work on others at all.

android-activity backend merged for Winit

Do you know if there has been work to replace it also in things like cpal?

@rib
Copy link
Contributor

rib commented Nov 14, 2022

cpal has switched over to using ndk-context now so that it's no longer dependent on any glue crate so should hopefully just work now 🤞

E.g. for android-activity I created an example to test cpal here: https://github.com/rib/android-activity/tree/main/examples/agdk-cpal which works with cpal = "0.14"

@rib
Copy link
Contributor

rib commented Nov 14, 2022

here's the cpal PR that switched to ndk-context: RustAudio/cpal#641 for reference

@enfipy
Copy link
Contributor

enfipy commented Nov 14, 2022

Also, our team was working on the toolkit for plugins and build cli for Android (iOS in the future): https://github.com/dodorare/crossbow.

We succeeded in running different Google Play plugins on Android but Bevy was also blocked by stuff like audio, etc. @rib it would be awesome to take a look at how we can integrate with AGDK, we tried to design crossbow similar to Xamarin and Godot - so I think there will be no big problems with it.

Here's an example with Macroquad (but will work with Bevy too):
Crossbow Plugins

Honestly, I think if we all here put all this stuff together, we can make pretty fine Android support in Bevy v0.10, IMO.

@rib
Copy link
Contributor

rib commented Nov 14, 2022

cool, I'd heard of crosssbow recently but hadn't had a chance to take a look so thanks for the pointer @enfipy

Skimming that example I see it has a main entry point like

#[macroquad::main("Macroquad UI")]
async fn main() -> anyhow::Result<()> {
}

which makes me think that you're possibly defining your own glue layer between native and jvm code? Or does this maybe currently build on the ndk-glue macro perhaps?

Winit (which Bevy usually builds on) is currently (for 0.28) implies needing to use android-activity as a glue crate (previously ndk-glue) for handling integration with the Activity class (either NativeActivity or GameActivity from AGDK) and both of those Activity crates also generally define the architecture for spawning a native thread that will run your application's main() function.

I'd need to poke into crossbow more to understand how it defines its main entry point macro to see if we can figure out how to join the dots with android-activity.

You should have some flexibility from the proc macro I guess but one notable thing about the entry point ABI for android-activity is that it is passed an app: AndroidApp argument that represents the app and represents a kind of connection between a native thread and JVM Activity (to avoid global static state) and I wonder how that would fit with crossbow's current main function definition.

@rib
Copy link
Contributor

rib commented Nov 14, 2022

okey, I realize now that the example main() entry point is specific to the macroquad library - not something imposed by crossbow

@rib
Copy link
Contributor

rib commented Nov 14, 2022

btw another build tool that's possibly in a similar space to crosstool is xbuild. I guess you've maybe seen it already @enfipy but figured I'd mentioned it while I didn't see it in the list to existing tools mentioned in the readme for crosstool.

@slyedoc
Copy link
Contributor

slyedoc commented Dec 4, 2022

Got bevy_audio working if I tell cpal to enable oboe "shared-stdcxx" feature, meaning I am having an issue with "c++_static" linking. Still working on why, and not sure if its just me or not.

@rib
Copy link
Contributor

rib commented Dec 4, 2022

In case it might give some insight/clue you could maybe poke at this minimal cpal example: https://github.com/rib/android-activity/tree/main/examples/agdk-cpal

That example doesn't currently do anything special to choose the stdc++_shared vs static implementation and seemed to work.

There's currently no standard convention amongst Rust Android crates for being able to configure whether to use the shared or static implementation which is a bit awkward, since it wouldn't really be desirable to have a mix and match amongst crates.

It sounds kinda surprising that it works when enabling shared stdcxx. I would have thought you'd also have to make sure to copy the libc++_shared.so out of your NDK/toolchain into your package for that to run - but maybe cargo apk automagically does that.

@rib
Copy link
Contributor

rib commented Dec 4, 2022

oh sorry, I lost track of which bug I was responding too and see I already linked that example quite recently in this thread so guess you've probably already seen it.

@slyedoc
Copy link
Contributor

slyedoc commented Dec 5, 2022

but maybe cargo apk automagically does that.

It does, and your examples have been my starting point for everything in this PR. Used na-winit-wgpu quite a bit, and made sure I could get agdk-cpal working before anything else this morning.

I do want to add AGDK bevy example (you put in so much work getting that working), plan to after I am happy with native activity with apk.

@MarijnS95
Copy link

It sounds kinda surprising that it works when enabling shared stdcxx. I would have thought you'd also have to make sure to copy the libc++_shared.so out of your NDK/toolchain into your package for that to run - but maybe cargo apk automagically does that.

cargo-apk takes care of all your library linking problems 😉, and we landed the same feature in xbuild just a few days ago 🙂

@rib
Copy link
Contributor

rib commented Dec 6, 2022

@MarijnS95 does cargo apk maybe default to making crates link against libc++_shared.so as opposed to static linking?

As a guess, that seem like it could explain a discrepancy between the agdk-cpal example working and needing to enable shared-stdcxx with Bevy (while building with cargo apk)

@MarijnS95
Copy link

@rib cargo apk looks at what libraries the final .so requests linking against; if you configure the library to not link against dynamic libc++ (via i.e. the cc crate when compiling native code), it won't embed libc++_shared.so in the final APK.

@rib
Copy link
Contributor

rib commented Dec 7, 2022

@rib cargo apk looks at what libraries the final .so requests linking against; if you configure the library to not link against dynamic libc++ (via i.e. the cc crate when compiling native code), it won't embed libc++_shared.so in the final APK.

right, makes sense

bors bot pushed a commit that referenced this issue Feb 6, 2023
# Objective

- Merge the examples on iOS and Android
- Make sure they both work from the same code

## Solution

- don't create window when not in an active state (from #6830)
- exit on suspend on Android (from #6830)
- automatically enable dependency feature of bevy_audio on android so that it works out of the box
- don't inverse y position of touch events
- reuse the same example for both Android and iOS

Fixes #4616
Fixes #4103
Fixes #3648
Fixes #3458
Fixes #3249
Fixes #86
bors bot pushed a commit that referenced this issue Feb 6, 2023
# Objective

- Merge the examples on iOS and Android
- Make sure they both work from the same code

## Solution

- don't create window when not in an active state (from #6830)
- exit on suspend on Android (from #6830)
- automatically enable dependency feature of bevy_audio on android so that it works out of the box
- don't inverse y position of touch events
- reuse the same example for both Android and iOS

Fixes #4616
Fixes #4103
Fixes #3648
Fixes #3458
Fixes #3249
Fixes #86
@bors bors bot closed this as completed in 7e0a9bf Feb 6, 2023
@alice-i-cecile
Copy link
Member

Android is reasonably well supported now, following #9937. #10158 will fix an important bug with audio playing while suspended, but in general, it should be functional.

It needs more and better documentation still, and better multi-touch + haptic support, but the foundations are all there. If you run into further problems, please open a new issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Build-System Related to build systems or continuous integration C-Feature A new feature, making something new possible O-Android Specific to the Android mobile operating system
Projects
None yet
Development

Successfully merging a pull request may close this issue.