iOS Support #87

cart · 2020-08-05T02:18:55Z

It should be possible to run Bevy Apps on iOS

MichaelHills · 2020-08-21T13:49:07Z

Hi, what’s involved in setting up Bevy on iOS? My understanding is that it uses winit which should already support iOS?

MichaelHills · 2020-08-22T12:23:31Z

I followed this guide to just get a Rust lib working from an ios project. Then I tried to substitute a bevy example app in. During cargo lipo --release it fails to build bevy-glsl-to-spirv.

It looks like there's no prebuilt binary in that repo so it tries to build glslang and then cmake fails

  -- Check for working C compiler: /usr/bin/cc -- broken
...
Building C object CMakeFiles/cmTC_97014.dir/testCCompiler.c.o
      /usr/bin/cc   -fPIC -m64 -m64 -mios-simulator-version-min=7.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.2.sdk -fembed-bitcode  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk   -o CMakeFiles/cmTC_97014.dir/testCCompiler.c.o   -c /Users/mikeh/rust-ios-example/rust/target/x86_64-apple-ios/release/build/bevy-glsl-to-spirv-51cc402e67486116/out/build/CMakeFiles/CMakeTmp/testCCompiler.c
      clang: warning: using sysroot for 'MacOSX' but targeting 'iPhone' [-Wincompatible-sysroot]

Apparently that last line is a key sign something is off. I take it the build for glslang doesn't work properly to target aarch64-apple-ios or x86_64-apple-ios.

MichaelHills · 2020-08-22T14:38:58Z

I tried a couple of things, but I'm new to rust and new to cross-compiling to iOS, though slowly figuring things out. The approach of spawning the glslValidator process probably isn't going to fly on iOS. MoltenVK apparently can build the iOS static lib libglslang.a so I considered trying to interface with that. The bevy-glsl-to-spirv repo says TODO to switch to shaderc-rs so I figured I'd have a crack at that but had trouble cross-compiling for iOS.

I ended up stubbing out the code in crates/bevy_render/src/shader/shader.rs just to see how far I could get. I also had to disable the optional bevy audio as it didn't compile either (forgot why).

I managed to finally get something to build, on the simulator gfx-rs / gfx-backend-metal failed with Target OS is incompatible: library was not compiled for the simulator so I ran it on my phone instead to which I got

EventLoop cannot be run after a call to UIApplicationMain on iOS.

Stopping for tonight. Here's my diff so far to get where i got to...

diff --git a/Cargo.toml b/Cargo.toml
index 90b2368d..3254fd6c 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -13,7 +13,8 @@ readme = "README.md"
 exclude = ["assets/**/*", "tools/**/*", ".github/**/*", "crates/**/*"]
 
 [features]
-default = ["bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr", "mp3"]
+#default = ["bevy_audio", "bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr", "mp3"]
+default = ["bevy_gltf", "bevy_wgpu", "bevy_winit", "png", "hdr"]
 profiler = ["bevy_ecs/profiler", "bevy_diagnostic/profiler"]
 wgpu_trace = ["bevy_wgpu/trace"]
 
@@ -22,10 +23,10 @@ png = ["bevy_render/png"]
 hdr = ["bevy_render/hdr"]
 
 # Audio format support (MP3 is enabled by default)
-mp3 = ["bevy_audio/mp3"]
-flac = ["bevy_audio/flac"]
-wav = ["bevy_audio/wav"]
-vorbis = ["bevy_audio/vorbis"]
+#mp3 = ["bevy_audio/mp3"]
+#flac = ["bevy_audio/flac"]
+#wav = ["bevy_audio/wav"]
+#vorbis = ["bevy_audio/vorbis"]
 
 serialize = ["bevy_input/serialize"]
 
@@ -56,7 +57,7 @@ bevy_ui = { path = "crates/bevy_ui", version = "0.1" }
 bevy_window = { path = "crates/bevy_window", version = "0.1" }
 
 # bevy (optional)
-bevy_audio = { path = "crates/bevy_audio", optional = true, version = "0.1" }
+#bevy_audio = { path = "crates/bevy_audio", optional = true, version = "0.1" }
 bevy_gltf = { path = "crates/bevy_gltf", optional = true, version = "0.1" }
 bevy_wgpu = { path = "crates/bevy_wgpu", optional = true, version = "0.1" }
 bevy_winit = { path = "crates/bevy_winit", optional = true, version = "0.1" }
diff --git a/crates/bevy_render/Cargo.toml b/crates/bevy_render/Cargo.toml
index ea29b856..53fe2af4 100644
--- a/crates/bevy_render/Cargo.toml
+++ b/crates/bevy_render/Cargo.toml
@@ -24,7 +24,7 @@ bevy_window = { path = "../bevy_window", version = "0.1" }
 
 # rendering
 spirv-reflect = "0.2.3"
-bevy-glsl-to-spirv = "0.1.7"
+#bevy-glsl-to-spirv = { path = "/Users/mikeh/repos/glsl-to-spirv" } #"0.1.7"
 image = { version = "0.23", default-features = false }
 
 # misc
diff --git a/crates/bevy_render/src/shader/shader.rs b/crates/bevy_render/src/shader/shader.rs
index f963c0c9..dd448e74 100644
--- a/crates/bevy_render/src/shader/shader.rs
+++ b/crates/bevy_render/src/shader/shader.rs
@@ -1,6 +1,6 @@
 use super::ShaderLayout;
 use bevy_asset::Handle;
-use bevy_glsl_to_spirv::compile;
+//use bevy_glsl_to_spirv::compile;
 use std::{io::Read, marker::Copy};
 
 /// The stage of a shader
@@ -11,25 +11,26 @@ pub enum ShaderStage {
     Compute,
 }
 
-impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
-    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
-        match self {
-            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
-            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
-            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
-        }
-    }
-}
+//impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
+//    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
+//        match self {
+//            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
+//            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
+//            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
+//        }
+//    }
+//}
 
 fn glsl_to_spirv(
     glsl_source: &str,
     stage: ShaderStage,
     shader_defs: Option<&[String]>,
 ) -> Vec<u32> {
-    let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
+    //let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
     let mut spv_bytes = Vec::new();
-    output.read_to_end(&mut spv_bytes).unwrap();
-    bytes_to_words(&spv_bytes)
+    //output.read_to_end(&mut spv_bytes).unwrap();
+    //bytes_to_words(&spv_bytes)
+    spv_bytes
 }
 
 fn bytes_to_words(bytes: &[u8]) -> Vec<u32> {

MichaelHills · 2020-08-23T00:13:03Z

Looks like cross-compiling iOS stuff with cmake-rs is currently broken rust-lang/cmake-rs#96 so that's probably the source of some of my woes

MichaelHills · 2020-08-23T05:36:22Z

Success! I managed to get the 3d_scene demo loading in portrait mode (albeit a bit warped) on my phone.

I pinned cmake-rs to 0.1.43 because that's the version which had the working iOS support
Dropped bevy-glsl-to-spirv in favour of using shaderc-rs directly from crates/bevy_render/src/shader/shader.rs
Created a main.swift to call out to my bevy_main() function directly which was the 3d_sprite main function renamed like this

#[no_mangle]
pub extern fn bevy_main() {

In xcode I had to add the static lib / header and search paths as per this site and I also had to link against lc++. Basically atm I build my "game" as a static lib and then call out to it manually from in an ios xcode project.

Trying to figure out how to get it into landscape mode now.

cart · 2020-08-23T06:02:22Z

Whoa that's huge! Thanks for the hard work here. People are going to love this!

MichaelHills · 2020-08-23T06:28:09Z

No worries. :) When I saw the post on Bevy on HN I thought it looked pretty awesome. I previously had started making a PC game in UE4 but life got busy. Decided I wanted to prototype it out as a mobile game instead and learn some Rust/ECS while I'm at it.

How should I go about getting stuff landed? I think the switch to shaderc-rs in general can be done now (tested a couple of examples still work). Did you want that to be done inside of bevy-glsl-to-spirv? I've currently just implemented it directly in crates/bevy_render/src/shader/shader.rs while hacking this all up.

In order to get iOS integration to "just work" the following issues need to be solved properly:

Land using shaderc-rs in bevy
Get that iOS support commit in cmake-rs fixed up and re-landed there
In shaderc-rs I had to edit CMakeLists.txt in the glsl and shaderc submodules (see below), so maybe... get those landed there?
Figure out why coreaudio-sys v0.2.5 panics during compile (that was why I had to disable audio)

shaderc

diff --git a/glslc/CMakeLists.txt b/glslc/CMakeLists.txt
index acf6fb0..4ec0067 100644
--- a/glslc/CMakeLists.txt
+++ b/glslc/CMakeLists.txt
@@ -40,6 +40,7 @@ shaderc_add_asciidoc(glslc_doc_README README)

 if(SHADERC_ENABLE_INSTALL)
   install(TARGETS glslc_exe
+    BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
     RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
 endif(SHADERC_ENABLE_INSTALL)

glslang

diff --git a/StandAlone/CMakeLists.txt b/StandAlone/CMakeLists.txt
index 2cf2899c..f84fdd2b 100644
--- a/StandAlone/CMakeLists.txt
+++ b/StandAlone/CMakeLists.txt
@@ -50,11 +50,13 @@ endif(WIN32)

 if(ENABLE_GLSLANG_INSTALL)
     install(TARGETS glslangValidator EXPORT glslangValidatorTargets
+            BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
     install(EXPORT glslangValidatorTargets DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake)

     if(ENABLE_SPVREMAPPER)
         install(TARGETS spirv-remap EXPORT spirv-remapTargets
+            BUNDLE DESTINATION ${CMAKE_INSTALL_BINDIR}
             RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR})
         install(EXPORT spirv-remapTargets DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake)
     endif()

Also here's the change to shader.rs

diff --git a/crates/bevy_render/src/shader/shader.rs b/crates/bevy_render/src/shader/shader.rs
index f963c0c9..2e710cd0 100644
--- a/crates/bevy_render/src/shader/shader.rs
+++ b/crates/bevy_render/src/shader/shader.rs
@@ -1,7 +1,6 @@
 use super::ShaderLayout;
 use bevy_asset::Handle;
-use bevy_glsl_to_spirv::compile;
-use std::{io::Read, marker::Copy};
+use std::{marker::Copy};
 
 /// The stage of a shader
 #[derive(Hash, Eq, PartialEq, Copy, Clone, Debug)]
@@ -11,14 +10,14 @@ pub enum ShaderStage {
     Compute,
 }
 
-impl Into<bevy_glsl_to_spirv::ShaderType> for ShaderStage {
-    fn into(self) -> bevy_glsl_to_spirv::ShaderType {
-        match self {
-            ShaderStage::Vertex => bevy_glsl_to_spirv::ShaderType::Vertex,
-            ShaderStage::Fragment => bevy_glsl_to_spirv::ShaderType::Fragment,
-            ShaderStage::Compute => bevy_glsl_to_spirv::ShaderType::Compute,
-        }
-    }
+impl Into<shaderc::ShaderKind> for ShaderStage {
+   fn into(self) -> shaderc::ShaderKind {
+       match self {
+           ShaderStage::Vertex => shaderc::ShaderKind::Vertex,
+           ShaderStage::Fragment => shaderc::ShaderKind::Fragment,
+           ShaderStage::Compute => shaderc::ShaderKind::Compute,
+       }
+   }
 }
 
 fn glsl_to_spirv(
@@ -26,10 +25,19 @@ fn glsl_to_spirv(
     stage: ShaderStage,
     shader_defs: Option<&[String]>,
 ) -> Vec<u32> {
-    let mut output = compile(glsl_source, stage.into(), shader_defs).unwrap();
-    let mut spv_bytes = Vec::new();
-    output.read_to_end(&mut spv_bytes).unwrap();
-    bytes_to_words(&spv_bytes)
+    let mut compiler = shaderc::Compiler::new().unwrap();
+    let mut options = shaderc::CompileOptions::new().unwrap();
+    if let Some(shader_defs) = shader_defs {
+        for def in shader_defs.iter() {
+            options.add_macro_definition(def, None);
+        }
+    }
+
+    let binary_result = compiler.compile_into_spirv(
+        glsl_source, stage.into(),
+        "shader.glsl", "main", Some(&options)).unwrap();
+
+    binary_result.as_binary().to_vec()
 }
 
 fn bytes_to_words(bytes: &[u8]) -> Vec<u32> {

MichaelHills · 2020-08-24T12:48:56Z

Stumbled across this issue in CMake https://gitlab.kitware.com/cmake/cmake/-/issues/20956 which suggests that the whole BUNDLE DESTINATION thing might be due to some CMake default that doesn't make sense when building libs, and I don't need to try and get BUNDLE DESINATION added to shaderc and glsl.

naithar · 2020-08-24T17:58:26Z

Basically atm I build my "game" as a static lib and then call out to it manually from in an ios xcode project.

Godot works in same way, though it implements it's own main which gets used by iOS application.

Obj-C can be used too, but this would require only main.mm file like this:

@import UIKit;

extern "C" void bevy_main(void);

int main(int argc, char * argv[]) {
    bevy_main();
}

And additional values in Other C Flags in Build Settings: -fmodules -fcxx-modules.

I've tested with 2d -> sprite.rs example, but got empty window.
I guess asset loading would require some work for iOS, but correct (?) texture handle was returned Handle<Texture>(82f42690-2c58-4843-a88a-aa066583f97d), so I'm not sure.

Figure out why coreaudio-sys v0.2.5 panics during compile (that was why I had to disable audio)

This PR (RustAudio/coreaudio-sys#33) seems to be addressing this issue, but I wasn't able to make it work even with this changes. But maybe it's just me as I don't really have much experience with rust.

MichaelHills · 2020-08-24T20:59:33Z

@naithar oh cool glad to see someone else getting it working too. Yeah I didn’t get the sprite example working, I also assumed it was related to asset loading but I haven’t looked into that yet.

Interesting I’ll try pulling that audio PR later to see if it works.

MichaelHills · 2020-08-24T22:46:31Z

By the way were you able to get the simulator working? I’m still getting the issue with Target OS is incompatible: library was not compiled for the simulator but supposedly there should be simulator metal support gfx-rs/gfx#2873 so not sure yet what the problem with that is.

naithar · 2020-08-24T23:15:18Z

No, I've got some error about unsupported architecture from dependency. I'll try to look more into it after I figure out asset loading :)

Godot uses 13.0 sdk version for simulator, since this the only simulator version supporting Metal. But while it does build, MoltenVK fails and application doesn't launch in the end.
So cargo lipo might be using lower sdk version that actually required for bevy.

MichaelHills · 2020-08-25T06:26:45Z

Looks like the ios audio support is still in-progress as per your link, I tried and failed to get that branch to work as well. I'll leave audio disabled for now and might have a try at getting the simulator going.

naithar · 2020-08-25T10:17:48Z

So compiled binary of bevy faces the same problem - not displaying sprite at specified path (icon.png).
Providing a full path fixed this for macOS, but haven't for iOS.

MichaelHills · 2020-08-25T12:23:07Z

I played around with the paths on iOS on my phone, I added icon.png to my xcode project and examined the various paths...

Bundle.main.bundlePath -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app
std::env::current_exe() -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/rust-ios
Bundle.main.path(forResource: "icon", ofType: "png") -> /var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/icon.png

I tried std::fs::canonicalize(std::env::current_exe().unwrap().join("..").join("icon.png")).unwrap(); to produce /private/var/containers/Bundle/Application/6AD264E9-6A09-47CE-B20A-56F906BE56CD/rust-ios.app/icon.png and pass that to the asset loader. Logs in the loader showed it was successfully loading the file. I guess you already checked the texture handle... but still nothing shows up on screen.

I tried adding some color to the texture to see if it'd help, it didn't. I also tried just the color by itself and no texture, but that doesn't seem to work even on mac.

        .spawn(SpriteComponents {
            material: materials.add(ColorMaterial {
                color: Color::RED,
                texture: texture_handle.into(),
            }),
            ..Default::default()
        });

If we can get a red square rendering, that'd be a good first step. I edited the fragment shader to show only red, worked on mac, didn't work on iOS.

naithar · 2020-08-25T12:35:32Z

It seems like iOS doesn't really handle/load assets in any way.
Also compiled binaries doesn't really like relative paths too, as I've reported in #344

I'm looking for a way to debug rust on iOS, so problem could be traced, but I've got no progress yet.

MichaelHills · 2020-08-25T13:08:36Z

In crates/bevy_render/src/render_graph/nodes/pass_node.rs I added some logs, basically can_draw_indexed() returns false because one of the "bind groups" is None. I don't know what a bind group is though.

                                    println!("maybe draw indexed can_draw={} index_buf={} bgs={} vbs={}", draw_state.can_draw(), draw_state.index_buffer.is_some(), draw_state.bind_groups.iter().all(|b| b.is_some()), draw_state.vertex_buffers.iter().all(|v| v.is_some()));
                                    for bg in draw_state.bind_groups.iter() {
                                        println!("bg is some {}", bg.is_some());

                                    }
                                    if draw_state.can_draw_indexed() {

logs are

1 visible entities
maybe draw indexed can_draw=false index_buf=true bgs=false vbs=true
bg is some true
bg is some false
bg is some true

MichaelHills · 2020-08-25T13:14:24Z

Oh nevermind, it was because I had left this in the frag shader

# ifdef COLORMATERIAL_TEXTURE
layout(set = 1, binding = 1) uniform texture2D ColorMaterial_texture;
layout(set = 1, binding = 2) uniform sampler ColorMaterial_texture_sampler;
# endif

I removed it and now it looks like it is issuing the draw call. Still no red square. :(

naithar · 2020-08-26T09:09:49Z

I've put some logs into fn load_asset(&self, load_request: &LoadRequest) -> Result<TAsset, AssetLoadError> and that's the result.

loading untyped at /private/var/containers/Bundle/Application/DF750ECE-8232-4398-B02E-F9749CCA6A03/BevyTest.app/icon.png
loading: Handle<Texture>(7841c797-5cf9-401d-b1ce-67245a08cd42)
loading asset: LoadRequest { path: "/private/var/containers/Bundle/Application/DF750ECE-8232-4398-B02E-F9749CCA6A03/BevyTest.app/icon.png", handle_id: HandleId(7841c797-5cf9-401d-b1ce-67245a08cd42), handler_index: 1, version: 0 }
file exists: true
bytes read: [... very big array ...]

So iOS actually seems to be loading assets correctly in the end if full path is given.

I removed it and now it looks like it is issuing the draw call. Still no red square. :(

Maybe iOS handles drawing differently or shaders are not compiled so it fails to draw anything?

Edit:
Seems like shaders are also compiled correctly, so I guess something might be wrong with shader itself or something is not supported on iOS.

MichaelHills · 2020-08-27T11:36:04Z

I finally got my red square working. WIthout the image/texture to determine the sprite size, you have to set the size manually:

        .spawn(SpriteComponents {
            material: materials.add(ColorMaterial {
                color: Color::RED,
                // texture: texture_handle.into(),
            }),
            sprite: Sprite { size: Vec2::new(100.0, 100.0) },
            ..Default::default()
        });

so I think the shaders are fine... but still something going wrong in image or texture loading, not sure what yet.

MichaelHills · 2020-08-27T11:58:28Z

in crates/bevy_render/src/texture/image_texture_loader.rs

        println!("do dyn img");
        let dyn_img = image::load_from_memory_with_format(bytes.as_slice(), img_format)?;
        println!("got dyn img");

It seems to never finish load_from_memory_with_format (at least I don't see the log afterwards), so this explains why the texture loading is not working.

Now the whole "it has a handle" thing is actually because you get a handle back but the png is loaded into a texture on a background thread. Eventually the handle will resolve to a valid texture, but only once it has loaded.

Discovered this initially by adding logs to crates/bevy_sprite/src/sprite.rs

pub fn sprite_system(
    materials: Res<Assets<ColorMaterial>>,
    textures: Res<Assets<Texture>>,
    mut query: Query<(&mut Sprite, &Handle<ColorMaterial>)>,
) {
    println!("sprite system");
    for (mut sprite, handle) in &mut query.iter() {
        let material = materials.get(&handle).unwrap();
        println!("got material with color {} {} {}", material.color.r, material.color.g, material.color.b);
        if let Some(texture_handle) = material.texture {
            println!("got texture handle");
            if let Some(texture) = textures.get(&texture_handle) {
                println!("got texture size {} {}", texture.size.x(), texture.size.y());
                sprite.size = texture.size;
            }
        }
    }
}

and noticed that on iOS it never printed the texture size but on mac it did.

Edit: finally got the error message out from the image loading... Format error decoding Png: CgBI chunk appeared before IHDR chunk

I think somehow the PNG is getting corrupted when added to the iOS project? It's smaller...

ios bytes=12244
mac bytes=15713

MichaelHills · 2020-08-27T12:19:31Z

Well, I found our problem! You need to disable XCode "Remove text metadata from PNG". I disabled everything though just to be sure... I guess the image loading library can't handle whatever XCode does to it.

My phone is now showing a red bevy bird. 👍

cart · 2020-08-27T18:37:55Z

Very exciting! Great work here!

naithar · 2020-08-28T21:41:33Z

Adding std::env::set_current_dir(std::env::current_exe().unwrap().parent().unwrap()); works as workaround for iOS and macOS .app not loading assets at relative path.

MichaelHills · 2020-09-20T17:01:54Z

Success! I managed to play a wav directly with Rodeo. The API has changed a bit in Rodeo/CPAL latest versions compared to where bevy is at, so I don't know if something is broken with that integration now on the newer versions. (Short story is that the cpal stream seems to not be retained and gets dropped from memory and so the mixer weak reference is null.)

Here's what I did

Rebased/squashed @simlay 's coreaudio-sys change on master and backed out the bindgen for the objective-c headers (wasn't needed)
Manually fixed coreaudio-rs compile error, I think that code path is unused though
No changes to rodio
Hacked up cpal coreaudio by using the webaudio host as a "simple" starting point for a single output device, then hacked in a build_output_stream_raw implementation based off https://stackoverflow.com/a/14478420 and the original cpal macos stuff. I completely butchered this file, that's possibly why I had issues with bevy/rodio integration.

This might take a while to clean up the hacks. i.e. no mic input, and no device enumeration atm. There is a bunch of shared code with the macos coreaudio. The existing enumeration is macOS-specific. It might be hard to untangle this stuff.

I guess I'm initially thinking splitting up the cpal code from mod.rs / enumeration.rs to something like:

mod.rs
macos.rs
macos_enumeration.rs
ios.rs
shared.rs

Then use target_os switch to pull in the correct implementation.

MichaelHills · 2020-09-22T13:23:48Z

I applied the audio_output.rs patch from here #310 and now I can finally play audio through Bevy and iOS. :)

MichaelHills · 2020-09-27T14:35:49Z

An update on where things are at:

With @simlay 's An initial xcode setup for using xcode #539 we have an iOS example building out of the box! No local hacks required. No touch and no sound though.
@naithar already landed simulator metal support upstream and it's been released, so simulator works great
I landed a fix in winit so that starting the app in landscape mode the window size works fine (needs new winit release though)
I'm working with @simlay with updates to coreaudio-sys Fixes iOS compiling. RustAudio/coreaudio-sys#33 and coreaudio-rs Added initial changes for iOS support. RustAudio/coreaudio-rs#69 as part of the audio support. Meanwhile I've been reading up on coreaudio docs to understand best to how implement the cpal host properly and productionize my hack.
Implement basic touch support #334 seems to have stalled. I posted my touch support diffs earlier in this thread if you scroll up, that's what I'm currently using. If someone wants to look at that PR and look at my diffs and take over touch support that'd be helpful. 👍
I filed a crash issue over at winit Crash in iOS when swiping up for the control center rust-windowing/winit#1705 but I can't seem to reproduce it on the latest bevy ios example even though I could reproduce it earlier. Once I get to rebasing my game to latest bevy I'll test it again.

Once touch and audio are working, I think we can call it as Bevy having iOS support? There might be bugs here and there and we can fix them as we go.

cart · 2020-09-27T20:04:48Z

#334 seems to have stalled. I posted my touch support diffs earlier in this thread if you scroll up, that's what I'm currently using. If someone wants to look at that PR and look at my diffs and take over touch support that'd be helpful.

Lol I can't find the diffs you're talking about because apparently my brain doesn't work 😄 I think I'll address my own comments in that pr, maybe apply your diff, then merge it. Alternatively i could address my comments and you could create a follow up pr (that way you get proper attribution).

Once touch and audio are working, I think we can call it as Bevy having iOS support? There might be bugs here and there and we can fix them as we go.

Yup that sounds good to me! I'm so happy that this came together so quickly ❤️

MichaelHills · 2020-09-27T21:38:42Z

Lol I can't find the diffs you're talking about because apparently my brain doesn't work 😄 I think I'll address my own comments in that pr, maybe apply your diff, then merge it. Alternatively i could address my comments and you could create a follow up pr (that way you get proper attribution).

I had implemented my own touch support in parallel to that PR because I didn't know about that PR. My comment is buried further up this page #87 (comment) (had to click load hidden items).

In my version I had the following and avoided using Input since it didn't map well. However, in my game code I never used the just_*. I just used active_touches to power my on-screen virtual joysticks and local system state to keep track of which touch id was which joystick. Perhaps the just_* stuff would be useful for UI buttons and things like that.

#[derive(Default)]
pub struct TouchInputState {
    pub active_touches: HashMap<u64, ActiveTouch>,
    pub just_pressed: HashSet<u64>,
    pub just_released: HashSet<u64>,
    pub just_cancelled: HashSet<u64>,
}

I also didn't implement TouchMotion event. I wasn't super clear on what's the correct API. Doing both TouchInputState and TouchMotion event lines up the closest with existing mouse implementation. Naming-wise TouchInputState would be better as TouchInput to line up with Input.

Probably best to just clean up and merge that other one and then I can follow up with anything useful from my diff. e.g. having dx() and dy() helpers was useful for calculating distance moved from when the touch was initiated. Might want to call it delta and return a Vec2 instead or something, not sure.

MichaelHills · 2020-10-06T12:21:40Z

Just an FYI, my personal situation has changed and I'll be strapped for time to continue developing my game / Bevy iOS support for the time being. I intend to finish off the audio support but will hand over a WIP cpal branch if I need to. Happy to keep helping out with testing though!

Once all the other pieces land, I still plan to improve the iOS example project with the following:

add assets file/directory reference to show how to use assets (purely because I did it the wrong way at first which led me down the bad path of Xcode compressing/destroying the PNGs)
add some png texture
add audio background track (and maybe a press a button to play a sound)
some kind of touch support, maybe just adding camera pitch/yaw look around

CleanCut · 2020-10-06T18:20:42Z

@MichaelHills Thank you for all your hard work. I love being around great contributors like you who push projects forward in meaningful ways. I hope you are able to return to your game in the future!

cart · 2020-10-06T19:00:24Z

@MichaelHills I agree with @CleanCut. You've done a bunch of great work already. Don't feel pressured to put in more work than you want to!

dvtkrlbs · 2020-10-07T04:52:45Z

The new version of rodio is now released. I am working on PR. (this helps with audio not working on macOS and ios)

naithar · 2020-10-07T12:10:03Z

I've made some changes to original @MichaelHills touch support diff here: In my fork
I plan to make a PR if #334 wouldn't be moving anywhere if that's okay :)
I'll also run some more test just to be sure that everything is working as intended.

MichaelHills · 2020-10-11T13:02:00Z

Update on audio:

dvtkrlbs · 2020-10-11T13:12:56Z

Another problem is it takes a long time for a new version of rodio to release on crates.io. So even if these issues gets fixed you wont be able use them for a long time (rodio makes new releases for every 4 months or so) so you need to maintain your own bevy-audio (since crates.io crates cant use git dependencies).

MichaelHills · 2020-10-11T13:24:34Z

Another problem is it takes a long time for a new version of rodio to release on crates.io. So even if these issues gets fixed you wont be able use them for a long time (rodio makes new releases for every 4 months or so) so you need to maintain your own bevy-audio (since crates.io crates cant use git dependencies).

I haven't needed to modify Rodio. Does that mean we're ok? Or at worst can just use [patch.crates-io] to change the cpal version if there's a significant version bump.

MichaelHills · 2020-10-17T13:23:44Z

I've made some changes to original @MichaelHills touch support diff here: In my fork
I plan to make a PR if #334 wouldn't be moving anywhere if that's okay :)
I'll also run some more test just to be sure that everything is working as intended.

@naithar Looks like no movement on #334, want to put up yours? Or give some diffs so that #334 can be landed with original attribution, and then put up another PR for enhancements?

naithar · 2020-10-17T14:37:29Z

@MichaelHills yeah, seems like it. I'll rebase and make a PR soon.

naithar · 2020-10-19T14:15:54Z

Well, I found our problem! You need to disable XCode "Remove text metadata from PNG". I disabled everything though just to be sure... I guess the image loading library can't handle whatever XCode does to it.

this shouldn't be required anymore after #693. Listing assets folder in Copy bundle resources is enough for assets to work. Tested with iOS device and macOS .app file.

Also there is no need for a workaround for set_current_dir as PR seems to be solving this too.

francesca64 · 2020-12-02T21:38:11Z

Is there any interest in using cargo-mobile to simplify generating Xcode projects / building / running on device / etc?

cart · 2020-12-02T21:52:54Z

Medium-to-long term I might be interested in adopting higher level abstractions. Short term I'd rather keep it simple, encourage people to become familiar with the "native" mobile tooling, and maintain control over "official" templates. We will likely create an official bevy_template repo in the near future.

alice-i-cecile · 2022-04-26T23:18:23Z

This now works!

Rust-Ninja-Sabi · 2022-09-25T10:02:41Z

Is there any tutorial about Bevy and ios today?

Pipeline override constants will be fully implemented in a followup PR. They're theoretically working, but they need tests and also need a function to map the unmangled names to the mangled names. Otherwise this appears to work. I've tested it with bevy and everything seemed to work. --------- Co-authored-by: robtfm <[email protected]>

karroffel added A-Build-System Related to build systems or continuous integration C-Feature A new feature, making something new possible labels Aug 12, 2020

MichaelHills mentioned this issue Aug 24, 2020

Build fails on osx with /usr/bin/cc is not able to compile a simple test program. rust-lang/cmake-rs#95

Closed

MichaelHills mentioned this issue Aug 24, 2020

iOS: use shaderc-rs for glsl to spirv compilation #324

Merged

naithar mentioned this issue Aug 25, 2020

Assets not loading for compiled binaries #344

Closed

simlay mentioned this issue Sep 21, 2020

An initial xcode setup for using xcode #539

Merged

MichaelHills mentioned this issue Sep 30, 2020

Fix iOS compilation error and add macOS + iOS feedback example applications RustAudio/coreaudio-rs#72

Merged

7 tasks

MichaelHills mentioned this issue Oct 11, 2020

Refactor coreaudio host and add iOS support + ios-feedback example RustAudio/cpal#485

Merged

3 tasks

naithar mentioned this issue Oct 17, 2020

Touch support implementation #696

Merged

2 tasks

memoryruins added the O-iOS Specific to the iOS mobile operating system label Oct 27, 2020

alice-i-cecile closed this as completed Apr 26, 2022

iOS Support #87

iOS Support #87

Comments

cart commented Aug 5, 2020 • edited Loading

MichaelHills commented Aug 21, 2020

MichaelHills commented Aug 22, 2020 • edited Loading

MichaelHills commented Aug 22, 2020 • edited Loading

MichaelHills commented Aug 23, 2020

MichaelHills commented Aug 23, 2020 • edited Loading

cart commented Aug 23, 2020

MichaelHills commented Aug 23, 2020

MichaelHills commented Aug 24, 2020 • edited Loading

naithar commented Aug 24, 2020

MichaelHills commented Aug 24, 2020

MichaelHills commented Aug 24, 2020

naithar commented Aug 24, 2020 • edited Loading

MichaelHills commented Aug 25, 2020

naithar commented Aug 25, 2020

MichaelHills commented Aug 25, 2020 • edited Loading

naithar commented Aug 25, 2020

MichaelHills commented Aug 25, 2020

MichaelHills commented Aug 25, 2020

naithar commented Aug 26, 2020 • edited Loading

MichaelHills commented Aug 27, 2020

MichaelHills commented Aug 27, 2020 • edited Loading

MichaelHills commented Aug 27, 2020

cart commented Aug 27, 2020

naithar commented Aug 28, 2020

MichaelHills commented Sep 20, 2020 • edited Loading

MichaelHills commented Sep 22, 2020

MichaelHills commented Sep 27, 2020 • edited Loading

cart commented Sep 27, 2020

MichaelHills commented Sep 27, 2020

MichaelHills commented Oct 6, 2020

CleanCut commented Oct 6, 2020

cart commented Oct 6, 2020

dvtkrlbs commented Oct 7, 2020

naithar commented Oct 7, 2020 • edited Loading

MichaelHills commented Oct 11, 2020 • edited Loading

dvtkrlbs commented Oct 11, 2020

MichaelHills commented Oct 11, 2020

MichaelHills commented Oct 17, 2020

naithar commented Oct 17, 2020

naithar commented Oct 19, 2020 • edited Loading

francesca64 commented Dec 2, 2020

cart commented Dec 2, 2020

alice-i-cecile commented Apr 26, 2022

Rust-Ninja-Sabi commented Sep 25, 2022

cart commented Aug 5, 2020 •

edited

Loading

MichaelHills commented Aug 22, 2020 •

edited

Loading

MichaelHills commented Aug 22, 2020 •

edited

Loading

MichaelHills commented Aug 23, 2020 •

edited

Loading

MichaelHills commented Aug 24, 2020 •

edited

Loading

naithar commented Aug 24, 2020 •

edited

Loading

MichaelHills commented Aug 25, 2020 •

edited

Loading

naithar commented Aug 26, 2020 •

edited

Loading

MichaelHills commented Aug 27, 2020 •

edited

Loading

MichaelHills commented Sep 20, 2020 •

edited

Loading

MichaelHills commented Sep 27, 2020 •

edited

Loading

naithar commented Oct 7, 2020 •

edited

Loading

MichaelHills commented Oct 11, 2020 •

edited

Loading

naithar commented Oct 19, 2020 •

edited

Loading