Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flutter engine building #49

Open
limbouser opened this issue Mar 24, 2020 · 19 comments
Open

Flutter engine building #49

limbouser opened this issue Mar 24, 2020 · 19 comments

Comments

@limbouser
Copy link

I can give you my cross compilation script though, link here. I compile in WSL because compiling on-target takes ages. This needs a cloned repo of GNU binutils, LLVM 9 and LLVM 10.

I have been trying to build the flutter engine for a long time now and followed the medium article too. Nothing worked. I would like to know why you have used two versions of LLVM(9 and 10). Since it's said to use TOT Clang/LLVM, I used v11 and always get stuck at building libcxx.

From what I understood from your script, you use LLVM v9.x for setting up the toolchain, ie, Clang/LLVM and then you setup the binutils. You use LLVM v10.x to build the libcxx and libcxxabi. After that comes the setting up the engine development environment and all. Is my understanding of your script right? I use a Ubuntu-16.04 64-bit to build all the packages.

I remember you sayingt that you wouldn't provide any help towards engine building but I've been stuck at this point for a while and I assure you I wouldn't bug you a lot on this. :)

@hhk7734
Copy link

hhk7734 commented Mar 24, 2020

Ref: https://docs.loliot.net/books/flutter/page/flutter-engine-for-linux-arm64

@ardera
Copy link
Owner

ardera commented Mar 24, 2020

TOT Clang/LLVM

Yeah, that's what it says in the article by Chinmay Garde, but it's actually not that accurate.

The flutter team uses tip-of-tree of their own fork of Clang/LLVM, which is not even stable Clang/LLVM 10. If you try to build the engine with clang-10/11, a lot of warnings are issued (these warnings didn't exist in clang-9, they were newly introduced to clang-10/11) and because the engine build is configured to abort on a warning (-Werror is given to clang) the build will fail. That's why I use clang-9.

For libcxx/libcxxabi I just use latest or latest stable clang. Using clang-9 would probably be fine though too.

The instructions posted by @hhk7734 may be helpful too.

I remember you sayingt that you wouldn't provide any help towards engine building but I've been stuck at this point for a while and I assure you I wouldn't bug you a lot on this.

Yeah we'll see 😄

@limbouser
Copy link
Author

@hhk7734 Thanks for posting this. Even though this is for a different platform I tried adjusting the flags for ARM platform and here too, I had an error at libcxx building. I have mentioned it in the comments section of your page.

@limbouser
Copy link
Author

@ardera Did you have to make any changes to the CMakeFiles.txt files of libcxxabi, like adding a rpath or something? Whichever manual I use, I'm always stuck at the same place.

/home/te_user/flutter-engine/sdk/toolchain/bin/clang++ --sysroot=/home/te_user/flutter-engine/sdk/sysroot CMakeFiles/cmTC_ea0ef.dir/testCXXCompiler.cxx.o -o cmTC_ea0ef && : /home/te_user/flutter-engine/sdk/toolchain/bin/arm-linux-gnueabihf-ld: warning: libm.so.6, needed by /home/te_user/flutter-engine/sdk/sysroot/usr/lib/gcc/arm-linux-gnueabihf/8/libstdc++.so, not found (try using -rpath or -rpath-link)

How do I add an rpath or rpath-link?

@ardera
Copy link
Owner

ardera commented Mar 25, 2020

I remember having that error too, IIRC the cause was some dangling symlinks in my sysroot

(i.e. some of the symlinks on the Raspberry Pi inside /lib, /usr are absolute, not relative, and thus become dangling when you copy them to your host)

I think I fixed it using the symlinks utility. Execute these on the Raspberry Pi:

sudo apt install symlinks
sudo symlinks -c /lib /usr /opt

You can add the -t flag to the symlinks invocation to dry-run it, i.e. see the changes it would make without applying them.

After that's done, copy the sysroot to your host machine again

EDIT: changed /lib/usr to /usr

@limbouser
Copy link
Author

/home/te_user/flutter-engine/sdk/toolchain/bin/clang++ --sysroot=/home/te_user/flutter-engine/sdk/sysroot CMakeFiles/cmTC_ea0ef.dir/testCXXCompiler.cxx.o -o cmTC_ea0ef && : /home/te_user/flutter-engine/sdk/toolchain/bin/arm-linux-gnueabihf-ld: warning: libm.so.6, needed by /home/te_user/flutter-engine/sdk/sysroot/usr/lib/gcc/arm-linux-gnueabihf/8/libstdc++.so, not found (try using -rpath or -rpath-link)

@ardera This error remains even after trying out symlinks utility. 😶
I used the symlinks utility, the rsynced them to my build machine.

@hhk7734 I also tried your method of avoiding the dynamic link issues, ended up having the same error.

@ardera
Copy link
Owner

ardera commented Mar 30, 2020

try this:

  • install the libc6 libc6-dev packages on your pi
  • run the symlinks utility again
  • rsync the files to your build machine again
  • try to build the engine again

if this doesn't fix the issue, try adding the following flags to your cmake invocation:

-DCMAKE_C_FLAGS=-L/home/te_user/flutter-engine/sdk/sysroot/lib/arm-linux-gnueabihf/ -DCMAKE_CXX_FLAGS=-L/home/te_user/flutter-engine/sdk/sysroot/lib/arm-linux-gnueabihf/

@hhk7734
Copy link

hhk7734 commented Apr 1, 2020

@limbouser
Modified the documentation while building it on a new board. Would you like to repeat it again with the link below??
https://docs.loliot.net/link/152#bkmrk-sysroot

Mabey it is the same way as @ardera 's comment because build-essential depends on libc6-dev Etc.

@limbouser
Copy link
Author

 
 ./flutter/tools/gn \
     --clang \
     --target-sysroot $HOME/flutter-engine/sdk/sysroot-lolit \
     --target-toolchain $HOME/flutter-engine/sdk/toolchain \
     --target-triple arm-linux-gnueabihf  \
     --linux-cpu arm \
     --runtime-mode debug \
     --embedder-for-target \
     --no-lto \
     --target-os linux \
     --arm-float-abi hard

I used the flags I showed above and when I tried ninja as the next step, got the following error. Do you have any experience dealing with this? @hhk7734 @ardera

 ninja -C out/linux_debug_arm/
ninja: Entering directory `out/linux_debug_arm/'
[1/3730] CC obj/third_party/boringssl/src/crypto/asn1/crypto.a_bitstr.o
FAILED: obj/third_party/boringssl/src/crypto/asn1/crypto.a_bitstr.o
/home/te_user/flutter-engine/sdk/toolchain/bin/clang -MD -MF obj/third_party/boringssl/src/crypto/asn1/crypto.a_bitstr.o.d --target=arm-linux-gnueabihf --sysroot /home/te_user/flutter-engine/sdk/sysroot-lolit -DUSE_OPENSSL=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_LIBCPP_DISABLE_AVAILABILITY=1 -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DBORINGSSL_ALLOW_CXX_RUNTIME -DBORINGSSL_IMPLEMENTATION -DBORINGSSL_NO_STATIC_INITIALIZER -DOPENSSL_SMALL -D_XOPEN_SOURCE=700 -I../.. -Igen -I../../third_party/libcxx/include -I../../third_party/libcxxabi/include -I../../third_party/libcxxabi/include -I../../third_party/boringssl/src/include -fno-strict-aliasing -fstack-protector --param=ssp-buffer-size=4 -march=armv7-a -mfloat-abi=hard -mtune=generic-armv7-a -mthumb -fPIC -pipe -pthread -fcolor-diagnostics -mfpu=neon -fvisibility=hidden --sysroot=/home/te_user/flutter-engine/sdk/sysroot-lolit -Wstring-conversion -Wnewline-eof -O2 -fno-ident -fdata-sections -ffunction-sections -g0 -Wendif-labels -Werror -Wno-missing-field-initializers -Wno-unused-parameter -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-non-c-typedef-for-linkage -Wno-deprecated-copy -Wno-range-loop-construct -std=c11 -c ../../third_party/boringssl/src/crypto/asn1/a_bitstr.c -o obj/third_party/boringssl/src/crypto/asn1/crypto.a_bitstr.o
error: unknown warning option '-Wno-implicit-int-float-conversion'; did you mean '-Wno-implicit-float-conversion'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-c99-designator'; did you mean '-Wno-gnu-designator'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-non-c-typedef-for-linkage'; did you mean '-Wno-return-type-c-linkage'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-deprecated-copy'; did you mean '-Wno-deprecated'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-range-loop-construct'; did you mean '-Wno-range-loop-analysis'? [-Werror,-Wunknown-warning-option]
[2/3730] CXX obj/flutter/runtime/test_font.test_font_data.o
FAILED: obj/flutter/runtime/test_font.test_font_data.o
/home/te_user/flutter-engine/sdk/toolchain/bin/clang++ -MD -MF obj/flutter/runtime/test_font.test_font_data.o.d --target=arm-linux-gnueabihf --sysroot /home/te_user/flutter-engine/sdk/sysroot-lolit -DEMBED_TEST_FONT_DATA=1 -DUSE_OPENSSL=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_LIBCPP_DISABLE_AVAILABILITY=1 -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DSK_GL -DSK_CODEC_DECODES_JPEG -DSK_ENCODE_JPEG -DSK_CODEC_DECODES_PNG -DSK_ENCODE_PNG -DSK_CODEC_DECODES_WEBP -DSK_ENCODE_WEBP -DSK_HAS_WUFFS_LIBRARY -DSK_R32_SHIFT=16 -DSK_DISABLE_REDUCE_OPLIST_SPLITTING -DSK_ENABLE_DUMP_GPU -DSK_DISABLE_AAA -DSK_DISABLE_READBUFFER -DSK_DISABLE_EFFECT_DESERIALIZATION -DSK_DISABLE_LEGACY_SHADERCONTEXT -DSK_DISABLE_LOWP_RASTER_PIPELINE -DSK_FORCE_RASTER_PIPELINE_BLITTER -DSK_GL -I../.. -Igen -I../../third_party/libcxx/include -I../../third_party/libcxxabi/include -I../../third_party/libcxxabi/include -I../.. -I../../third_party/skia -fno-strict-aliasing -fstack-protector --param=ssp-buffer-size=4 -march=armv7-a -mfloat-abi=hard -mtune=generic-armv7-a -mthumb -fPIC -pipe -pthread -fcolor-diagnostics -mfpu=neon -Wall -Wextra -Wendif-labels -Werror -Wno-missing-field-initializers -Wno-unused-parameter -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-non-c-typedef-for-linkage -Wno-deprecated-copy -Wno-range-loop-construct -fvisibility=hidden --sysroot=/home/te_user/flutter-engine/sdk/sysroot-lolit -Wstring-conversion -Wnewline-eof -O2 -fno-ident -fdata-sections -ffunction-sections -g0 -fvisibility-inlines-hidden -std=c++17 -fno-rtti -nostdinc++ -nostdinc++ -fno-exceptions -c ../../flutter/runtime/test_font_data.cc -o obj/flutter/runtime/test_font.test_font_data.o
error: unknown warning option '-Wno-implicit-int-float-conversion'; did you mean '-Wno-implicit-float-conversion'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-c99-designator'; did you mean '-Wno-gnu-designator'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-non-c-typedef-for-linkage'; did you mean '-Wno-return-type-c-linkage'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-deprecated-copy'; did you mean '-Wno-deprecated'? [-Werror,-Wunknown-warning-option]
error: unknown warning option '-Wno-range-loop-construct'; did you mean '-Wno-range-loop-analysis'? [-Werror,-Wunknown-warning-option]
[10/3730] ASM obj/third_party/boringssl/linux-arm/crypto/fipsmodule/boringssl_asm.bsaes-armv7.o
ninja: build stopped: subcommand failed.

@ardera
Copy link
Owner

ardera commented Apr 3, 2020

This means that your LLVM/Clang is too old for your engine sources / your engine sources are too new for your LLVM/Clang.

This is because, when you follow the steps by @hhk7734 or Chinmay Garde's blogpost, you're setting up a development environment for the flutter engine, so you're basically downloading the most recent sources in existance.

This is the correct way to do it if you want to build flutter master, and it seems like you'll have to use LLVM/Clang-10 instead if you really want to do it this way.

If you instead want to build flutter stable, you'll need some older sources. (which will compile fine with LLVM/Clang-9) Go into your engine directory and run:

gclient sync --revision=https://github.com/flutter/engine.git@<engine commit hash here>

where <engine commit hash here> is the most recent stable engine commit, which you can get from this file in the flutter SDK repo.

If you encounter issues while running the gclient sync --revision=... command, just delete your engine folder, follow the steps by @hhk7734 to initialize it again, and then, instead of running the initial gclient sync without arguments as it says in the instructions, you run gclient sync --revision=... as written above. (gclient sync doesn't seem to work that well for downgrading the sources)

@limbouser
Copy link
Author

Thanks @ardera. I'll check. So are the engine-binaries you use based upon this stable branch?

@ardera
Copy link
Owner

ardera commented Apr 3, 2020

So are the engine-binaries you use based upon this stable branch?

yep

@hhk7734
Copy link

hhk7734 commented Apr 14, 2020

https://github.com/flutter/engine/commits/master/shell/platform/linux/config/BUILD.gn
Added host-sidetarget-side compilation dependency

@ardera
Copy link
Owner

ardera commented May 9, 2020

@hhk7734 Yep, just had some issues with that when I was building the engine.

In case someone finds this useful: You'll probably encounter some errors while running the gn ... command for stable engine version 1.17. This is the because it uses pkg-config to locate the gtk+ headers. Since Raspbian stores its pkg-config files in a somewhat non-standard location, you need to apply some hacks.

On your host, after every time you sync the sysroot, delete the /usr/lib/pkgconfig directory (it's empty) and create a symlink /usr/lib/pkgconfig ==> /usr/lib/arm-linux-gnueabihf/pkgconfig.

Should look something like the following in practice:

rsync ... ~/my_pi_sysroot
rm -r ~/my_pi_sysroot/usr/lib/pkgconfig
ln --symbolic --relative ~/my_pi_sysroot/usr/lib/arm-linux-gnueabihf/pkgconfig ~/my_pi_sysroot/usr/lib/pkgconfig

There is an option for setting the path in which the flutter's pkg-config will look for the package config files, but it's not that easy to set. To be more precise, it must be specified in the args.gn file, which means you can't use flutter/tools/gn tool for easy build configuration. You'd have to manually configure the engine yourself.

@hhk7734
Copy link

hhk7734 commented May 10, 2020

engine/src/build/config/linux/pkg-config.py

def SetConfigPath(options):
  """Set the PKG_CONFIG_LIBDIR environment variable.

  This takes into account any sysroot and architecture specification from the
  options on the given command line.
  """

  sysroot = options.sysroot
  assert sysroot

  # Compute the library path name based on the architecture.
  arch = options.arch
  if sysroot and not arch:
    print("You must specify an architecture via -a if using a sysroot.")
    sys.exit(1)

  libdir = sysroot + '/usr/' + options.system_libdir + '/pkgconfig'
  libdir += ':' + sysroot + '/usr/share/pkgconfig'
  os.environ['PKG_CONFIG_LIBDIR'] = libdir
  return libdir

Maybe, I fixed the code above when the error occurred.

PS. https://youtu.be/p6bzmdAJqjo
Thanks to flutter-pi, I was able to shoot this video. 👍

@ardera
Copy link
Owner

ardera commented May 10, 2020

Maybe, I fixed the code above when the error occurred.

Yeah, that works too. Ideally, of course, one would specify a custom system_libdir when invoking gn. Or even better, have pkg-config.py calculate a working one out of the box

PS. https://youtu.be/p6bzmdAJqjo
Thanks to flutter-pi, I was able to shoot this video. 👍

Looks very nice! It's cool to see flutter become a real alternative for embedded UIs. It's just so much better than Qt, electron or CEF in my opinion.

@jwinarske
Copy link
Contributor

@limbouser I added support to build the engine, flutter-pi and Deb packages. By default it builds the stable channel, and is tuned for the RPI3. In case below I mounted the RPI3 SD card on host.

git clone https://github.com/jwinarske/flutter_embedded
cd flutter_embedded
mkdir build && cd build
cmake .. -DBUILD_PLATFORM_SYSROOT=OFF -DTARGET_SYSROOT=/media/joel/rootfs -DBUILD_FLUTTER_PI=ON -DBUILD_GLFW_FLUTTER=OFF
make package -j8

@jwinarske
Copy link
Contributor

@spokV
Copy link

spokV commented Feb 20, 2022

@limbouser I added support to build the engine, flutter-pi and Deb packages. By default it builds the stable channel, and is tuned for the RPI3. In case below I mounted the RPI3 SD card on host.

git clone https://github.com/jwinarske/flutter_embedded
cd flutter_embedded
mkdir build && cd build
cmake .. -DBUILD_PLATFORM_SYSROOT=OFF -DTARGET_SYSROOT=/media/joel/rootfs -DBUILD_FLUTTER_PI=ON -DBUILD_GLFW_FLUTTER=OFF
make package -j8

Hi @jwinarske, can you give the cmake command line for building a cm4 arm64 raspbian?

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants