Add extension rdkafka support

[crazywhalecc]

Linux support is not available yet: confluentinc/librdkafka#4495

[mihaileu]

how can we move further here, we need this extension for our setup ?!

[crazywhalecc]

@mihaileu I can't do anything yet, unless someone can provide a patch for the librdkafka library, or the discussion started above is answered.

[daddeffe]

@crazywhalecc Hi, there has been the latest 2.4.0 release of libkafka with a fix for static builds. I don't know if the fix is whole or related but there is. Sorry for not testing myself but I never worked on c projects at all.. (my lack).

[crazywhalecc]

@crazywhalecc Hi, there has been the latest 2.4.0 release of libkafka with a fix for static builds. I don't know if the fix is whole or related but there is. Sorry for not testing myself but I never worked on c projects at all.. (my lack).

@daddeffe Unfortunately it still failed to build statically on Linux. I think I need to open an issue about it (discussion may not be noticed)

[daddeffe]

Hi, I dont want to create entropy, but I managed to install it on my machine statically
Hoping that help here the diff from the configure from static-php and my native build

[crazywhalecc]

@daddeffe Thanks for your work! It would be even better if it can be verified to work on Alpine Linux (for testing compatibility with musl-libc).

[daddeffe]

./bin/spc-alpine-docker build "rdkafka" --build-cli  --build-micro
     _        _   _                 _
 ___| |_ __ _| |_(_) ___      _ __ | |__  _ __
/ __| __/ _` | __| |/ __|____| '_ \| '_ \| '_ \
\__ \ || (_| | |_| | (_|_____| |_) | | | | |_) |
|___/\__\__,_|\__|_|\___|    | .__/|_| |_| .__/   v2.3.4
                             |_|         |_|
[16:22:45] [INFO] Build OS:       Linux (x86_64)
[16:22:45] [INFO] Build SAPI:     cli, micro
[16:22:45] [INFO] Extensions (1): rdkafka
[16:22:45] [INFO] Libraries (3):  zlib,openssl,librdkafka
[16:22:45] [INFO] Strip Binaries: yes
[16:22:45] [INFO] Enable ZTS:     no
[16:22:45] [INFO] PHP Version:    8.2.23
[16:22:45] [NOTI] Build will start after 2s ...
[16:22:47] [NOTI] lib [pkg-config] already built
[16:22:47] [NOTI] lib [zlib] already built
[16:22:47] [NOTI] lib [openssl] already built
[16:22:47] [NOTI] lib [librdkafka] already built
[16:22:47] [INFO] Entering dir: /app/source/php-src
[16:22:47] [INFO] [EXEC] ./buildconf --force
[16:22:51] [INFO] Entering dir: /app/source/php-src
[16:22:51] [INFO] [EXEC] pkg-config --libs --static rdkafka
[16:22:51] [INFO] rdkafka is using --with-rdkafka=/app/buildroot LIBS="-L/app/buildroot/lib/pkgconfig/../../lib -lrdkafka -lm -ldl -lpthread -lrt -lpthread -lrt"
[16:22:51] [INFO] [EXEC] pkg-config --libs --static rdkafka
[16:22:51] [INFO] [EXEC] LD_LIBRARY_PATH=/usr/local/musl/x86_64-linux-musl/lib ./configure --prefix= --with-valgrind=no --enable-shared=no --enable-static=yes --disable-all --disable-cgi --disable-phpdbg --enable-cli --disable-fpm --disable-embed --enable-micro=all-static --with-rdkafka=/app/buildroot LIBS="-L/app/buildroot/lib/pkgconfig/../../lib -lrdkafka -lm -ldl -lpthread -lrt -lpthread -lrt" CFLAGS='' CPPFLAGS='-I/app/buildroot/include' LDFLAGS='-L/app/buildroot/lib' LIBS='-ldl -lpthread -lm'
[16:23:01] [INFO] Extension [rdkafka] patched before make
[16:23:01] [INFO] cleaning up
[16:23:01] [INFO] Entering dir: /app/source/php-src
[16:23:01] [INFO] [EXEC] make clean
[16:23:02] [INFO] building cli
[16:23:02] [INFO] Entering dir: /app/source/php-src
[16:23:02] [INFO] [EXEC] sed -i "s|//lib|/lib|g" Makefile
[16:23:02] [INFO] [EXEC] $SPC_CMD_PREFIX_PHP_MAKE EXTRA_CFLAGS='-g -Os -fno-ident -fPIE' EXTRA_LIBS='/app/buildroot/lib/librdkafka.a /app/buildroot/lib/librdkafka++.a /app/buildroot/lib/librdkafka-static.a /app/buildroot/lib/libssl.a /app/buildroot/lib/libcrypto.a /app/buildroot/lib/libz.a -lstdc++  ' EXTRA_LDFLAGS_PROGRAM='-all-static' cli
[16:23:40] [INFO] Entering dir: /app/source/php-src/sapi/cli
[16:23:40] [INFO] [EXEC] strip --strip-all php
[16:23:40] [INFO] Deploying cli file
[16:23:40] [INFO] [EXEC] cp '/app/source/php-src/sapi/cli/php' '/app/buildroot/bin/'
[16:23:40] [INFO] building micro
[16:23:40] [INFO] Entering dir: /app/source/php-src
[16:23:40] [INFO] [EXEC] sed -i "s|//lib|/lib|g" Makefile
[16:23:40] [INFO] [EXEC] $SPC_CMD_PREFIX_PHP_MAKE EXTRA_CFLAGS='-g -Os -fno-ident -fPIE' EXTRA_LIBS='/app/buildroot/lib/librdkafka.a /app/buildroot/lib/librdkafka++.a /app/buildroot/lib/librdkafka-static.a /app/buildroot/lib/libssl.a /app/buildroot/lib/libcrypto.a /app/buildroot/lib/libz.a -lstdc++  ' EXTRA_LDFLAGS_PROGRAM='-all-static' micro
[16:23:41] [INFO] [EXEC] strip --strip-all /app/source/php-src/sapi/micro/micro.sfx
[16:23:41] [INFO] Deploying micro file
[16:23:41] [INFO] [EXEC] cp '/app/source/php-src/sapi/micro/micro.sfx' '/app/buildroot/bin/'
[16:23:41] [INFO] running cli sanity check
[16:23:41] [INFO] [EXEC] /app/buildroot/bin/php -r "echo \"hello\";"
[16:23:41] [INFO] [EXEC] /app/source/micro_ext_test.exe
[16:23:41] [INFO] [EXEC] /app/source/micro_zend_bug_test.exe
[16:23:41] [INFO] Build complete, used 56.35 s !
[16:23:41] [INFO] Static php binary path (host system): /home/df/work/static-php-cli/app/buildroot/bin/php
[16:23:41] [INFO] phpmicro binary path (host system): /home/df/work/static-php-cli/app/buildroot/bin/micro.sfx
[16:23:41] [INFO] License path (host system): /home/df/work/static-php-cli/app/buildroot/license/

@crazywhalecc Is this a success?? Built on alpine, merged main, added just --disable-ssl --disable-gssapi --disable-zlib to the src/SPC/builder/unix/library/librdkafka.php file.
I will try to test it asap but I'm a little overwhelmed this days

[crazywhalecc]

@daddeffe --disable-ssl --disable-gssapi --disable-zlib: Can't we choose to enable zlib and openssl?

[mensfeld]

I was able to compile the .so to have only linux basic stuff + sasl2 as deps:

[ext (main)]$ ldd librdkafka.so 
	linux-vdso.so.1 (0x00007ffd644f9000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fefb1c68000)
	libsasl2.so.2 => /lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007fefb1c4d000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fefb1a24000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fefb24dd000)

wouldn't' this be sufficient for distribution?

[mensfeld]

features as follows:

(builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer, STATIC_LINKING GCC GXX PKGCONFIG INSTALL GNULD LDS C11THREADS LIBDL PLUGINS ZLIB SSL SASL_CYRUS ZSTD HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0xfffff)

[mensfeld]

@crazywhalecc I would still like your answer so I can move with similar work in https://github.com/karafka/rdkafka-ruby 🙏

[crazywhalecc]

@mensfeld Seems you are linking shared libraries. static-php is using musl-libc and all other libraries that can be statically linked, I don't know if your goal is to compile a .so or executable file, but if it is the final static compilation, you need to add all the required libxxx.a files.