A Rust eBPF toolchain.
The redbpf project is a collection of tools and libraries to build eBPF programs using Rust. It includes:
-
redbpf - a user space library that can be used to load eBPF programs or access eBPF maps.
-
redbpf-probes - an idiomatic Rust API to write eBPF programs that can be loaded by the linux kernel
-
redbpf-macros - companion crate to
redbpf-probes
which provides convenient procedural macros useful when writing eBPF programs. For example,#[map]
for defining a map,#[kprobe]
for defining a BPF program that can be attached to kernel functions. -
cargo-bpf - a cargo subcommand for creating, building and debugging eBPF programs
- Allows users to write both BPF programs and userspace programs in Rust
- Offers many BPF map types
HashMap
,PerCpuHashMap
,LruHashMap
,LruPerCpuHashMap
,Array
,PerCpuArray
,PerfMap
,TcHashMap
,StackTrace
,ProgramArray
,SockMap
,DevMap
,RingBuf
- Offers several BPF program types
KProbe
,KRetProbe
,UProbe
,URetProbe
,SocketFilter
,XDP
,StreamParser
,StreamVerdict
,TaskIter
,SkLookup
,Tracepoint
- Provides attribute macros that define various kind of BPF programs and BPF
maps in a declarative way.
#[kprobe]
,#[kretprobe]
,#[uprobe]
,#[uretprobe]
,#[xdp]
,#[tc_action]
,#[socket_filter]
,#[stream_parser]
,#[stream_verdict]
,#[task_iter]
,#[tracepoint]
#[map]
- Can generate Rust bindings from the Linux kernel headers or from the BTF of
vmlinux
- Provides API for both BPF programs and userspace programs to help users write Rust idiomatic code
- Supports BTF for maps
- Supports pinning maps and loading maps from pins
- Supports BPF iterator for
task
- Enables users to write BPF programs for
tc
action and RedBPF compiles the programs into the ELF object file that is compatible withtc
command - Provides wrappers of BPF helper functions
- Offers asynchronous stream of
perf events
for userspace programs - Supports multiple versions of LLVM
- Shows BPF verifier logs when loading BPF programs, BPF maps or BTF fails
- Has several example programs that are separated into two parts: BPF programs and userspace programs
LLVM
is required in your build system to compile BPF bytecode using RedBPF.
-
LLVM 13
It is needed to compile BPF bytecode. -
One of the followings:
- The Linux kernel headers
vmlinux
, the Linux kernel image that contains.BTF
section- Raw BTF data i.e.
/sys/kernel/btf/vmlinux
These are needed to generate Rust bindings of the data structures of the Linux kernel.
Install LLVM 13 and the Linux kernel headers
# apt-get update \
&& apt-get -y install \
wget \
build-essential \
software-properties-common \
lsb-release \
libelf-dev \
linux-headers-generic \
pkg-config \
&& wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 13 && rm -f ./llvm.sh
# llvm-config-13 --version | grep 13
Install LLVM 13 and the Linux kernel headers
# dnf install -y \
clang-13.0.0 \
llvm-13.0.0 \
llvm-libs-13.0.0 \
llvm-devel-13.0.0 \
llvm-static-13.0.0 \
kernel \
kernel-devel \
elfutils-libelf-devel \
make \
pkg-config \
zstd
# llvm-config --version | grep 13
Install LLVM 13 and the Linux kernel headers
# pacman --noconfirm -Syu \
&& pacman -S --noconfirm \
llvm \
llvm-libs \
libffi \
clang \
make \
pkg-config \
linux-headers \
linux
# llvm-config --version | grep -q '^13'
If your Linux distro does not support the latest LLVM as pre-built packages yet, you may build LLVM from the LLVM source code.
$ tar -xaf llvm-13.0.0.src.tar.xz
$ mkdir -p llvm-13.0.0.src/build
$ cd llvm-13.0.0.src/build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/llvm-13-release -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=1
$ cmake --build . --target install
Then you can use your LLVM by specifying the custom installation path when
installing cargo-bpf
or building RedBPF like this:
$ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo install cargo-bpf
$ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo build
Make sure correct -DCMAKE_BUILD_TYPE
is specified. Typically Debug
type is
not recommended if you are not going to debug LLVM itself.
cargo-bpf
is a command line tool for compiling BPF program written in Rust
into BPF bytecode.
$ cargo install cargo-bpf
$ cargo bpf --version
You can learn how to use this from tutorial.
If you want to build RedBPF from source to fix something, you can do as follows:
$ git clone https://github.com/foniod/redbpf.git
$ cd redbpf
$ git submodule sync
$ git submodule update --init
$ cargo build
$ cargo build --examples
The easiest way to get started is reading a basic tutorial.
You can find several examples in this directory. All example
programs are splitted into two parts: example-probes
and
example-userspace
. example-probes
contains BPF programs that execute in
kernel context. example-userspace
includes userspace programs that load BPF
programs into kernel space and communicate with BPF programs through BPF maps.
See also documentation of cargo-bpf
. It provides a
CLI tool for compiling BPF programs easily.
redbpf-tools is a
cargo-bpf
generated crate that includes simple examples you can use to
understand how to structure your programs.
Finally, check the foniod project that includes more advanced, concrete production ready examples of redbpf programs.
rustc
is linked to its own bundled version of LLVM. And cargo-bpf
also uses
its own version of LLVM that is statically linked into cargo-bpf
itself. But
note that users can control the LLVM version of cargo-bpf
by providing other
versions of LLVM in their system when building cargo-bpf
.
Why do we care about two LLVM versions?
Because both two versions of LLVMs are all participating in the process of
compiling BPF programs.
- RedBPF executes
rustc
to compile BPF programs. Andrustc
calls LLVM functions to emit LLVM bitcode. - And then RedBPF parses the emitted LLVM bitcode to convert it into BPF
bytecode. To do so, it calls LLVM functions that are statically linked into
cargo-bpf
.
What happens if LLVM of rustc
is newer than the LLVM of cargo-bpf
? You
already feel it. BAM! Typically older version of LLVM can not properly handle
the bitcode that is generated by newer version of LLVM. i.e., cargo-bpf
with
older LLVM can not properly handle what rustc
with newer LLVM emits.
What happens if LLVM of rustc
is older than the LLVM of cargo-bpf
? Normally
LLVM is likely to support backward compatibility for intermediate
representation.
Let's put things together.
There are two LLVM versions involved in compiling BPF programs:
- the version of LLVM**(1)** that
cargo-bpf
is statically linked to whencargo-bpf
is built. - the version of LLVM**(2)** that
rustc
is linked to.
And, (1) should be greater than or equal to (2).
It is the best case if (1) == (2)
but (1) > (2)
is also okay.
Rust version | LLVM version of the rustc | Valid LLVM version of system |
---|---|---|
1.56 ~ | LLVM 13 | LLVM 13 and newer |
You can refer to various Dockerfile
s that contain minimal necessary packages
to build RedBPF
properly: Dockerfiles for
RedBPF
These docker images are pushed to ghcr.io:
x86_64
ghcr.io/foniod/redbpf-build:latest-x86_64-ubuntu21.04
ghcr.io/foniod/redbpf-build:latest-x86_64-fedora35
ghcr.io/foniod/redbpf-build:latest-x86_64-alpine3.15
ghcr.io/foniod/redbpf-build:latest-x86_64-debian11
ghcr.io/foniod/redbpf-build:latest-x86_64-archlinux
ARM64
ghcr.io/foniod/redbpf-build:latest-aarch64-ubuntu21.04
ghcr.io/foniod/redbpf-build:latest-aarch64-fedora35
ghcr.io/foniod/redbpf-build:latest-aarch64-alpine3.15
ghcr.io/foniod/redbpf-build:latest-aarch64-debian11
See build-test.yml for more information. It describes build tests of RedBPF that run inside docker containers.
If you want docker images that are prepared to build foniod
then refer to
this: Dockerfiles for foniod
You need to specify KERNEL_SOURCE
or KERNEL_VERSION
environment variables
that indicate kernel headers. The headers should be found inside the
container. For example, inside the Ubuntu 21.04 container that contains the
Linux 5.11.0-25-generic
kernel headers, you should specify KERNEL_VERSION
environment variable as follows:
# KERNEL_VERSION=5.11.0-25-generic cargo build --examples
If your container has vmlinux
, the Linux kernel image that contains .BTF
section in it, you can specify it instead of the Linux kernel headers.
# REDBPF_VMLINUX=/boot/vmlinux cargo build --examples
See build-test.yml for more information. It describes build tests of RedBPF that run inside docker containers.
Currently, x86-64
and aarch64
architectures are supported.
This repository contains code from other software in the following directories, licensed under their own particular licenses:
bpf-sys/libbpf
: LGPL2 + BSD-2
Where '+' means they are dual licensed.
RedBPF and its components, unless otherwise stated, are licensed under either of
- Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.
This project is for everyone. We ask that our users and contributors take a few minutes to review our code of conduct.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
For further advice on getting started, please consult the Contributor's Guide. Please note that all contributions MUST contain a Developer Certificate of Origin sign-off line.