-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
tests: integrate fuzz-dwfl-core into the test suite
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
- Loading branch information
Showing
15 changed files
with
388 additions
and
7 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,6 +66,7 @@ | |
| /find-prologues | ||
| /funcretval | ||
| /funcscopes | ||
| /fuzz-dwfl-core | ||
| /get-aranges | ||
| /get-files | ||
| /get-lines | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| #!/bin/bash -eu | ||
|
|
||
| # This script is supposed to be compatible with OSS-Fuzz, i.e. it has to use | ||
| # environment variables like $CC, $CFLAGS and $OUT, link the fuzz targets with CXX | ||
| # (even though the project is written in C) and so on: | ||
| # https://google.github.io/oss-fuzz/getting-started/new-project-guide/#buildsh | ||
|
|
||
| # The fuzz targets it builds can't make any assumptions about | ||
| # their runtime environment apart from /tmp being writable: | ||
| # https://google.github.io/oss-fuzz/further-reading/fuzzer-environment/ . | ||
| # Even though it says there that it's possible to link fuzz targets against | ||
| # their dependencies dynamically by moving them to $OUT and changing | ||
| # rpath, it tends to break coverage reports from time to time https://github.com/google/oss-fuzz/issues/6524 | ||
| # so all the dependencies are linked statically here. | ||
|
|
||
| # This script is configured via https://github.com/google/oss-fuzz/blob/master/projects/elfutils/project.yaml | ||
| # and used to build the elfutils project on OSS-Fuzz with three fuzzing engines | ||
| # (libFuzzer, AFL++ and honggfuzz) on two architectures (x86_64 and i386) | ||
| # with three sanitizers (ASan, UBSan and MSan) with coverage reports on top of | ||
| # all that: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_elfutils/latest | ||
| # so before changing anything ideally it should be tested with the OSS-Fuzz toolchain | ||
| # described at https://google.github.io/oss-fuzz/advanced-topics/reproducing/#building-using-docker | ||
| # by running something like: | ||
| # | ||
| # ./infra/helper.py pull_images | ||
| # ./infra/helper.py build_image --no-pull elfutils | ||
| # for sanitizer in address undefined memory; do | ||
| # for engine in libfuzzer afl honggfuzz; do | ||
| # ./infra/helper.py build_fuzzers --clean --sanitizer=$sanitizer --engine=$engine elfutils PATH/TO/ELFUTILS | ||
| # ./infra/helper.py check_build --sanitizer=$sanitizer --engine=$engine -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 elfutils | ||
| # done | ||
| # done | ||
| # | ||
| # ./infra/helper.py build_fuzzers --clean --architecture=i386 elfutils PATH/TO/ELFUTILS | ||
| # ./infra/helper.py check_build --architecture=i386 -e ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 elfutils | ||
| # | ||
| # ./infra/helper.py build_fuzzers --clean --sanitizer=coverage elfutils PATH/TO/ELFUTILS | ||
| # ./infra/helper.py coverage --no-corpus-download --fuzz-target=fuzz-dwfl-core --corpus-dir=PATH/TO/ELFUTILS/tests/fuzz-dwfl-core-crashes/ elfutils | ||
| # | ||
| # It should be possible to eventually automate that with ClusterFuzzLite https://google.github.io/clusterfuzzlite/ | ||
| # but it doesn't seem to be compatible with buildbot currently. | ||
|
|
||
| # The script can also be used to build and run the fuzz target locally without Docker. | ||
| # After installing clang and the build dependencies of libelf by running something | ||
| # like `dnf build-dep elfutils-devel` on Fedora or `apt-get build-dep libelf-dev` | ||
| # on Debian/Ubuntu, the following commands should be run: | ||
| # | ||
| # $ ./tests/build-fuzzers.sh | ||
| # $ ./out/fuzz-dwfl-core tests/fuzz-dwfl-core-crashes/ | ||
|
|
||
| set -eux | ||
|
|
||
| cd "$(dirname -- "$0")/.." | ||
|
|
||
| SANITIZER=${SANITIZER:-address} | ||
| flags="-O1 -fno-omit-frame-pointer -g -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=$SANITIZER -fsanitize=fuzzer-no-link" | ||
|
|
||
| export CC=${CC:-clang} | ||
| export CFLAGS=${CFLAGS:-$flags} | ||
|
|
||
| export CXX=${CXX:-clang++} | ||
| export CXXFLAGS=${CXXFLAGS:-$flags} | ||
|
|
||
| export OUT=${OUT:-"$(pwd)/out"} | ||
| mkdir -p "$OUT" | ||
|
|
||
| export LIB_FUZZING_ENGINE=${LIB_FUZZING_ENGINE:--fsanitize=fuzzer} | ||
|
|
||
| make clean || true | ||
|
|
||
| # ASan isn't compatible with -Wl,--no-undefined: https://github.com/google/sanitizers/issues/380 | ||
| find -name Makefile.am | xargs sed -i 's/,--no-undefined//' | ||
|
|
||
| # ASan isn't compatible with -Wl,-z,defs either: | ||
| # https://clang.llvm.org/docs/AddressSanitizer.html#usage | ||
| sed -i 's/^\(ZDEFS_LDFLAGS=\).*/\1/' configure.ac | ||
|
|
||
| autoreconf -i -f | ||
| if ! ./configure --enable-maintainer-mode --disable-debuginfod --disable-libdebuginfod \ | ||
| --without-bzlib --without-lzma --without-zstd \ | ||
| CC="$CC" CFLAGS="-Wno-error $CFLAGS" CXX="-Wno-error $CXX" CXXFLAGS="$CXXFLAGS" LDFLAGS="$CFLAGS"; then | ||
| cat config.log | ||
| exit 1 | ||
| fi | ||
|
|
||
| ASAN_OPTIONS=detect_leaks=0 make -j$(nproc) V=1 | ||
|
|
||
| $CC $CFLAGS \ | ||
| -D_GNU_SOURCE -DHAVE_CONFIG_H \ | ||
| -I. -I./lib -I./libelf -I./libebl -I./libdw -I./libdwelf -I./libdwfl -I./libasm \ | ||
| -c tests/fuzz-dwfl-core.c -o fuzz-dwfl-core.o | ||
| $CXX $CXXFLAGS $LIB_FUZZING_ENGINE fuzz-dwfl-core.o \ | ||
| ./libdw/libdw.a ./libelf/libelf.a -l:libz.a \ | ||
| -o "$OUT/fuzz-dwfl-core" | ||
| zip -r -j "$OUT/fuzz-dwfl-core_seed_corpus.zip" tests/fuzz-dwfl-core-crashes |
Binary file not shown.
Empty file.
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| #include <assert.h> | ||
| #include <config.h> | ||
| #include <stdlib.h> | ||
| #include ELFUTILS_HEADER(dwfl) | ||
| #include "fuzz.h" | ||
| #include "system.h" | ||
|
|
||
| /* This fuzz target was initially used to fuzz systemd and | ||
| there elfutils is hidden behind functions receiving file | ||
| names and file descriptors. To cover that code the fuzz | ||
| target converts bytes it receives into temporary files | ||
| and passes their file descriptors to elf_begin instead of calling | ||
| something like elf_memory (which can process bytes directly). | ||
| New fuzzers covering elfutils should avoid this pattern. */ | ||
|
|
||
| static const Dwfl_Callbacks core_callbacks = | ||
| { | ||
| .find_elf = dwfl_build_id_find_elf, | ||
| .find_debuginfo = dwfl_standard_find_debuginfo, | ||
| }; | ||
|
|
||
| int | ||
| LLVMFuzzerTestOneInput (const uint8_t *data, size_t size) | ||
| { | ||
| char name[] = "/tmp/fuzz-dwfl-core.XXXXXX"; | ||
| int fd = -1; | ||
| off_t offset; | ||
| ssize_t n; | ||
| Elf *core = NULL; | ||
| Dwfl *dwfl = NULL; | ||
|
|
||
| fd = mkstemp (name); | ||
| assert (fd >= 0); | ||
|
|
||
| n = write_retry (fd, data, size); | ||
| assert (n >= 0); | ||
|
|
||
| offset = lseek (fd, 0, SEEK_SET); | ||
| assert (offset == 0); | ||
|
|
||
| elf_version (EV_CURRENT); | ||
| core = elf_begin (fd, ELF_C_READ_MMAP, NULL); | ||
| if (core == NULL) | ||
| goto cleanup; | ||
| dwfl = dwfl_begin (&core_callbacks); | ||
| assert(dwfl != NULL); | ||
| if (dwfl_core_file_report (dwfl, core, NULL) < 0) | ||
| goto cleanup; | ||
| if (dwfl_report_end (dwfl, NULL, NULL) != 0) | ||
| goto cleanup; | ||
|
|
||
| cleanup: | ||
| dwfl_end (dwfl); | ||
| elf_end (core); | ||
| close (fd); | ||
| unlink (name); | ||
| return 0; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| #include <assert.h> | ||
| #include <stdio.h> | ||
| #include <stdlib.h> | ||
| #include "fuzz.h" | ||
|
|
||
| int | ||
| main (int argc, char **argv) | ||
| { | ||
| for (int i = 1; i < argc; i++) | ||
| { | ||
| fprintf (stderr, "Running: %s\n", argv[i]); | ||
|
|
||
| FILE *f = fopen (argv[i], "r"); | ||
| assert (f); | ||
|
|
||
| int p = fseek (f, 0, SEEK_END); | ||
| assert (p >= 0); | ||
|
|
||
| long len = ftell (f); | ||
| assert (len >= 0); | ||
|
|
||
| p = fseek (f, 0, SEEK_SET); | ||
| assert (p >= 0); | ||
|
|
||
| void *buf = malloc (len); | ||
| assert (buf != NULL || len == 0); | ||
|
|
||
| size_t n_read = fread (buf, 1, len, f); | ||
| assert (n_read == (size_t) len); | ||
|
|
||
| (void) fclose (f); | ||
|
|
||
| int r = LLVMFuzzerTestOneInput (buf, len); | ||
|
|
||
| /* Non-zero return values are reserved by LibFuzzer for future use | ||
| https://llvm.org/docs/LibFuzzer.html#fuzz-target */ | ||
| assert (r == 0); | ||
|
|
||
| free (buf); | ||
|
|
||
| fprintf (stderr, "Done: %s: (%zd bytes)\n", argv[i], n_read); | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| #ifndef _FUZZ_H | ||
| #define _FUZZ_H 1 | ||
|
|
||
| #include <stddef.h> | ||
| #include <stdint.h> | ||
|
|
||
| int LLVMFuzzerTestOneInput (const uint8_t *data, size_t size); | ||
|
|
||
| #endif /* fuzz.h */ |
Oops, something went wrong.