Cflite test #53
Closed
Cflite test #53
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
evverx
force-pushed
the
cflite-test
branch
3 times, most recently
from
f8d85ee to
6fffe3d
Compare
evverx
force-pushed
the
cflite-test
branch
19 times, most recently
from
1a5f154 to
2be3f93
Compare
evverx
added a commit
that referenced
this pull request
Dec 19, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Dec 19, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is tested. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
force-pushed
the
cflite-test
branch
2 times, most recently
from
8b04bf2 to
9d9ce08
Compare
evverx
added a commit
that referenced
this pull request
Dec 21, 2021
[v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
force-pushed
the
cflite-test
branch
2 times, most recently
from
8c4801a to
f1d2bec
Compare
evverx
added a commit
that referenced
this pull request
Mar 21, 2022
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 21, 2022
[v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 21, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were added. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 21, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were added. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 22, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 23, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 25, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Mar 30, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Apr 3, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Apr 5, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Apr 23, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
evverx
added a commit
that referenced
this pull request
Apr 25, 2022
[v5] 1) `fuzz-libdwfl` and `fuzz-libelf` were moved from OSS-Fuzz. 2) The regression testsuite was extended. 3) The OSS-Fuzz build script was removed. It should probably be kept on OSS-Fuzz at this point. 4) The honggfuzz kludges were removed because google/honggfuzz#431 was fixed. [v4] 1) `--enable-afl` was added to make it possible to build and run the fuzz target with afl-gcc/afl-g++/afl-fuzz. It's compatible with both AFL and AFL++ and can be tested on Ubuntu by running the following commands: ``` apt-get install afl++ autoreconf -i -f ./configure --enable-maintainer-mode --enable-afl make -j$(nproc) V=1 make check V=1 VERBOSE=1 TESTS=run-fuzz-dwfl-core.sh FUZZ_TIME=600 ``` It's compatible with ASan/UBsan as well so something like `--enable-sanitize-address` and `--enable-sanitize-undefined` can additionally be passed to run the fuzzer under ASan/UBsan. It was tested on Fedora with AFL (https://github.com/google/AFL) and on Ubuntu with AFL++ (https://github.com/AFLplusplus/AFLplusplus/). 2) Both `use_afl` and `use_honggfuzz` are now shown among the additional test features printed by `./configure` to make it easier to figure out how exactly elfutils is tested. [v3] The test handles infinite loops much better now. In https://sourceware.org/bugzilla/show_bug.cgi?id=28715#c4 it took it about 5 hours on Packit to discover an infinite loop on 32 bit platforms because it didn't enforce any timeouts. It was fixed by passing --tmout_sigvtalrm to honggfuzz (which treats timeouts as normal crashes) and by additionally running the fuzz target with timeout -s9. [v2] 1) At https://sourceware.org/pipermail/elfutils-devel/2021q4/004541.html it was pointed out that build-fuzzers.sh is too tied to OSS-Fuzz and while it was kind of decoupled from it as much as possible in the sense that it was enough to install clang and run the script to build the fuzz target with libFuzzer it's true that it can't be integrated smoothly into buildbot for example where gcc is used and various configure options control what exactly is testsed. To address that, `--enable-honggfuzz` is introduced. It looks for hfuzz-gcc, hfuzz-g++ and honggfuzz and if they exist elfutils is built with those wrappers and the fuzz target is additionally run for a few minutes under honggfuzz to make regression testing more effective. It was tested on Fedora 35 and in #53 on Ubuntu Focal with both gcc and clang with and without sanitizers/Valgrind and with two versions of honggfuzz (including the latest stable version). To make it work on Ubuntu the following commands should be run ``` apt-get install libbfd-dev libunwind8-dev git clone https://github.com/google/honggfuzz cd honggfuzz git checkout 2.4 make make PREFIX=/usr install cd PATH/TO/ELFUTILS autoreconf -i -f ./configure --enable-maintainer-mode --enable-honggfuzz make check V=1 VERBOSE=1 # FUZZ_TIME can be optionally passed ``` If hongfuzz is installed elsewhere it's possible to point configure to it with CC, CXX and HONGGFUZZ ``` ./configure CC=path-to-hfuzz-gcc CXX=path-to-hfuzz-g++ HONGGFUZZ=path-to-honggfuzz ``` I decided to use honggfuzz instead of AFL because AFL doesn't seem to be maintained actively anymore. Other than that I can't seem to make it work with various combinations of compilers, sanitizers and so on. But thanks to the way the fuzz target is written it should be possible to add it eventually by analogy with honggfuzz. 2) fuzz-dwfl-core-corpus was renamed to fuzz-dwfl-core-crashes to make it more clear that it isn't exaclty a seed corpus. 3) run-strip-g.sh and run-strip-nothing.sh started to compile test programs using temporary files instead of gcc -xc -. It should address google/honggfuzz#431 but more generally autoconf uses temporary files to make sure compiler works so it seems in general it's safer to rely on compiler features that are known to work. 4) A comment was added where I tried to expand on why the fuzz target is written that way. [v1] The fuzz target was integrated into OSS-Fuzz in google/oss-fuzz#6944 and since then it has been running there continously (uncovering various issues along the way). It's all well and good but since OSS-Fuzz is far from the elfutils repository it's unnecessarily hard to build the fuzz target locally, verify patches and more generally test new code to make sure that it doesn't introduce new issues ( or reintroduce regressions). This patch aims to address all those issues by moving the fuzz target into the elfutils repository, integrating it into the testsuite and also providing a script that can be used to build full-fledged fuzzers utilizing libFuzzer. With this patch applied `make check` can be used to make sure that files kept in tests/fuzz-dwfl-core-corpus don't crash the code on various architecures. `--enable-sanitize-{address,undefined}` and/or `--enable-valgrind` can additionally be used to uncover issues like https://sourceware.org/bugzilla/show_bug.cgi?id=28685 that don't always manifest themselves in simple segfaults. On top of all that now the fuzz target can be built and linked against libFuzzer locally by just running `./tests/build-fuzzers.sh`. The patch was tested in #53 : * the testsuite was run on aarch64, armhfp, i386, ppc64le, s390x and x86_64 * Fedora packages were built on those architectures; * elfutils was built with both clang and gcc with and without sanitizers to make sure the tests pass there; * `make distcheck` passed; * coverage reports were built to make sure "static" builds are intact; * the fuzz target was built and run with ClusterFuzzLite to make sure it's still compatible with OSS-Fuzz; * the code was analyzed by various static analyzers to make sure new alerts aren't introduced. Signed-off-by: Evgeny Vereshchagin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.