From fad71f6824d36caa5c14a6784f0ada3236912d3f Mon Sep 17 00:00:00 2001 From: Takeshi Yoneda Date: Tue, 28 Feb 2023 09:10:38 +0900 Subject: [PATCH 1/2] fuzz: update README instructions Signed-off-by: Takeshi Yoneda --- internal/integration_test/fuzz/README.md | 30 ++++++++++++++++++++---- 1 file changed, 26 insertions(+), 4 deletions(-) diff --git a/internal/integration_test/fuzz/README.md b/internal/integration_test/fuzz/README.md index 11799ddea6..fed737f068 100644 --- a/internal/integration_test/fuzz/README.md +++ b/internal/integration_test/fuzz/README.md @@ -9,17 +9,26 @@ Fuzzing infrastructure for wazero engines via [wasm-tools](https://github.com/by ### Run Fuzzing -Currently, we only have one kind of fuzzing named `basic` where we compare the results from the compiler -and interpreter engines, and see if there's a diff in them. To run the test, execute the following command: +Currently, we only have the following fuzzing targets: + +- `basic`: compares the results from the compiler and interpreter engines, and see if there's a diff in them. +- `memory_no_diff`: same as `basic` except that in addition to the results, it also compares the entire memory buffer between engines to ensure the consistency around memory access. + Therefore, this takes much longer than `basic`. +- `validation`: try compiling maybe-invalid Wasm module binaries. This is to ensure that our validation phase works correctly as well as the engines do not panic during compilation. + + +To run the fuzzer on a target, execute the following command: ``` # Running on the host archictecture. -cargo fuzz run basic +cargo fuzz run # Running on the specified architecture which is handy when developping on M1 Mac. -cargo fuzz run basic-x86_64-apple-darwin +cargo fuzz run -x86_64-apple-darwin ``` +where you replace `` is one of the targets described above. + See `cargo fuzz run --help` for the options. Especially, the following flags are useful: - `-jobs=N`: `cargo fuzz run` by default only spawns one worker, so this flag helps do the parallel fuzzing. @@ -29,6 +38,19 @@ See `cargo fuzz run --help` for the options. Especially, the following flags are - `-timeout` sets the timeout seconds _per fuzzing run_, not the entire job. +#### Example commands + +``` +# Running the `basic` target with 15 concurrent jobs with total runnig time with 2hrs. +$ cargo fuzz run basic -- -max_len=5000000 -max_total_time=7200 -jobs=15 + +# Running the `memory_no_diff` target with 15 concurrent jobs with timeout 2hrs and setting timeout per fuzz case to 30s. +$ cargo fuzz run memory_no_diff -- -timeout=30 -max_total_time=7200 -jobs=15 + +# Running the `validation` target with 4 concurrent jobs with timeout 2hrs and setting timeout per fuzz case to 30s. +# cargo fuzz run validation -- -timeout=30 -max_total_time=7200 -jobs=4 +``` + ### Reproduce errors If the fuzzer encounters error, you would get the output like the following: From c25a59bd2cb296826bec5d91e582fb55261fe3d7 Mon Sep 17 00:00:00 2001 From: Takeshi Yoneda Date: Tue, 28 Feb 2023 09:11:59 +0900 Subject: [PATCH 2/2] fuzz Signed-off-by: Takeshi Yoneda --- internal/integration_test/fuzz/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/internal/integration_test/fuzz/README.md b/internal/integration_test/fuzz/README.md index fed737f068..1b8bc36f3c 100644 --- a/internal/integration_test/fuzz/README.md +++ b/internal/integration_test/fuzz/README.md @@ -9,7 +9,7 @@ Fuzzing infrastructure for wazero engines via [wasm-tools](https://github.com/by ### Run Fuzzing -Currently, we only have the following fuzzing targets: +Currently, we have the following fuzzing targets: - `basic`: compares the results from the compiler and interpreter engines, and see if there's a diff in them. - `memory_no_diff`: same as `basic` except that in addition to the results, it also compares the entire memory buffer between engines to ensure the consistency around memory access. @@ -27,7 +27,7 @@ cargo fuzz run cargo fuzz run -x86_64-apple-darwin ``` -where you replace `` is one of the targets described above. +where you replace `` with one of the targets described above. See `cargo fuzz run --help` for the options. Especially, the following flags are useful: