| 1 | # Fuzzing |
| 2 | |
| 3 | Each fuzzing target can be built with multiple engines. |
| 4 | Zstd provides a fuzz corpus for each target that can be downloaded with |
| 5 | the command: |
| 6 | |
| 7 | ``` |
| 8 | make corpora |
| 9 | ``` |
| 10 | |
| 11 | It will download each corpus into `./corpora/TARGET`. |
| 12 | |
| 13 | ## fuzz.py |
| 14 | |
| 15 | `fuzz.py` is a helper script for building and running fuzzers. |
| 16 | Run `./fuzz.py -h` for the commands and run `./fuzz.py COMMAND -h` for |
| 17 | command specific help. |
| 18 | |
| 19 | ### Generating Data |
| 20 | |
| 21 | `fuzz.py` provides a utility to generate seed data for each fuzzer. |
| 22 | |
| 23 | ``` |
| 24 | make -C ../tests decodecorpus |
| 25 | ./fuzz.py gen TARGET |
| 26 | ``` |
| 27 | |
| 28 | By default it outputs 100 samples, each at most 8KB into `corpora/TARGET-seed`, |
| 29 | but that can be configured with the `--number`, `--max-size-log` and `--seed` |
| 30 | flags. |
| 31 | |
| 32 | ### Build |
| 33 | It respects the usual build environment variables `CC`, `CFLAGS`, etc. |
| 34 | The environment variables can be overridden with the corresponding flags |
| 35 | `--cc`, `--cflags`, etc. |
| 36 | The specific fuzzing engine is selected with `LIB_FUZZING_ENGINE` or |
| 37 | `--lib-fuzzing-engine`, the default is `libregression.a`. |
| 38 | Alternatively, you can use Clang's built in fuzzing engine with |
| 39 | `--enable-fuzzer`. |
| 40 | It has flags that can easily set up sanitizers `--enable-{a,ub,m}san`, and |
| 41 | coverage instrumentation `--enable-coverage`. |
| 42 | It sets sane defaults which can be overridden with flags `--debug`, |
| 43 | `--enable-ubsan-pointer-overflow`, etc. |
| 44 | Run `./fuzz.py build -h` for help. |
| 45 | |
| 46 | ### Running Fuzzers |
| 47 | |
| 48 | `./fuzz.py` can run `libfuzzer`, `afl`, and `regression` tests. |
| 49 | See the help of the relevant command for options. |
| 50 | Flags not parsed by `fuzz.py` are passed to the fuzzing engine. |
| 51 | The command used to run the fuzzer is printed for debugging. |
| 52 | |
| 53 | Here's a helpful command to fuzz each target across all cores, |
| 54 | stopping only if a bug is found: |
| 55 | ``` |
| 56 | for target in $(./fuzz.py list); do |
| 57 | ./fuzz.py libfuzzer $target -jobs=10 -workers=10 -max_total_time=1000 || break; |
| 58 | done |
| 59 | ``` |
| 60 | Alternatively, you can fuzz all targets in parallel, using one core per target: |
| 61 | ``` |
| 62 | python3 ./fuzz.py list | xargs -P$(python3 ./fuzz.py list | wc -l) -I__ sh -c "python3 ./fuzz.py libfuzzer __ 2>&1 | tee __.log" |
| 63 | ``` |
| 64 | Either way, to double-check that no crashes were found, run `ls corpora/*crash`. |
| 65 | If any crashes were found, you can use the hashes to reproduce them. |
| 66 | |
| 67 | ## LibFuzzer |
| 68 | |
| 69 | ``` |
| 70 | # Build the fuzz targets |
| 71 | ./fuzz.py build all --enable-fuzzer --enable-asan --enable-ubsan --cc clang --cxx clang++ |
| 72 | # OR equivalently |
| 73 | CC=clang CXX=clang++ ./fuzz.py build all --enable-fuzzer --enable-asan --enable-ubsan |
| 74 | # Run the fuzzer |
| 75 | ./fuzz.py libfuzzer TARGET <libfuzzer args like -jobs=4> |
| 76 | ``` |
| 77 | |
| 78 | where `TARGET` could be `simple_decompress`, `stream_round_trip`, etc. |
| 79 | |
| 80 | ### MSAN |
| 81 | |
| 82 | Fuzzing with `libFuzzer` and `MSAN` is as easy as: |
| 83 | |
| 84 | ``` |
| 85 | CC=clang CXX=clang++ ./fuzz.py build all --enable-fuzzer --enable-msan |
| 86 | ./fuzz.py libfuzzer TARGET <libfuzzer args> |
| 87 | ``` |
| 88 | |
| 89 | `fuzz.py` respects the environment variables / flags `MSAN_EXTRA_CPPFLAGS`, |
| 90 | `MSAN_EXTRA_CFLAGS`, `MSAN_EXTRA_CXXFLAGS`, `MSAN_EXTRA_LDFLAGS` to easily pass |
| 91 | the extra parameters only for MSAN. |
| 92 | |
| 93 | ## AFL |
| 94 | |
| 95 | The default `LIB_FUZZING_ENGINE` is `libregression.a`, which produces a binary |
| 96 | that AFL can use. |
| 97 | |
| 98 | ``` |
| 99 | # Build the fuzz targets |
| 100 | CC=afl-clang CXX=afl-clang++ ./fuzz.py build all --enable-asan --enable-ubsan |
| 101 | # Run the fuzzer without a memory limit because of ASAN |
| 102 | ./fuzz.py afl TARGET -m none |
| 103 | ``` |
| 104 | |
| 105 | ## Regression Testing |
| 106 | |
| 107 | The regression test supports the `all` target to run all the fuzzers in one |
| 108 | command. |
| 109 | |
| 110 | ``` |
| 111 | CC=clang CXX=clang++ ./fuzz.py build all --enable-asan --enable-ubsan |
| 112 | ./fuzz.py regression all |
| 113 | CC=clang CXX=clang++ ./fuzz.py build all --enable-msan |
| 114 | ./fuzz.py regression all |
| 115 | ``` |
| 116 | |
| 117 | ## Fuzzing a custom sequence producer plugin |
| 118 | Sequence producer plugin authors can use the zstd fuzzers to stress-test their code. |
| 119 | See the documentation in `fuzz_third_party_seq_prod.h` for details. |