Fuzzer

How a fuzzer works

A fuzzer does one cycle many times:

1. Make an input. Change a known good input, or make random data.
2. Start the software with that input.
3. Monitor the result. A crash, a hang or a memory error is a find.
4. Keep the useful inputs. Keep each input that caused an error, and each input that got to new code.

Fuzz testing started as a class project of Barton Miller at the University of Wisconsin in 1988. The first report came out in Communications of the ACM in December 1990. Modern engines are coverage guided.

The LLVM documentation calls libFuzzer “an in-process, coverage-guided, evolutionary fuzzing engine”. It changes the inputs in its corpus and keeps the ones that get to new code. LLVM advises to use libFuzzer with AddressSanitizer, UndefinedBehaviorSanitizer or the two together. These checkers make silent memory errors into crashes.

Type	Input source	Example
Generator	Random data or known dangerous values	OWASP description
Coverage guided	Mutations, kept if they get to new code	libFuzzer
Prompt fuzzer	Mutated attack prompts, judged by a model	GPTFuzz

Why fuzzers are important for security

A fuzzer finds bugs, vulnerabilities and unexpected behaviour automatically. OWASP calls fuzzing a complement to manual methods. It can find bugs that persons did not see.

• Scale. Google says OSS-Fuzz helped find and repair more than 10,000 vulnerabilities and 36,000 bugs. The count covers 1,000 open source projects, as of August 2023.
• Bugs stay. In a 2020 study, classic fuzz tests crashed or hung 9 of 74 Linux utilities. The counts were 15 of 78 on FreeBSD and 12 of 76 on macOS (Miller, Zhang, Heymann).

How AI changes fuzzing

AI and fuzzing connect in two directions. A language model helps the fuzzer, or a fuzzer tests the model.

AI helps the fuzzer

• Fuzz targets that an LLM writes. Google reported 26 new vulnerabilities in OSS-Fuzz in November 2024, one in OpenSSL (CVE-2024-9143). The work added more than 370,000 lines of coverage in 272 C/C++ projects.
• AI-assisted fuzzing campaigns. Trail of Bits built Buttercup for the DARPA AI Cyber Challenge. Buttercup starts with an AI/ML-assisted fuzzing campaign on OSS-Fuzz, then uses an AI patcher.
• AI that finds what fuzzing missed. In November 2024, Google Project Zero reported that its Big Sleep agent found a stack buffer underflow in SQLite. The OSS-Fuzz harness did not include the affected extension. With AFL on the SQLite CLI, the bug “remained undiscovered after 150 CPU-hours of fuzzing”. The authors say the results are “highly experimental”. They add that a target-specific fuzzer is likely as good today.

A fuzzer tests the model

GPTFuzz (Yu and colleagues, 2023) applies the AFL method to jailbreaking. It starts from human-written jailbreak templates and mutates them. A judgment model decides if each try worked. It reported more than 90% attack success rate against ChatGPT and Llama-2.

In practice

The last competition of the DARPA AI Cyber Challenge, in August 2025, shows what automated systems can do. Competitor systems found 54 of 63 synthetic vulnerabilities and patched 43. They also found 18 vulnerabilities that were not synthetic, at an average cost of about $152 for each task.

A prompt fuzzer such as GPTFuzz is one tool of AI red teaming for an AI system. Its authors built it to replace jailbreak templates that persons write by hand.

Questions and answers

What is the difference between a fuzzer and a fuzz target?

A fuzzer is the engine that makes and sends the inputs, such as libFuzzer or AFL++. A fuzz target is a small piece of code. It takes one input and gives it to the function in test. The OSS-Fuzz team at Google uses a language model to write fuzz targets.

Can a fuzzer test an AI model?

Yes. GPTFuzz applies the AFL method to prompts. It starts from human-written jailbreak templates and mutates them. A judgment model takes the position of a crash check. Its authors reported more than 90% attack success against ChatGPT and Llama-2 in 2023.