Fuzz: Add process_messages harness (tests)


Host: MarcoFalke  -  PR author: MarcoFalke

The PR branch HEAD was fa6a0084 at the time of this review club meeting.


  • A few weeks ago, we looked at how fuzzing can find consensus bugs, such as money printing (brrr̅). This week we will use fuzzing to find a remote crasher.

  • BIP 37 describes a way to filter transaction relay by bloom filters. For the purposes of this pull request we will only need to know the different message types it introduces and how the Bitcoin Core node processes them.

  • CVE-2013-5700 was a vulnerability that could crash a node by merely sending two P2P messages.

  • Bitcoin core has documentation on how to compile with AFL and libFuzzer on Linux. Other fuzz engines and operating systems might work, but are currently undocumented.

  • CNode is the data structure to represent a connection in Bitcoin Core.

  • CConnman is the connection manager in Bitcoin Core. A thread to handle messages from connections is created at startup. This is often referred to as the “main” thread in Bitcoin Core, taking care of message processing and validation. See ThreadMessageHandler.

  • The pull request we are looking at this week is extending CConnman for testing purposes with several features:

    • Adding and removing nodes directly (without having to create a socket)
    • Push a serialized message onto a nodes’ receive buffer
  • The fuzz test itself does mainly two things for each fuzz input or fuzz seed:

    • Add a random amount of test nodes (the number of nodes is read from the fuzz input)
    • Pick a random peer and send some random bytes as a message from this peer, i.e. put the bytes into the receive buffer of this peer
    • Repeat the last step


  1. Did you review the PRs? Concept ACK, approach ACK, tested ACK, or NACK? (Don’t forget to put your PR review on GitHub.)

  2. How would a remote node crash a Bitcoin Core node that is vulnerable to the attack? What steps would it need to take and what messages would it send?

  3. How was the vulnerability fixed?

  4. Where in CNode are bloom filters for transaction relay filtering stored?

  5. Where are bloom filter messages handled by Bitcoin Core?

  6. Based on the previous questions, how would a fuzz test for Bitcoin Core exploit this vulnerability? What are the steps needed on a high level to achieve this setup?

  7. What patch needs to be applied to current Bitcoin Core to reintroduce the vulnerability?

  8. Did you get fuzzing to run on your machine? If not, what issues did you run into?

  9. Did you find the crash?

Meeting Log

  112:59 <MarcoFalke> #startmeeting
  213:00 <MarcoFalke> hi everyone
  313:00 <lightlike> hi
  413:00 <theStack> hi
  513:00 <andrewtoth> hi
  613:00 <michaelfolkson> hi
  713:00 <sipa> hi
  813:01 <shaunsun__> hi
  913:01 <vasild> hi
 1013:01 <brikk> hi
 1113:01 <robot-visions> hi
 1213:01 <MarcoFalke> Reminder that you are welcome to ask questions. Don't ask to ask. We are all here to learn (including myself)
 1313:01 <emzy> Hi
 1413:01 <rjected> hi
 1513:01 <jnewbery> hi
 1613:01 <MarcoFalke> To get started, did you get a chance to review the pull request or take a look at it?
 1713:01 <nehan_> hi
 1813:02 <robot-visions> (y) reviewed
 1913:02 <nehan_> y
 2013:02 <vasild> y, briefly
 2113:02 <emzy> n
 2213:02 <soup> n
 2313:02 <lightlike> yes
 2413:02 <andrewtoth> y
 2513:02 <theStack> n (still occupied with #17860)
 2613:02 <jonatack> hi
 2713:03 <michaelfolkson> Took a look at it yes. Trying to learn more about fuzz testing generally so I can better understand where we are going with it.
 2813:03 <MarcoFalke> ok, so to get started with the net_processing vulnerability in the bloom filter that the fuzz tests is trying to find. What are the messages a remote node needs to send?
 2913:04 <robot-visions> Send a `filterload` where the bloom filter's underlying array is empty, then send a `filteradd` with arbitrary data; see [#18515](https://github.com/bitcoin/bitcoin/pull/18515)
 3013:04 <MarcoFalke> michaelfolkson: Good to hear. I am hosting this club to get everyone introduced to fuzzing :)
 3113:04 <MarcoFalke> robot-visions: Correct
 3213:04 <MarcoFalke> Did anyone come up with alternative ways to trigger the vulnerability?
 3313:05 <michaelfolkson> This was good. Any other good recommended resources would be gratefully received https://diyhpl.us/wiki/transcripts/cppcon/2017/2017-10-11-kostya-serebryany-fuzzing/
 3413:05 <robot-visions> Is it necessary to send the second `filteradd`, or is it enough to just wait for an `inv` from elsewhere?
 3513:05 <jonatack> michaelfolkson: you'll find the PR description most helpful :p
 3613:05 <michaelfolkson> Yup that was good too ;)
 3713:06 <MarcoFalke> robot-visions: Anything that eventually calls CBloomFilter::Hash should crash the node
 3813:07 <robot-visions> Thanks MarkoFalke! Makes sense.
 3913:07 <MarcoFalke> So I believe waiting for the node to process a transaction either from itself or from another peer should also hit the crash
 4013:07 <MarcoFalke> How was this vulnerability fixed?
 4113:07 <michaelfolkson> So practicalswift and robot-visions were able to trigger it. They triggered it in different ways?
 4213:08 <MarcoFalke> michaelfolkson: I haven't looked at their fuzz inputs, but we can do that later
 4313:08 <michaelfolkson> Ok
 4413:08 <MarcoFalke> (If they share them)
 4513:09 <robot-visions> Vulnerability was fixed by introducing `isEmpty` and `isFull` flags to return early if the Bloom filter's underlying array is empty; see [#2914](https://github.com/bitcoin/bitcoin/pull/2914)
 4613:10 <lightlike> It was fixed covertly, by adding a UpdateEmptyFull() function that is processed on filterload (net_processing) an sets isFull if the underlaying array is empty. Filteradd will not insert anything if isFull is set
 4713:10 <theStack> is there plans to commit certain fuzz inputs into the repository? (or is that already the case?)
 4813:10 <MarcoFalke> theStack: They are located in bitcoin-core/qa-assets on GitHub
 4913:11 <theStack> MarcoFalke: ah, i see. tend to forget that there are also other repos outside of bitcoin/bitcoin ;-)
 5013:11 <MarcoFalke> robot-visions: lightlike: Correct
 5113:12 <lightlike> I understand why it was fixed this way back then, but wouldn't it make sense to do something more obvious today (like also check for zero in Hash()?)
 5213:12 <theStack> i was quite fascinated by this covert fix. i can imagine is it really hard to fix something but needing to pretend doing something different
 5313:12 <theStack> lightlike: the point is, if you do it in an obvious way, the vulnerability becomes public and is exploited
 5413:13 <chanho> I think the question is, should the covert fix be removed and updated with a proper fix, e.g. check for zero division?
 5513:13 <lightlike> theStack: yes, I understand, but my question referred to today where basically all nodes are no longer vulnerable
 5613:14 <MarcoFalke> I think theStack has a good point. If the fix for this trivial exploit is a one-line patch, it might make it easy for bad players to exploit it
 5713:15 <theStack> chanho: lightlike: to my understanding, it could be removed and done in a simpler way
 5813:16 <jnewbery> lightlike chanho: I agree that after some time has passed (perhaps a couple of years) these covert fixes should be cleared up and made public. Leaving them in goes against the goal of having the code as clear and straightforward as possible.
 5913:17 <MarcoFalke> I haven't thought about this question upfront, but I think even today the patch would look similar.
 6013:17 <jnewbery> to be fair, I don't think they're left in this way on purpose. I expect the person who fixes it probably just forgets and it's not their priority after two years
 6113:18 <MarcoFalke> But let's not talk about how the code can be cleaned up today and focus on fuzzing :)
 6213:18 <MarcoFalke> All we need to know about the vulnerability for now is how it was triggered and how it was fixed
 6313:18 <michaelfolkson> The fuzz inputs generated here. How were they generated? Much thought go into it? https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus
 6413:19 <MarcoFalke> michaelfolkson: Anyone can generate them, but we'll cover this in the second half of the meeting
 6513:19 <michaelfolkson> Ok sorry :)
 6613:19 <MarcoFalke> no worries
 6713:19 <MarcoFalke> Let's take a look at the structure of Bitcoin Core. Where in CNode are the bloom filters stored?
 6813:20 <robot-visions> I think it's in `CNode->m_tx_relay->pfilter`
 6913:20 <MarcoFalke> Right
 7013:20 <theStack> robot-visions: +1
 7113:20 <MarcoFalke> And where are bloom filter messages handled?
 7213:21 <theStack> net_processing.cpp, directly in the huge ProcessMessage() function
 7313:21 <robot-visions> theStack: (y)
 7413:23 <MarcoFalke> A fuzzing harness needs an entry point, and to execute the bloomfilter code, it needs to call into the ProcessMessage function
 7513:24 <MarcoFalke> Does anyone have questions about the structure of the fuzzing harness?
 7613:25 <MarcoFalke> Pretty much what it does is, (1) create a few CNode (with the bloomfilters initialized), then (2) pretend they are sending random messages by passing them into the ProcessMessage function
 7713:26 <sipa> jnewbery: an alternative is (in general) not removing covert fixes (because touching the code again introduces risks of its own), but still documenting them in the code
 7813:26 <andrewtoth> What is the purpose of having a random number of nodes created? Is it because it affects what they do with the messages if they have different number of peers?
 7913:27 <michaelfolkson> I suppose you are receiving different numbers of messages depending on how many peers you have
 8013:28 <MarcoFalke> sipa: Good point . I've done something for the money printing bug in #17080
 8113:29 <theStack> If a constant number of nodes were be sufficient, could that CNode creation code move into initialize()? Or is there any other reason it has to be created again and again for each input?
 8213:29 <theStack> s/were be/were/
 8313:29 <MarcoFalke> andrewtoth: Good question. The purpose of the fuzz test was to be more than just a test to find the bloomfilter bug. It should also test other parts of the code like transaction relay, which has a lot of state
 8413:30 <lightlike> theStack: I believe the service flags are fuzzed too, so maybe that is the reason?
 8513:30 <MarcoFalke> Unfortunately, the current fuzzer can not test transaction relay because messages are constructed from something that looks like a random stream
 8613:31 <theStack> lightlike: you are right, there are numerous other properties that are fuzzed
 8713:31 <MarcoFalke> lightlike: Jup, I tried to be as permissive as possible. Give the fuzz test as much space to search as possible.
 8813:31 <andrewtoth> MarcoFalke: so to test transaction relay, a separate harness that uses the fuzz inputs to generate different relay messages needs to be built?
 8913:32 <lightlike> I noticed that the fuzzer needs quite some of tries (>10k) until it gets to the ProcessMessage part the first time
 9013:32 <sipa> MarcoFalke: a special fuzzregtest mode that disables signature validation?
 9113:32 <MarcoFalke> andrewtoth: In theory the fuzz engine could figure out how to construct transactions. All it has to do is guess the inputs correctly
 9213:32 <MarcoFalke> sipa: It needs to guess the hash of the input
 9313:33 <MarcoFalke> Which has low probability assuming the best you can do is random guesses
 9413:33 <jnewbery> sipa: (re removing covert fixes) yes. I think it depends on context (riskiness of change vs benefit of clarifying code to match intent)
 9513:34 <andrewtoth> So a harness which better guides the fuzzer would be more efficient for tx relay
 9613:34 <sipa> MarcoFalke: oh, right
 9713:34 <MarcoFalke> However, modern fuzz engines do a lot better on many things than just random guesses. For example, they can extract strings easily
 9813:35 <MarcoFalke> You can see that when running the fuzzer. In a few seconds it has already guessed some of the message types
 9913:35 <MarcoFalke> andrewtoth: Yes
10013:35 <MarcoFalke> I am still thinking what the best way is to achieve this
10113:35 <sipa> right, because the coverage guiding can observe the characters being compared against
10213:35 <sipa> this wouldn't be the case for a hash comparison
10313:36 <theStack> MarcoFalke: would you count both libfuzzer and AFL to the category of "modern fuzz engines"? or are there better ones (commercial or anything)?
10413:36 <MarcoFalke> sipa: Yes. I don't know how fuzz engines work, but I could also imagine that it might dump the binary and see if there are any strings in it.
10513:37 <sipa> MarcoFalke: i believe not
10613:37 <MarcoFalke> (Not sure if you can even inspect your own binary with C++)
10713:38 <MarcoFalke> theStack: I have only tried the two
10813:38 <lightlike> I thought it is some kind of genetic algorithm that tries to optimize code coverage and performs all kind of mutations.
10913:38 <MarcoFalke> So did anyone find the crasher? What patch did you have to apply to re-introduce the vulnerability?
11013:38 <michaelfolkson> theStack: For fuzzing open source projects, libFuzzer and AFL were modern in 2017
11113:39 <robot-visions> To re-introduce the vulnerability: It suffices to remove the `if (isFull) return` check at the beginning of `CBloomFilter::insert`
11213:39 <lightlike> I noticed the fuzzer finds easy messages ("inv") much faster than longer ones ("filterload")
11313:40 <MarcoFalke> For me the first thing it found was -dropmessagetest
11413:40 <nehan_> i'm running the fuzzer but it doesn't seem to be showing me which messages it's using:
11513:40 <MarcoFalke> Which is a command line flag, ouch
11613:40 <lightlike> nehan: i grepped the corpus directory for the strings to see if they were found yet
11713:41 <MarcoFalke> nehan_: If you use libfuzzer, sometimes it will tell you in the stdout lines which "dictionary" words were added or removed
11813:41 <MarcoFalke> It is also possible to inspect the input directory directly
11913:41 <sipa> MarcoFalke: are you familiar with the -dict optionm
12013:41 <sipa> ?
12113:41 <MarcoFalke> Though, be warned if you use cat to redirect a fuzz input to your terminal
12213:41 <nehan_> lightlike: thanks
12313:41 <nehan_> MarcoFalke: I'm wondering if I didn't compile something correctly, I only seem to have symbols:
12413:41 <nehan_> #868056 REDUCE cov: 17995 ft: 84042 corp: 1298/1621Kb exec/s: 446 rss: 510Mb L: 3483/4096 MS: 4 InsertByte-InsertByte-InsertRepeatedBytes-EraseBytes-
12513:42 <MarcoFalke> sipa: no, is that an option to libfuzzer?
12613:42 <sipa> nehan_: it lists the mutations it is applying to existing seeds
12713:42 <MarcoFalke> nehan_: It shows it only rarely for me
12813:42 <sipa> nehan_: if it finds a crash, it will create a file with the input that causes the crash
12913:43 <sipa> MarcoFalke: yeah, to tell it about strings that are likely relevant to your fuzzer
13013:43 <jonatack> nehan_: i'm seeing the same, and occasionally there's a message at the end of the line
13113:43 <jonatack> 371324REDUCE cov: 14900 ft: 65908 corp: 907/963Kb exec/s: 388 rss: 534Mb L: 2020/4096 MS: 5 CMP-ChangeBinInt-InsertRepeatedBytes-InsertRepeatedBytes-EraseBytes- DE: "verack"-
13213:43 <MarcoFalke> sipa: Ah, so I could pass in the utxo set as a dict?
13313:44 <MarcoFalke> jonatack: Thanks, that is what I meant
13413:44 <sipa> MarcoFalke: well for that you could also just manually create a seed
13513:44 <jonatack> nehan_: (good question)
13613:44 <sipa> i believe DE is in fact it using a dictionary word
13713:44 <nehan_> i've been running for an hour or so and it has created 1330 files. All of those can't be crashes?
13813:44 <sipa> nehan_: only if they're called crash-XXXXX
13913:45 <nehan_> sipa: ah, thanks. let me go learn more about libfuzzer...
14013:45 <MarcoFalke> If you inspect the seeds manually with cat, please pass in `cat --show-nonprinting`, otherwise it might execute arbitary code in your terminal
14113:45 <sipa> the files you usually see are the seeds
14213:46 <sipa> they're the input that maximize coverage (of code and of "features")
14313:46 <nehan_> how long did it take folks to find the crash?
14413:46 <MarcoFalke> To run the fuzz harness can supply a folder where to put the seeds: ./src/test/fuzz/process_messages ./where/to/put/the/seeds
14513:46 <MarcoFalke> And then `cat --show-nonprinting ./where/to/put/the/seeds/000fff...`
14613:47 <MarcoFalke> nehan_: Good question. I am also interesed in that.
14713:47 <lightlike> I noticed if I stop the fuzzer (Ctrl+C) and start it again with the same seed dir, the coverage will be significantly lower than before, as if it didn't reload all the work done before. Does anyone know why this is the case? (I thought the seeds would be updated after each try)
14813:48 <MarcoFalke> I think the fastest I got was 600k, but with 9 workers in parallel, so I am not sure if that counts because you can't compare it against a single worker. (The libfuzzer workers work together on the same seed corpus)
14913:49 <jonatack> I'm still running it, will comment in the PR when it crashes
15013:49 <sipa> lightlike: as in the "cov" number is lower?
15113:49 <sipa> that'd surprise me
15213:49 <lightlike> sipa: yes, cov and ft
15313:49 <MarcoFalke> lightlike: Good question. I believe it is because of "adjacant" coverage. Bitcoin core runs a lot of threads (like the stale tip check in the scheduler), and those are not included in the coverage if you start afresh
15413:49 <sipa> ah
15513:50 <MarcoFalke> Which is unfortunate, but I don't know how to "fix"
15613:50 <sipa> could be the result of non-deterministic behavior
15713:50 <MarcoFalke> sipa: Or that of course
15813:50 <sipa> if the fuzzed binary uses randomness anywhere, fuzzing is less.efficient
15913:50 <MarcoFalke> I think my greatest fear would be to find a crasher and then try to reproduce it with the same seed and it wouldn't crash because of non-determinism
16013:51 <MarcoFalke> deterministic randomness is fine (as long as it is derived by the fuzz engine)
16113:51 <lightlike> I didn't find the bug after 10M steps and stoppped, but found it after 2.2 million steps with a version in which I replaced the MsgType Fuzzing with an array of all known Messages, for which then only the index is fuzzed.
16213:52 <MarcoFalke> Did anyone run into issues while fuzzing or setting up the fuzzer?
16313:52 <emzy> The fuzzing is deterministic? So generated by a pseudorandom number generator?
16413:53 <robot-visions> MarkoFalke: I was able to get things work by following the "macOS hints for libFuzzer", but I had to remove the `--disable-asm` flag when running `./configure`
16513:53 <robot-visions> Marco* (sorry)
16613:53 <MarcoFalke> emzy: I think the fuzz engine picks a random seed at the beginning, but it might be possible to make even the fuzzing deterministic
16713:54 <emzy> I see.
16813:54 <sipa> you can pick the rng seed
16913:54 <MarcoFalke> robot-visions: Is that wrong in our docs? It could help others if you amend that note there
17013:56 <MarcoFalke> sipa: Correct, for libfuzzer you can pick the internal starting seed
17113:56 <theStack> MarcoFalke: i had a linking issue with the http_request fuzz test, which used internal libevent functions (evhttp_parse_firstline_) that were named slightly different on my libevent -- could fix it locally though, will open an issue later
17213:56 <MarcoFalke> theStack: Does Bitcoin Core compile fine without libFuzzer?
17313:57 <theStack> MarcoFalke: yes it does
17413:57 <MarcoFalke> hm
17513:58 <nehan_> i had an unused variable warning in test/fuzz/locale.cpp:59
17613:58 <jonatack> sipa: MarcoFalke: found this on the -dict option sipa mentioned https://llvm.org/docs/LibFuzzer.html#dictionaries
17713:58 <jonatack> full list of options here: https://llvm.org/docs/LibFuzzer.html#options
17813:58 <andrewtoth> nehan_ same
17913:58 <robot-visions> MarcoFalke: I think the docs are reasonable right now. It says (1) "you may need to run `--disable-asm` to avoid errors, and (2) "here's what worked on our setup (which included `--disable-asm`)". It doesn't say you *must* use that flag.
18013:59 <MarcoFalke> nehan_: Good point. The warning is caused because a fuzz test was modified. To not invalidate existing seeds, the fuzz harness still needs to consume the same bytes it did previously, but now they are unused. I think this can be fixed by prefixing the line with (void)
18113:59 <nehan_> MarcoFalke: interesting!
18214:00 <MarcoFalke> #endmeeting
18314:00 <MarcoFalke> Thanks everyone!
18414:00 <robot-visions> Thanks!
18514:00 <vasild> Thanks!
18614:00 <theStack> thanks for hosting!
18714:00 <nehan_> thanks!
18814:00 <jonatack> or passing -help=1 ... e.g. src/test/fuzz/process_messages -help=1
18914:00 <emzy> Thanks!
19014:00 <jonatack> thanks MarcoFalke!
19114:01 <lightlike> thanks!
19214:01 <nehan_> i've never done any fuzzing before so this was cool
19314:02 <andrewtoth> Thanks MarcoFalke!
19414:03 <jnewbery> thanks MarcoFalke. Great Review Club!
19514:03 <michaelfolkson> You going to hang around for a bit MarcoFalke or you need to go?
19614:03 <jonatack> If useful, MarcoFalke also hosted a review club session on fuzzing in January: https://bitcoincore.reviews/17860
19714:04 <theStack> to shortly bring up the subject of remaining covert fixes again: it can be confusing to code readers if it remains. E.g. it led to the issue #16886 and its fix PR #16922 (the latter one by me), both made in the wrong assumption that the point of the empty/full flags were optimization, as we didn't know about the covert fix
19814:04 <jonatack> theStack: the meeting log for that session begins with us trying to get fuzz/test_runner.py to work :)
19914:05 <robot-visions> On a similar note to what theStack mentioned, would it make sense to now consider a peer "misbehaving" if they send an empty bloom filter?
20014:05 <theStack> jonatack: awesome -- though from my side it was more of a conclusion that i don't need the fuzz test_runner ;)
20114:05 <jonatack> theStack: i agree that a pr like https://github.com/bitcoin/bitcoin/pull/17080 that documents this could be helpful
20214:07 <theStack> robot-visions: personally i think that would be fine, don't know though how strict BIP37 is interpreted -- it afair only defines an upper filter size limit, but not a lower limit
20314:08 <theStack> robot-visions: on the other hand i wouldn't know the point of an empty filter
20414:09 <theStack> jonatack: yes i also think that a short explanation with CVE number mentioning would be more appropriate
20514:12 <jonatack> robot-visions: interesting question
20614:13 <lightlike> theStack: I found it interesting that I didn't find any detailed description on how to crash the node using the CVE apart from the one in your PR. Did you find any, or did you just figure it out yourself?
20714:17 <theStack> lightlike: i was also wondering the same. the only information i got was that it is triggered by a division by zero. then i inspected the CBloomFilter class for divisions, and since the class is not that large i could figure it out quite fast
20814:19 <theStack> now at least in the bitcoin-wiki about CVEs both the covert fix and were linked by someone (https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures#CVE-2013-5700)
20914:19 <andrewtoth> I think the exploit is not described in detail anywhere conspicuous intentially
21014:19 <andrewtoth> *intentionally
21114:19 <sipa> at the time, it was certainly intentionally
21214:19 <sipa> though it should be documented now
21314:20 <andrewtoth> sipa: +1
21414:20 <lightlike> andrewtoth: I thought that whenever the CVE is published (in the wiki/mailing list) everything will be made public.
21514:21 <sipa> lightlike: it's a very ad-hoc process really
21614:23 <robot-visions> Thanks again everyone! I haven't seen fuzzing before, it's really interesting. Hope to see you at a future session.
21714:23 <theStack> interestingly enough there still seem to be listening nodes that are vulnerable to this (looking at the user agents on bitnodes.io)
21814:24 <emzy> Also altcoins may be on old versions.
21914:25 <sipa> theStack: there are still nodes with code from 2013? :s
22014:25 <emzy> But i think there was enought time to update for both.
22114:27 <MarcoFalke> I think the issue with making CVEs public is mostly people forgetting about them
22214:28 <sipa> yeah
22314:28 <MarcoFalke> Obviously you can't make them public on the first day they are discovered, and if the time has passed to make them public, they might be forgotten about
22414:29 <theStack> sipa: well according to that site five nodes still are on 0.8.x
22514:29 <theStack> x >= 5 though... this CVE was fixed in 0.8.4, so they are at least not vulnerable to this
22614:30 <lightlike> I'd guess the finder would like attribution - even if there are no bounties, finding a CVE in core seems like something to be proud of.
22714:31 <lightlike> at least if they are not a regular
22814:32 <MarcoFalke> Everyone who reports an issue to the security email of Bitcoin Core will get mentioned in the release notes of the release that fixed it
22914:39 <jonatack> About the "DE:" keys that prefix new messages in the fuzzer output, https://llvm.org/docs/LibFuzzer.html#output doesn't mention them, my guess is they might mean Dictionary Entry?
23014:40 <sipa> that's my guess
23114:44 <lightlike> Internally to the fuzzer, there is also "PersAutoDict" that you see in the output: "Persistent dictionary modified by the fuzzer, consists entries that led to successfull discoveries in the past mutations."
23215:01 <jonatack> yes, in the mutation operations output, e.g. MS: 3 ChangeBit-PersAutoDict-EraseBytes- DE: "\xdf\...
23315:02 <jonatack> lightlike: thanks, what doc are you quoting from?
23415:03 <lightlike> jonatack: I googled for PersAutoDict, and this is a code comment from libfuzzer
23515:05 <lightlike> https://llvm.org/doxygen/FuzzerMutate_8h_source.html
23615:09 <jonatack> lightlike: thanks -- the code confirms that DE is for Dictionary Entry too
23715:33 <instagibbs> another example of covert fix being documented but not touched otherwise: https://github.com/bitcoin/bitcoin/pull/16885