Fuzz: BIP 42, BIP 30, CVE-2018-17144 (tests )
Jan 15, 2020
https://github.com/bitcoin/bitcoin/pull/17860
Host:
MarcoFalke
PR author: MarcoFalke
The PR branch HEAD was 8b868f17 at the time of this review club meeting.
Notes
Questions
Did you review the PR? Concept ACK, approach ACK, ACK <commit>, or
NACK? Â
Don’t forget to put your PR review on GitHub or ask
questions .
What is the difference between black-box and white-box fuzzing? Which technique does Bitcoin Core use? Refer to “The
Art, Science, and Engineering of Fuzzing: A Survey” in the notes.
Did you compile the new fuzz target and run the fuzzer?
How can CVE-2018-17144 be exploited? Can you explain conceptually how to create an example block that exploits the
bug?
What does the new fuzz test do on a high level? What “Actions” can it run?
On a low level, the fuzz test reads bytes from a given seed via the FuzzedDataProvider. Integral values
are read from individual bytes starting at the end of the seed. Take a look at the example
seed
(zzzii, without a new line) in the qa-assets repo. How many blocks are mined? Either explain by looking at the
seed and code or obtain the result by modifying and running the fuzz test on the seed.
How can the fuzz target be tested for accuracy? Hint: How would you modify the Bitcoin Core source code to trigger
the CVE? See also this pull request: consensus: Explain why fCheckDuplicateInputs can not be skipped and remove
it
Could you find a seed that triggers the CVE? You can either create it manually, if you found an answer to question 4,
or you can let the fuzzer run until it hits an assertion. If you let the fuzzer run: Was it successful? If not, what
is the bottleneck of the fuzz test and could it be improved?
Meeting Log
1 11:31 <instagibbs> I can't actually get the fuzztester to "Do anything". The readme focuses on setting up but doesn't actually give an example with expected output.
2 11:31 <instagibbs> ./test/fuzz/test_runner.py ${DIR_FUZZ_IN}
3 11:32 <instagibbs> results in it just printing out a bunch of names in arrays "Fuzz targets found" and "Fuzz targets selected" then exiting
4 11:33 <instagibbs> I'll gladly improve the readme, if I just knew what the heck is supposed to be happening :)
7 13:39 <jonatack> and it runs indefinitely until halted with C-c or C-d
8 13:40 <jonatack> I'm not sure what a fuzz failure looks like yet, it just runs and runs :p
9 13:41 <instagibbs> hmmmmm
10 13:41 <instagibbs> mine definitely just stops
11 13:41 <instagibbs> with nothing useful
12 13:46 <instagibbs> jonatack, it's possible the readme is super poorly written and made me think it would run something, when all its doing is telling me what tests are available and exiting
13 13:48 <jonatack> yup definitely, improving fuzz.md has been on my todo list since last spring
14 13:49 <jonatack> just stopping seems wrong
15 13:50 <instagibbs> ok, ive upgraded to a subprocess command error message :)
16 13:50 <jonatack> i asked practicalswift when reviewing one of his fuzz prs and he confirmed that it just runs ad infinitum until halted
17 13:50 <instagibbs> somehow
18 13:50 <instagibbs> if i run specific ones like you did in that PR it works fwiw
19 13:50 <instagibbs> it's the python script the readme has that isn't working for me
20 13:52 <jonatack> hm, i recall the readme not working for me either, only following per-pr practicalswift instructions did
21 14:00 <instagibbs> actually scratch that, i actually broke it more, back to "nothing" for the `./test/fuzz/test_runner.py` incantation. If I specify a particular target, it says so and exits even faster
22 14:00 <instagibbs> :thinking:
23 14:01 <instagibbs> anyways I can at least run them one at a time thanks to your link :)
24 15:37 <jonatack> instagibbs: hmmm i retried that test runner script and am seeing the same as what you report
25 15:37 <jonatack> list of targets found and selected with: ./test/fuzz/test_runner.py ../qa-assets/fuzz_seed_corpus hex
26 15:38 <jonatack> or when no target supplied, then: subprocess.CalledProcessError: Command '['/home/jon/projects/bitcoin/bitcoin/src/test/fuzz/bech32', '-runs=1', '-detect_leaks=0', '../qa-assets/fuzz_seed_corpus/bech32']' returned non-zero exit status 1.
27 15:50 <instagibbs> make sure the path is right, i was getting no file found error, once i fixed the path it did nothing again
— Day changed Fri Jan 10 2020
28 17:12 <fjahr> instagibbs: are you on macos or at least using libFuzzer? I have the same problem you are describing. jonatack: how about you? AFL or libFuzzer?
— Day changed Sat Jan 11 2020
29 05:02 <jonatack> fjahr: i've been using libFuzzer so far and it works fine on individual fuzz tests so i'm probably using the python runner improperly
— Day changed Tue Jan 14 2020
— Day changed Wed Jan 15 2020
31 11:00 <pinheadmz> having a bit of trouble fuzzing on macos. i followed the guide in bitcoin/fuzzing and the comment about macos specifically.
32 11:00 <pinheadmz> getting "subprocess timed out: Currently only libFuzzer is supported"
33 11:00 <pinheadmz> the comment says "Take care of having an LLVM/Clang environment that contains fuzzing libraries. The default one from Apple is not enough, so that you will have to install it with brew, if not already done."
34 11:00 <pinheadmz> so, hm: brew install clang ?
35 11:01 <MarcoFalk_> The test_runner.py only works with libFuzzer
36 11:01 <MarcoFalk_> The test_runner.py is not a requirement for any of the questions in today's review club
37 11:01 <pinheadmz> I did clone and make AFL -- thats something different ?
38 11:02 <MarcoFalk_> It is mostly used by travis to read all seeds, not (yet) for generating seeds
39 11:02 <MarcoFalk_> Yes, afl is a bit different from libFuzzer
40 11:03 <pinheadmz> so the python isnt necessary - the fuzz tests will run with configure --enable-fuzz; make ?
41 11:04 <MarcoFalk_> no, you still need to run the target ./src/test/fuzz/utxo_total_supply
42 11:04 <MarcoFalk_> (with appropriate options)
43 11:04 <MarcoFalk_> afl needs some more memory, e.g. -m200
44 11:08 <pinheadmz> ok I see, libFuzzer for the test_runner, but individual tests can be executed with AFL, and theres a specific command for that in bitcoin/fuzzing.md
45 11:09 <MarcoFalk_> libFuzzer instrumented binaries can also be run individually
48 11:48 <pinheadmz> theres a Makefile option `LDFLAGS = -L/usr/local/lib/darwin/` but that directory doesnt exist on my computer, could it be referring to /usr/local/lib/ ? that directory has all my .dylib 's
49 11:52 <MarcoFalk_> Hmm, disappointing that mac is making it so hard to run fuzzers
50 11:53 <MarcoFalk_> We'll get started in about one hour (18 UTC)
51 11:53 <pinheadmz> Yeah :-/ i really wanna try and watch this run. Configure says "linker did not accept requested flags, you are missing required libraries" - is there a way to see whcih libs exactly are missing?
52 11:54 <MarcoFalk_> Did you switch to libFuzzer?
53 11:55 <pinheadmz> no still AFL
54 11:55 <raj_> pinheadmz: what exactly is the issue?
55 11:55 <MarcoFalk_> hmm. Not sure if I can help with afl or mac. Maybe pastebin the config log for others to take a look?
56 11:56 <pinheadmz> raj_: trying to follow docs/fuzzing.md cloned and make'd AFL, trying to run configure with CC=${AFLPATH}/afl-clang CXX=${AFLPATH}/afl-clang++
57 11:56 <pinheadmz> and trying different LDFLAGS since /usr/local/lib/darwin/ is nonexistnet
58 11:56 <pinheadmz> getting errors about missing libraries
59 11:57 <pinheadmz> ignoring the configure errors, making anyway and running afl-fuzz, getting "No instrumentation detected"
60 11:59 <raj_> have you tried `make distclean` and starting over?
61 11:59 <pinheadmz> i make clean each time ;-) is that sufficient?
62 12:00 <raj_> not very sure. When in doubt i just distclean. :p
63 12:00 <raj_> usually works.
64 12:01 <raj_> also check if you have afl-clang in path.
65 12:04 <pinheadmz> don't the CXX= flags handle that?
66 12:05 <raj_> it does provided you have setup $ALFPATH correctly. So thats what i meant to check.
67 12:06 <pinheadmz> yeah, like `ls ${AFLPATH}/afl-clang` returns the afl-clang executable
68 12:07 <pinheadmz> this missing libraries error is the thing i think is the culprit
69 12:08 <raj_> seems like be some dependencies are missing?
70 12:14 <raj_> have you tried libfuzzer? that worked with lesser issues for me.
71 12:21 <pinheadmz> yeah going down that rabbit hole next :-)
72 12:48 <fjahr> pinheadmz: I am on mac and having issues as well. I read somewhere that AFL is not supported on mac, so I have been trying libfuzzer but I am stuck with fuzz binaries getting stuck without output (see my posts a few days ago here in the channel and in #bitcoin-core-dev).
73 12:49 <MarcoFalk_> Is it not possible to install a vanilla clang on MacOS?
74 12:50 <fjahr> I am using a non-systems clang installed via brew, if that's what you mean
75 12:51 <MarcoFalk_> Ah, so compilation worked, but the output is empty?
76 12:52 <fjahr> yeah, the fuzz bin starts but just doesn't do anything, no output and it doesn't shut down either
77 12:52 <fjahr> even when running with `-help=1`
78 12:56 <pinheadmz> fjahr: shoot me a link to install libfuzzer on osx ?
79 12:57 <MarcoFalk_> It might be bundeled with the clang+llvm version shipped in brew
80 12:57 <fjahr> it's included in llvm/clang if you install via brew
81 12:57 <fjahr> just not in the systems one
82 13:00 <MarcoFalk_> hi everyone. Let's get started. It sounds like some people had technical problems getting the fuzzer to run
83 13:00 <jnewbery> #startmeeting
92 13:01 <TheBigCohooNah> lol, hi, going to listen in and not understand anything =)
94 13:01 <MarcoFalk_> Has everyone had a chance to look at the pull request?
101 13:02 <MarcoFalk_> Nice
102 13:03 <MarcoFalk_> Ok, so let's start with some general questions. Anyone want to explain the difference between a black-box and a white-box fuzzer?
103 13:03 <jonatack> Black-box fuzzing, aka i/o-driven or data-driven testing, includes most
104 13:03 <jonatack> Btraditional fuzzers and is blind to the internals of the PUT (program under test),
105 13:03 <jonatack> observing only the i/o behavior and treating it as a black-box.
106 13:03 <emzy> white-box is with source code
107 13:04 <jonatack> White-box fuzzing generates test cases by analysing the internals of the PUT and the information gathered when executing the PUT.
108 13:04 <raj_> black-box: doesnt look into internals. white-box: looks at internals and takes test decisions
109 13:04 <MarcoFalk_> So what is coverage guided fuzzing and how does it fit in to the black/white categories?
110 13:05 <andrewtoth_> gray-box?
111 13:05 <jonatack> Grey to black
112 13:05 <MarcoFalk_> libFuzzer and AFL are both coverage guided and that is what Bitcoin Core uses, so we will focus on these
113 13:05 <MarcoFalk_> How does the coverage guiding work?
114 13:06 <andrewtoth_> a function is exposed in the harness to allow fuzz data to get passed into it
115 13:06 <jonatack> Guided by API coverage?
116 13:07 <jonatack> API meaning certain functions put under test.
117 13:07 <MarcoFalk_> Yes, some part of the interface is exposed in a fuzz target
118 13:07 <jonatack> idea is FDD turst APIs into fuzz targets
119 13:07 <raj_> anything to do with code path?
120 13:07 <jonatack> turns
121 13:07 <MarcoFalk_> But how does the coverage guiding help the fuzzer to fuzz that function?
122 13:07 <jonatack> FDD (fuzz-driven development :D)
123 13:08 <MarcoFalk_> raj_: Yes, something with code path :)
124 13:08 <raj_> i am struggling with the concept of coverage. What does it means exactly? Is it like a set of all possible execution paths?
125 13:09 <MarcoFalk_> coverage can mean many things. Common ones are "function coverage" "line coverage" or "branch coverage"
128 13:10 <MarcoFalk_> Let's not go into the details of what kind of coverage the fuzzers use, but discuss the general idea
129 13:10 <andrewtoth_> ok, but then what is coverage "guiding"? Is that just exposing functions to the fuzzer and mutating the fuzz data to be a proper input?
130 13:10 <raj_> jonatack: really cool. thanks..
131 13:12 <MarcoFalk_> Ok, let's take a step back. What is a "seed"?
132 13:12 <raj_> initial set of test cases.
133 13:12 <MarcoFalk_> raj_: right
134 13:12 <MarcoFalk_> What happens with a "seed"?
135 13:13 <andrewtoth_> it's given to the fuzzer
136 13:13 <fjahr> i guess it is saved and different inputs are generated from it so that it can be used to replay the results if an error is detected
137 13:14 <MarcoFalk_> andrewtoth_: Right
138 13:14 <MarcoFalk_> fjahr: Right
139 13:15 <MarcoFalk_> Ok, and when compiling with a fuzzer, the binary is instrumented with the fuzz engine and some annotations to collect coverage data
141 13:16 <MarcoFalk_> For each input (or seed) coverage data is collected when run
142 13:16 <MarcoFalk_> How does the coverage data help the fuzzer?
143 13:16 <felixweis> how did you generate the fuzz_seed_corpus?
144 13:17 <raj_> gives the fuzzer clue on how to increase coverage in next iteration?
145 13:17 <fjahr> it shows if new branches were discovered with the given seed or not
146 13:17 <MarcoFalk_> fjahr: Correct
147 13:17 <MarcoFalk_> Does it influence the future decisions of the fuzzer? What happens with a seed that did or did not increase coverage?
148 13:18 <andrewtoth_> that would depend on the fuzzer no?
149 13:18 <felixweis> also the filenames are random or simply hashes of the file contents
150 13:18 <MarcoFalk_> andrewtoth_: Yes. (Assuming AFL and libFuzzer for now, i.e. coverage-guided fuzzers)
151 13:19 <MarcoFalk_> felixweis: The name of the seeds or inputs depends on the fuzzer
152 13:20 <jonatack> In this case, yes, seed trimming takes place to use the coverage to progressively remove a portion of the seed as long as the coverage remains constant
153 13:21 <pinheadmz> Is the data sent to bitcoind totally random or can you give it structure ie. transaction format with some limits on field size and value etc
154 13:21 <felixweis> if an input increases coverage it is saved to the directory of the other inputs. otherwise it is discarded
155 13:21 <MarcoFalk_> jonatack: Correct. Coverage is useful to minimize a seed corpus or even a single seed
156 13:23 <MarcoFalk_> pinheadmz: The data or inputs are not sent to bitcoind they are sent to the fuzz target (which is a separate binary, but linked with all the bitcoind libraries)
157 13:23 <MarcoFalk_> felixweis: correct
158 13:24 <MarcoFalk_> And to answer felix's question how the initial set of inputs is created: AFL needs an initial set, which is recommended to be generated by hand. libFuzzer does not, it can start with an empty string as the first seed
159 13:25 <MarcoFalk_> Any other questions?
160 13:25 <felixweis> but that search space is just too enormous
161 13:25 <MarcoFalk_> felixweis: Yes, the search space is generally never exhausted
162 13:25 <raj_> for this the seed is "zzzii"
163 13:25 <andrewtoth_> so how does this PR guide the fuzzer exactly?
164 13:25 <raj_> does it mean anything, or just random?
165 13:26 <MarcoFalk_> felixweis: However, with coverage data, at least it can span the relevant parts relatively quickly
166 13:26 <raj_> for `utxo_total_supply` target i mean.
167 13:26 <andrewtoth_> does that happen automatically in utxo_total_supply? it knows which branches have been taken?
168 13:28 <MarcoFalk_> andrewtoth_: Yes, the fuzz engine is hidden in AFL or libFuzzer and with the coverage instrumentation it can figure out by itself what branches have been taken
169 13:28 <MarcoFalk_> raj_: We will get into that seed later
171 13:29 <MarcoFalk_> Ok, if there are no further questions about the general idea of coverage guided fuzzing, we will jump into the idea of the CVE.
172 13:29 <MarcoFalk_> How can CVE-2018-17144 be exploited?
173 13:29 <pinheadmz> a single tx spending from the smae input twice
174 13:29 <MarcoFalk_> Can you explain conceptually how to create an example block that exploits the bug?
175 13:29 <MarcoFalk_> pinheadmz: Correct
176 13:29 <pinheadmz> it happened on testnet!
177 13:29 <pinheadmz> and there are still testnet nods stuck on that fork from like 18 months ago
178 13:30 <andrewtoth_> but only when included in a block, will be rejected from mempool
179 13:30 <MarcoFalk_> andrewtoth_: Correct
180 13:30 <pinheadmz> luke-jr mentioned ot me that the bug is so bad, the bad block can't even be re-org'ed out
181 13:31 <MarcoFalk_> ok, looking into the code of the newly added fuzz target.
182 13:31 <MarcoFalk_> What does the new fuzz test do on a high level? What “Actions” can it run?
183 13:31 <pinheadmz> so for this bug specifically, what are the benefits of a fuzz test instead of jsut an explicit test in the python suite?
184 13:31 <MarcoFalk_> pinheadmz: Good q. We do have a test in the python test suite
185 13:32 <MarcoFalk_> However, a fuzz test is more flexible in the sense that it can also actively search for similar CVEs
186 13:33 <jonatack> Fuzzing might find cases the test-writer did not think of.
187 13:33 <MarcoFalk_> The CVE showed different behavior when the duplicate inputs were created in the same block, vs created in an earlier block
188 13:33 <MarcoFalk_> jonatack: correct
189 13:33 <raj_> 3 actions. Crate input, Create tx and Create Block
190 13:33 <MarcoFalk_> So when the python test only checks for the case where the inputs were confirmed, the fuzz test might find the case where the inputs were created in the same block
191 13:34 <MarcoFalk_> Or even find a case that was not imagined by anyone
192 13:34 <MarcoFalk_> raj_: Correct
193 13:34 <pinheadmz> and maybe im underestimating the size of the fuzz test, bt with random data, what is the probability of getting anything useful like that? especially with such a dense protocol like bitcoin
194 13:34 <raj_> jus to clarify if i got it correctly. The bug is one utxo being spent by 2 different inputs in a same tx?
195 13:35 <MarcoFalk_> raj_: Yes
196 13:35 <andrewtoth_> pinheadmz i believe the goal is to have these tests run continuously with lots of cpus, to find a very large space of inputs
197 13:35 <jnewbery> pinheadmz: I think that goes back to your previous question about whether the fuzzed data is totally random or whether it has some structure. Perhaps we could talk about that?
199 13:35 <MarcoFalk_> jnewbery: Yes, thx
200 13:36 <pinheadmz> yeah thanks
201 13:36 <andrewtoth_> raj_ the bug is one tx spending the same utxo twice
202 13:36 <MarcoFalk_> So is the fuzzer sending raw blocks?
204 13:37 <MarcoFalk_> As mentioned by raj_ the fuzzer has 3 actions. Crate input, Create tx and Create Block
205 13:37 <MarcoFalk_> Why would it not be feasible to send a raw block?
206 13:38 <jnewbery> It looks to me like the fuzzer is providing entropy which is being used by utxo_total_supply.cpp to generate inputs
207 13:38 <MarcoFalk_> jnewbery: Correct
208 13:38 <jnewbery> anything that calls fuzzed_data_provider.ConsumeIntegralInRange is taking that entropy and generating stuff with it
209 13:38 <MarcoFalk_> But why is it not possible to just treat the input seed as a raw block?
210 13:39 <jnewbery> one reason is that generating blocks is difficult because they need proof-of-work!
211 13:39 <jnewbery> apart from all the structure that blocks have
212 13:40 <MarcoFalk_> jnewbery: Correct. Also, blocks are highly structured. E.g. the transactions need to deserialize correctly and match the merkle root
213 13:40 <raj_> `const auto action = static_cast<Action>(fuzzed_data_provider.ConsumeIntegralInRange(0, 2));` How is an integer being casted into the enums? is it like 0 for Create Input, 1 for Create tx?
214 13:40 <MarcoFalk_> raj_: I think in c++ you can cast integers to non-scoped enums
215 13:40 <pinheadmz> Structuring the entropy with minimal formatting makes sense
216 13:41 <pinheadmz> so the input scripts are randomly generated though? and we just hope to brute force enough of them that something interesting happens?
217 13:41 <MarcoFalk_> So looking at my example seed zzzii, what does it do?
218 13:41 <MarcoFalk_> How many blocks are mined? Either explain by looking at the seed and code or obtain the result by modifying and running the fuzz test on the seed.
219 13:43 <pinheadmz> 2000 blocks
220 13:43 <pinheadmz> 20 * coinbase maturity (which is 100)
221 13:43 <jnewbery> is "zzzii" ascii? 0x7a7a7a6969?
222 13:44 <raj_> would really appreciate a walk through of this.
223 13:44 <MarcoFalk_> jnewbery: Yes, this happens to be ascii. In general seeds are not ascii, though
224 13:45 <MarcoFalk_> Hint: The fuzzer reads this byte by byte from the right
225 13:45 <MarcoFalk_> The first read is here
226 13:45 <MarcoFalk_> int64_t duplicate_coinbase_height = fuzzed_data_provider.ConsumeIntegralInRange(0, 20 * COINBASE_MATURITY);
227 13:46 <pinheadmz> while (fuzzed_data_provider.remaining_bytes())
228 13:46 <pinheadmz> trying to see where the data provider gets initialized
229 13:47 <andrewtoth_> 105?
231 13:47 <MarcoFalk_> andrewtoth_: why?
232 13:47 <pinheadmz> and the buffer data is that 5-char string ?
233 13:47 <MarcoFalk_> pinheadmz: Yes, in this case the seed is only 5-chars
234 13:48 <MarcoFalk_> or let's say 5 bytes
235 13:48 <andrewtoth_> ConsumeIntegralInRange just gets the first byte, checking if it's in the range?
236 13:48 <andrewtoth_> so first byte is 105?
237 13:48 <dude> Converts the byte to range
238 13:48 <MarcoFalk_> andrewtoth_: It will read as much bytes as it needs to reach the upper range
239 13:49 <pinheadmz> ah, then the int in range(0, 2) becomes a command: input / tx / block
240 13:49 <pinheadmz> so its like a little opcode script !
241 13:49 <MarcoFalk_> So how many bytes does it read for duplicate_coinbase_height?
242 13:49 <pinheadmz> just one ?
244 13:50 <MarcoFalk_> pinheadmz: What is the upper bound?
247 13:50 <MarcoFalk_> hint: COINBASE_MATURITY is 100
248 13:50 <pinheadmz> oh i see it consumes as many bytes as it needs to find something in the range (0, 2000
249 13:50 <andrewtoth_> if upper bound is 20000, then upper bound is 20000?
250 13:50 <MarcoFalk_> pinheadmz: Yes!
251 13:50 <dude> Wait are you sure?
252 13:51 <dude> I thought it modded the input to fit into the range
253 13:51 <pinheadmz> so thats what 2 bytes? max value is 65535
254 13:51 <MarcoFalk_> How many bytes are needed to represent a random number up to 2000?
255 13:51 <MarcoFalk_> pinheadmz: Yes
256 13:51 <MarcoFalk_> Ok, so the "ii" in the seed gets deserialized as the duplicate_coinbase_height
257 13:52 <MarcoFalk_> What is the next read of the fuzzer?
258 13:52 <andrewtoth_> ii is 6969, so that's 26985, greater than 2000 though...
259 13:52 <pinheadmz> while there's bytes remaining, it grabs one at a time and runs the corresponding Action::...
260 13:52 <MarcoFalk_> andrewtoth_: Yes, it is wrapped by the fuzzed data provider
262 13:52 <MarcoFalk_> dude: Correct
263 13:53 <MarcoFalk_> thx for the link
264 13:53 <MarcoFalk_> So what is the next place where bytes are consumed by the fuzzer?
265 13:53 <raj_> so duplicate_coinbase_height is always 0-2000 right?
266 13:53 <MarcoFalk_> yup, in this fuzz target
267 13:54 <jnewbery> MarcoFalke: so the coinbase height is 985?
268 13:54 <pinheadmz> MarcoFalk_: L112?
269 13:54 <MarcoFalk_> pinheadmz: Correct
270 13:54 <pinheadmz> the last 3 bytes are interpreted as actions?
272 13:55 <MarcoFalk_> Jup
273 13:55 <jnewbery> Jup to height or Jup to actions?
276 13:56 <MarcoFalk_> jnewbery: Seems plausible. Haven't checked, but you can with std::cout << duplicate_coinbase_height << std::endl; and running this seed
277 13:56 <dude> Can answer later if it's a tangent
278 13:56 <MarcoFalk_> So how many blocks are mined?
279 13:56 <MarcoFalk_> by the seed "zzzii"?
280 13:57 <pinheadmz> 985? 0x6969 % 2000
281 13:57 <MarcoFalk_> Oh, that is just the duplicate coinbase height. Not used to determine the number of blocks mined.
282 13:58 <MarcoFalk_> Blocks are only mined when the action is CREATE_BLOCK
283 13:58 <raj_> 'z's are ascii 122. so what action they translates into??
284 13:58 <raj_> thats outside (0,2)
285 13:58 <pinheadmz> 122 % 2 = 0
286 13:59 <pinheadmz> which is create_input
288 13:59 <raj_> so just create input three times?
289 13:59 <MarcoFalk_> 0x7a % 3 = 2
290 13:59 <andrewtoth_> 0 blocks is the answer? hmm...
292 13:59 <pinheadmz> MarcoFalk_: since you know the seeds will be modded like this anyway, why not just use actual bytes 0x00 0x01 0x02 ?
293 13:59 <jnewbery> it's got to be modded by (range + 1) because the index starts at 0
294 14:00 <andrewtoth_> ahh it's modded by 3
295 14:00 <pinheadmz> jnewbery: ah ty!
296 14:00 <andrewtoth_> so 3 blocks are mined
297 14:00 <MarcoFalk_> pinheadmz: It was created by libFuzzer, which does not know that it is being modded anyway
298 14:00 <MarcoFalk_> andrewtoth_: Correct!
299 14:00 <andrewtoth_> this is a confusing PR...
300 14:00 <pinheadmz> its great haha
301 14:00 <dude> If you just have switch statements for 0x00, 0x01, 0x02, the fuzzer will generate bytes outside of the range and not get modded
302 14:00 <dude> So it's more efficient that way
303 14:00 <MarcoFalk_> last q: Did anyone run the fuzzer?
305 14:00 <MarcoFalk_> Did it find the CVE?
307 14:00 <pinheadmz> did anyone git it to run on OSX by any chance?
308 14:00 <dude> Eh, not sure, was there an assert I had to disable?
309 14:01 <dude> Yeah I got it running on OSX
310 14:01 <andrewtoth_> i couldn't get AFL to work, similar to raj_'s comment it kept crashing
311 14:01 <dude> Did you turn off AFL_NO_FORKSRV?
312 14:01 <dude> or I mean "on"
313 14:01 <MarcoFalk_> raj_: Nice
314 14:01 <pinheadmz> dude: yah, did you use AFL or libFuzzer ?
316 14:01 <andrewtoth_> didn't know you could build and run and then just `src/test/fuzz/utxo_total_supply`, could have some docs for that
318 14:01 <raj_> I tried with libfuzzer. Ran 5 workers for 24 hours. didint find any crash.
319 14:01 <andrewtoth_> haven't tried AFL_NO_FORKSRV, but i'm running linux
322 14:02 <raj_> 4 of them stoped at times out.
323 14:02 <MarcoFalk_> Note that the CVE was fixed, so it has to be reintroduced first ;)
324 14:02 <emzy> I was runnig it on linux. And I found after 2h nothing.
325 14:02 <dude> Timeout can usually just be erroneous, you would need to double-check
326 14:02 <MarcoFalk_> How would you reintroduce the CVE?
327 14:02 <andrewtoth_> i tried running afl with -m200 but same error for me
328 14:02 <raj_> got few slow-unit test artifacts. not sure what they are..
329 14:02 <raj_> but no cve..
330 14:03 <andrewtoth_> revert the PR that fixed it, as well I think there was a PR that removed the bool flag entirely
331 14:03 <jnewbery> revert 14247
332 14:03 <emzy> andrewtoth_: -m200 woked for me
333 14:03 <MarcoFalk_> Yes
335 14:04 <MarcoFalk_> Let's wrap up
336 14:04 <MarcoFalk_> We can chat about technical issues on MacOS later
337 14:04 <MarcoFalk_> Any last questions?
338 14:04 <pinheadmz> lol at "deadly signal"
339 14:04 <pinheadmz> This was really cool thanks MarcoFalk_
340 14:04 <jnewbery> Thanks MarcoFalk_! Before everyone goes, what did you all think? It sounds like there were a few issues getting fuzzing set up. Would you all like to do another session on fuzzing?
341 14:05 <dude> Yes! but need better docs
342 14:05 <MarcoFalk_> dude: Agree
344 14:05 <MarcoFalk_> Unfortunately I don't have MacOs, so I can't provide documenation
345 14:05 <andrewtoth_> yes, more fuzzing docs please
346 14:05 <dude> I have MacOS
347 14:05 <andrewtoth_> i'm running on linux and couldn't get afl working
348 14:05 <jonatack> Yes, I've had rewriting fuzzing.md on my list since last Spring
349 14:05 <jnewbery> I think improving the docs (including for MacOS) would be an excellent contribution from any one of you
350 14:05 <andrewtoth_> MarcoFalk_ is the seed 7P9p/sc7BQHvJto7OzsyZZKSkpKSklBQUFBQUFBQUFBQUFCSkpKSkpI+Pj4+Pj4+PpJQUFBQUFBQ
351 14:05 <andrewtoth_> UFBQUFBQkpKSkpKSPj4+Pj4+Pj7smGH7xdR67Ho7j4M7L2VwO/tidNQvNfX19fXsmGH7xdR67Ho7
352 14:05 <andrewtoth_> j4M7L2VwO/tidNQvNfX19fU=?
353 14:06 <raj_> jnewbery: yes would love to. It seems like an important concept. would love to learn more. Thanks for the PR. it was a great one, have learnt a lot in last week.
354 14:06 <jonatack> andrewtoth_: same, i use libFuzzer
355 14:06 <jnewbery> anyone who's set up fuzzing for the first time this week is best placed to improve the docs since you've probably recently run into all the problems and gotchas
356 14:06 <andrewtoth_> I'll have to get fuzzing working well first before I can document how to do it ;)
357 14:06 <MarcoFalk_> jnewbery: Great suggestion
358 14:07 <jnewbery> #endmeeting
359 14:07 <andrewtoth_> thanks MarcoFalk_ and everyone!
360 14:07 <emzy> thanks MarcoFalk_ and everyone!
363 14:08 <jonatack> thanks MarcoFalk_, excellent meeting and study resources. Learning a lot.
364 14:08 <dude> MarcoFalk_, do you run afl-tmin, afl-cmin on your corpus?
365 14:08 <MarcoFalk_> If you find the docs hard, please also file an issue or improve them :)