Calculate UTXO set hash using Muhash (utils/log/libs)

https://github.com/bitcoin/bitcoin/pulls/19055

Host: fjahr  -  PR authors: fjahr , sipa

Notes

PR history

  • The idea to use Muhash in Bitcoin Core was initially introduced by Pieter Wuille in this mailing list post from 2017.

  • Pieter proposed an implementation in PR #10434, which still comprises most of the code in this week’s PR.

  • Fabian Jahr then picked up the proposal and made further research in 2019. A snapshot of the work can be seen in this gist.

  • Based on further insights and feedback, the idea evolved into implementing an index for all coin statistics and not only for the hash of the UTXO set. This was implemented in PR #18000.

  • This week’s PR is the first in a series of PRs to incrementally implement PR #18000.

PR 19055

  • This PR modifies how the hash of the UTXO set is calculated. That hash, as well as other coin statistics, can be accessed through the gettxoutsetinfo RPC. It uses the Muhash algorithm which allows for incremental hashing.

  • The truncated hash SHA512/256 is used to prepare data for use in Muhash.

Questions

  1. What does “incremental hashing” mean? Which of its properties are interesting for our use case?

  2. What were the use cases for RPC gettxoutsetinfo described in the resource links? Can you think of others? Which use cases are the most relevant to you?

  3. What do you think could be the biggest downside to introducing Muhash for the UTXO set hash?

  4. This PR not only adds Muhash but also TruncatedSHA256Writer. Why?

  5. Why is the Muhash class implemented in Bitcoin Core and not in libsecp256k1?

  6. Did you look into the other proposed hashing algorithms in some of the resources? What were their drawbacks? Do you think Muhash is the right choice?

  7. What are your thoughts on the Muhash benchmarks, e.g. (a) the benchmarking code, and (b) your observations if you tested the code locally?

  8. Considering the trade-offs, should the old hashing algorithm be kept around and accessible (using a flag, for example)?