Introduce internal kernel library (build system)

https://github.com/bitcoin/bitcoin/pull/28690

Host: stickies-v  -  PR author: TheCharlatan

2023-11-29 edit: the questions and references have been updated slightly to reflect the latest force-push to 5a6a0b1693cf739a5e6cc1161160a1d621d215f9

Notes

  • The libbitcoinkernel project is an effort to decouple Bitcoin Core’s consensus engine from other non-consensus modules in the codebase. We have previously covered libbitcoinkernel-related PRs #27711, #25527, #24410 and #20158. However, they are not essential to understanding this PR as they focused more on internal code reorganization as opposed to this PR’s focus on the build system.

  • Suggested reading: Static and dynamic libraries (up until the “Installing and using libraries” section which is not relevant).

  • Most of the libraries (required reading) that are built are internal and static, and can’t be used outside of Bitcoin Core. Examples of internal libraries are libbitcoin_cli, libbitcoin_common and libbitcoin_node. There are only two external (dynamic, or shared) libraries (required reading): libbitcoinconsensus and libbitcoinkernel, even though the latter is not currently documented as such since the API is not ready.

  • This PR introduces a new internal libbitcoin_kernel static library, in addition to the existing external libbitoincoinkernel dynamic library.

  • Internal libraries are solely used to modularize the build system, which helps with build performance as well as maintainability. In the future, the Bitcoin Core build system may start to use the external libbitcoinkernel library, but that would probably require more kernel work (such as having a more complete and stable API) to be completed first.

  • An explanation of some abbreviations used:

    • a vs la(as in libbitcoin_kernel_a_SOURCES and libbitcoinkernel_la_SOURCES):
      • a stands for archive (i.e. a static library)
      • la stands for Libtool Archive

Questions

  1. Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK? What was your review approach?

  2. Did you read the Static and dynamic libraries, libraries.md and shared-libraries.md documentation?

  3. Are the Bitcoin Core internal libraries all statically or dynamically built, or a mix of both? Why?

  4. Why do we need to build external libraries? What’s the purpose of having both an internal libbitcoin_kernel and an external libbitoinkernel? Why do we build external libraries in the first place?

  5. Which of the libraries {libbitcoin_cli, libbitcoin_consensus, libbitcoinconsensus, libbitcoin_kernel, libbitcoinqt} are internal?

  6. How are we using kernel functionality in Bitcoin Core before this PR, if there is no internal library?

  7. Why does the libbitcoinkernel_la_SOURCES source file list specifically include $(libbitcoin_util_a_SOURCES) and $(libbitcoin_consensus_a_SOURCES) but libbitcoin_kernel_a_SOURCES doesn’t seem to?

  8. Commit 41a80de mentions: “Moving util/message.cpp to the common library also breaks an undocumented dependency of the util library on the consensus library’s CPubKey::RecoverCompact symbol.”. What was this dependency?

  9. Commit 5a6a0b1 de-duplicates the source file list. Are there any changes to the source file list used for libbitcoinkernel?