zmq)

Dec 22, 2021

https://github.com/bitcoin/bitcoin/pull/10102

The PR branch HEAD was 8a18a12a232 at the time of this review club meeting.

Notes

Currently Bitcoin Core code always runs within a single operating system process. So if you are running bitcoind, then you are running node and wallet code together in the same process. If you are running bitcoin-qt, then you are running node, wallet, and GUI code in the same process. Running all code in one process does not provide a lot of isolation, so it means a crash in wallet code could bring down the node, or a vulnerablility in the node could expose wallet data. Additionally it’s not possible to run the node and wallet on different machines, or to stop and start the GUI independently of the node, or use the GUI to connect to nodes or wallets on headless machines.

PR #10102 starts to move away from the single process model by adding basic support needed for bitcoin node, wallet, and GUI code to run in different processes and communicate with each other internally. Followups #19460 and #19461 expand on it to allow wallet and GUI processes to be started and stopped independently, and allow the processes to communicate with each other externally, and run on different machines.

PR #10102, combined with the --enable-multiprocess build option, builds a new bitcoin-node executable that can be used as a drop-in replacement for bitcoind, and a new bitcoin-gui executable that can be used a drop-in replacement of bitcoin-qt. The new executables are used the same way as the previous ones and don’t provide any new features externally, but internally they will spawn multiple processes and use process separation. Instructions for building and testing #10102 can be found in doc/multiprocess.md and can be done in a few commands on linux:

cd <BITCOIN_SOURCE_DIRECTORY>
make -C depends NO_QT=1 MULTIPROCESS=1
CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
make
src/bitcoin-node -regtest -printtoconsole -debug=ipc

When this is run, bitcoin-node will spawn a bitcoin-wallet executable to run wallet code. The node and wallet processes will communicate across a socket, with the node code controlling the wallet with interfaces::WalletClient methods, and the wallet code calling the node with interfaces::Chain methods. Similarly when GUI support is enabled, bitcoin-gui will spawn a bitcoin-node process, and control it by calling interfaces::Node methods, and control wallets by calling interface::Wallet methods.

#10102 adds the plumbing that allows gui <-> node <-> wallet cross-process communication to work transparently, without changing existing code, only adding new code. This is possible because the interfaces in src/interfaces/ (interfaces::Chain, interfaces::Node, interfaces::Wallet described above) are all abstract classes with virtual methods, so different implementations can be substituted without changes to calling code. This PR adds new implementations of each interface that forward method calls from one process to another. In early versions of this PR, adding new implementations of each interface required adding a lot of boilerplate code. Every method in every interface, and every argument and return value of every method had to had custom C++ code written that would handle the method call serialization and forwarding from the calling process to the called process. For example, the Wallet::encryptWallet abstract method:

    virtual bool encryptWallet(const SecureString& wallet_passphrase) = 0;

had an ipc::capnp::WalletImpl::encryptWallet implmentation which forwarded the encrypt call from the gui process to the wallet process using the Cap’n Proto RPC framework:

    bool encryptWallet(const SecureString& wallet_passphrase) override
    {
        auto call = MakeCall(m_loop, [&]() {
            auto request = m_client.encryptWalletRequest();
            request.setWalletPassphrase(ToArray(wallet_passphrase));
            return request;
        });
        return call.send([&]() { return call.m_response->getResult(); });
    }

with corresponding code in an ipc::capnp::WalletServer::encryptWallet method in the wallet process to handle the incoming RPC:

    kj::Promise<void> encryptWallet(EncryptWalletContext context) override
    {
        context.getResults().setResult(
            m_impl->encryptWallet(ToSecureString(context.getParams().getWalletPassphrase())));
        return kj::READY_NOW;
    }

by forwarding it to the local ipc::local::WalletImpl::encryptWallet method:

    bool encryptWallet(const SecureString& wallet_passphrase) override
    {
        return m_wallet.EncryptWallet(wallet_passphrase);
    }

which is the same method that would have been called directly if code were running in a single process instead of multiple processes.

Because the ipc::capnp::WalletImpl::encryptWallet and ipc::capnp::WalletServer::encryptWallet methods above and all similar methods just contain boilerplate code forwarding arguments and return values, newer versions of this PR no longer define these methods manually, and instead generate them automatically. (You can see these methods in generated src/ipc/capnp/wallet.capnp.proxy-client.c++ and src/ipc/capnp/wallet.capnp.proxy-server.c++ files after building this PR, but they are not part of the PR source code). The code generation means that adding a new method or changing an existing method signature now just requires editing a single line in the interface’s .capnp file:

    encryptWallet @1 (context :Proxy.Context, walletPassphrase :Data) -> (result :Bool);

and the corresponding forwarding code is generated from that.

Library support for everything described above was merged previously in #19160 Multiprocess: Add basic spawn and IPC support (review club), so the most significant part of #10102 is just adding .capnp interfaces for all the interfaces in src/ipc/capnp/.

#10102 is a large PR and it is divided into multiple commits.

The most significant commits are the “Add canp serialization…” and “Add capnp wrapper…” commits which add Cap’n Proto schema definitions describing Bitcoin’s struct and interface types that are shared between processes: (common.capnp, chain.capnp, node.capnp, wallet.capnp). These commits also add a lot of glue code in BuildField / ReadField/ BuildMessage / ReadMessage function. There is a lot of ugly C++ template syntax needed to declare the function types, but the actual function bodies are pretty straightforward and just copy information between Bitcoin Core’s native data structures and corresponding capnp messages.

The other two significant commits in this PR are the “Make bitcoin-gui spawn a bitcoin-node process” commit and “Make bitcoin-node spawn a bitcoin-wallet process” commit. These commits don’t add much new code. Instead they strip out wallet initialiatation code from the BitcoinNodeInit class (so the bitcoin-node process no longer has wallet functionality linked in) and strip out node and wallet initialization code from the BitcoinGuiInit class (so the bitcoin-gui class no longer has node and wallet functionality linked in. Then they add spawnProcess calls in appropriate places to start making cross-process calls. A good entry point into these two commits is to search for the new spawnProcess calls they add.

The other commits in this PR are needed for completeness but aren’t related to its core functionality. They are smaller and just update dependencies, logging, and test code.