This PR is the latest in a sequence of PRs to clean up the node-wallet
We previously reviewed PR
15713 in Bitcoin Core
review club. See the notes and logs for that meeting for more information
about the interface and the recent work to tidy it up.
One of the main goals of that work is to remove the wallet’s ability to lock
cs_main. PR 16426 is a
proof-of-concept PR which does that.
This PR is a big step towards removing the wallet’s ability (and requirement)
to lock cs_main. It removes the locked_chain dependency from the
For a given wallet transaction, GetDepthInMainChain() returns how many
confirmations that transaction has in the block chain.
When a wallet transaction is included in a block, the block’s hash is stored
in the CWalletTx object (see hashBlock and SetMerkleBranch().
GetDepthInMainChain() previously worked by taking that hashBlock and
checking its depth in the block chain. That requires locking cs_main since
block chain state is being accessed.
After this PR, each wallet transaction stores the height of the block that
it was confirmed in, and the wallet stores the height of the best block
in the block chain. By storing these values internally, the wallet no
longer needs to query the block chain state to calculate the transaction’s
number of confirmation.
How does the wallet learn about new transactions in the mempool or included
What are the wallet’s expectations about block notifications? Is it a problem
if the wallet is informed of the same block more than once? If blocks arrive
in the wrong order? If a block is skipped? If a block is re-orged out of the
<jnewbery> right - the PR author pulled out a small part of this PR and made it into its own PR. I'd already chosen 15931 for today's discussion and didn't want to change it in case you'd all started reviewing it
<michaelfolkson> It is certainly one that is interlocking with a bunch of other PRs and requires some organizational roadmap so they are merged in the right order. Seems like a high quality PR by itself.
<nehan> jnewbery: no, but looking forward to discussing. the goal is great, but given i'm pretty unfamiliar with the wallet code i can't convince myself that these PRs don't change behavior (or that the behavior they change is 'safe')
<jnewbery> for PRs like 15931 and 16624, where there are changes in serialization (or at least in the way we deserialize and hold data at runtime), being able to do regression testing on old wallet files would be really useful
<jnewbery> ok, next question: The PR author offers two ways for the wallet to populate the wallet transactions’ heights (save the transaction height to disk or calculate the height for each transactions at wallet load time). What are the trade-offs? Which approach do you prefer?
<jnewbery> jonatack: quite difficult. We absolutely need to remain compatible with old wallet.dat files, and it's also important that new wallet.dat files are compatible with old versions, so we couldn't just change it completely
<jnewbery> if we were designing it from scratch, we might use a type-length-value scheme so wallet software could just ignore fields that it doesn't recognize, but we're constrained by all the existing wallet.dat files already in existence
<jnewbery> lightlike: going back to your previous point: all of the wallet callbacks come from CValidationInterface functions. I think there are only 5 of those methods that are overridden by the wallet: TransactionAddedToMempool, TransactionRemovedFromMempool, BlockConnected, BlockDisconnected, UpdatedBlockTip, ChainStateFlushed
<jnewbery> I haven't seen any answers to: The PR author offers two ways for the wallet to populate the wallet transactions’ heights (save the transaction height to disk or calculate the height for each transactions at wallet load time). What are the trade-offs? Which approach do you prefer?
<provoostenator> @michaelfolkson at the moment it tests back to 0.17.1, but more could be added. The main drawback is having too many switch statements in Python test framework to deal with ancient RPC stuff
<jnewbery> to add some more detail: the wallet has a 'locator', which is a sparse list of blockhashes in its view of the block chain. It uses that to try to find a fork point with the nodes view of the block chain at startup
<hugohn> to me the decision to persist to disk or not probably shouldn't impact how the wallet reacts to reorgs - one is a memory management issue, one is a consensus issue. reorgs could happen _after_ we have recomputed the height at startup anyway, so the issues should be completely orthogonal. but I could be wrong.
<jnewbery> bcribles: not directly, but it would know that there had been a re-org since it went offline, and therefore rescan the block chain from a height where it knows it shares history with the node
<jnewbery> if a re-org happened when the wallet is online, we'd expect the node to inform the wallet of the blocks being rewound with BlockDisconnected calls, followed by BlockConnected calls for the new chain
<jnewbery> ok, final question: What are the wallet’s expectations about block notifications? Is it a problem if the wallet is informed of the same block more than once? If blocks arrive in the wrong order? If a block is skipped? If a block is re-orged out of the main chain?
<lightlike> A comment in ValidationInterface says that it is guaranteed that calls arrive in the same order as events are generated in validation - in that case, how is it possible that a block is skipped?