The PR branch HEAD was 200a49f at the time of this review club meeting.
Notes
Fees for the wallet are calculated using a fee estimator attached to the node’s mempool,
CBlockPolicyEstimator.
At a high level, it records the feerate and time at which transactions enter the mempool and
how many blocks later they confirm. Later, it uses this information to provide the opposite
result: it provides a feerate estimate based on a given confirmation target (measured by how
many blocks a user is willing to wait for their transaction to be included in a block).
To learn more about how the fee estimator works,
this article and
this gist are good places to start.
Then, read the comments
in the source code and verify that the behavior matches the description!
The
estimatesmartfee RPC provides feerate estimates based on the user’s confirmation target.
The test-only
estimaterawfee
RPC provides more detailed information specific to the implementation.
Since the fee estimator is data-based, it’s crucial to accurately record what feerate a
transaction is “offered” at. For example, if a transaction is fee-bumped using Child Pays for
Parent (CPFP), the parent’s individual feerate would be an
underestimation and the child’s would be an overestimation of their “actual” feerate.
The fee estimator skips transactions entering the mempool with unconfirmed
inputs,
alleviating potential overestimation due to a high-feerate transaction sponsoring its ancestors.
However, since it’s impossible to know whether a transaction will have unconfirmed descendants, a
low-feerate parent in a CPFP is still included. This means the fee estimator may underestimate feerates.
One solution is to calculate a transaction’s “actual” feerate, taking ancestors and descendants
into consideration, (see #23074), but the
calculation is not simple.
PR #25380 prevents the fee estimator from
underestimating feerates due to inaccurately recording feerates for CPFP’d transactions.
<larryruane> Yes, only for tx that are in the mempool, and that needs much more metadata since it's in the mempool. A CTransaction describes a transaction no matter where it is
<Amirreza> glozow: Sorry for disrupting the meeting flow, so mempool entries are not only for unconfirmed txs? even confirmed ones can be an object of mempool entry?
<glozow> @ everyone, please never about disrupting the meeting, the purpose of this meeting is to learn more about Bitcoin Core so feel free to ask questions whenever :)
<ishaanam[m]> This PR addresses the issue of our fee estimator under-estimating fees because of CPFP. Fees are under-estimated because we ignore transactions with unconfirmed parents, which means that, in the case of CPFP, we only account for the lower feerate of the parent even though miners are also incentivized to include that transaction because of the the child's higher feerate, which is the one getting ignored.
<larryruane> And I think the problem with fee estimation being too low is that if the user uses this too-low estimate to submit a transaction, it may not confirm as quickly as expected
<glozow> ishaanam: svav: larryruane: yes thank you! My only correction is that the underestimation doesn't arise because we ignore txns with unconfirmed parents, but because we aren't accurately assessing a transaction's "actual" feerate.
<larryruane> glozow: I did have a question on your Notes, "However, since it’s impossible to know whether a transaction will have unconfirmed descendants" ... could you elaborate on what you mean?
<Amirreza> Does the estimation is for a special txs? Isn't it for a given number of blocks? (what feerate pay to include the tx in n blocks) So what why it under-estimates the CPFP'd txs?
<bitcoinbassist> are fees for the child txns as part of cpfp generally overestimated (to ensure that it will make it into the block)? Could this introduce overestimation of fees?
<larryruane> so do you mean that if descendants arrive later (into the mempool), then we don't really "know" that there's an upstream transaction in the mempool?
<larryruane> ok so when a tx first enters the mempool, we adjust the fee estimate based on it ... and later when (if) it gets a descendant, we don't go back and like correct it?
<glozow> Amirreza: The estimation is not for special txs. Yes, when a user calls `estimatesmartfee`, it inputs a confirmation target (a number of blocks) and gets a feerate.
<Amirreza> glozow: the input I think is the number of blocks they want to wait for the tx to be confirmed. The output is the feerate that is more probable to be include in the given number of blocks.
<lightlike> We check if the individual feerate of a child is higher than its ancestor score. If that is a case, we remove all parents with a feerate lower than that of the child from the fee calculation.
<glozow> ishaanam: the ancestor score is the aggregate feerate for a transaction and all of its ancestors. the parent feerate is the individual feerate of the parent alone.
<lightlike> I wonder why we do the first check, comparing with the ancestor score. Why not just drop this check and remove parents that have a lower fee than their child?
<glozow> lightlike: yes exactly, there are 2 conditions: individual is higher than ancestor score and the parent's individual feerate is lower than the child feerate.
<lightlike> i think, with the current rule we'd not drop a CPFP parent if its child happens to depend on another high-paying parent (that would have been included in the block regardless of the CPFP'ing with the low-fee parent). That other parent could increase the ancestor score of the child, right?
<lightlike> i think the nuclear approach would be to simply drop all txes that are parents with a child in the block. That way we wouldn't make any mistakes, but the downside is we'd have less samples for fee estimation.
<bitcoinbassist> I wonder if the check of being greater than ancestor_score is saying that the fee of the child must be sufficiently higher than its ancestors, otherwise its ancestors would have already been confirmed? (so it's likely not paying for it's parent?)
<lightlike> it's convenient that since all transaction that are not on the top level (=parents) are already excluded from the fee estimation anyway, so I think we might really need to take complicated parent-child relationships into account where an ancestor score helps.
<glozow> oh i think maybe that's why it has both parent and ancestor conditions. the second condition would have been "iterate through each ancestor and drop the one that's low feerate," but we only have access to parents from the `CTxMemPoolEntry`.
<lightlike> glozow: right, but the algorithm in this PR wouldn't detect it as CPFP, right? E.g. if the Parent has one low-fee child C1, and C1 has a large-fee child C2, then we wouldn't remove the parent because we don't loop through all ancestors.