CPFP fee bumping within packages (tx fees and policy, validation)


Host: glozow  -  PR author: glozow

The PR branch HEAD was 7daf3d0 at the time of this review club meeting.


  • Package Mempool Accept is a project to add policies for accepting packages of transactions (i.e. ones that would not otherwise be accepted individually) into the mempool, in preparation for package relay. We have covered other Package Mempool Accept PRs in previous review club meetings: PR #20833, PR #21800, and PR #22674.

  • PR #24152 is the next PR in the series. It is centered around creating an incentive-compatible policy for assessing package feerates. It introduces three new concepts:

    • Transactions are validated individually first at the start of package validation. Any transactions that pass individually are removed from the package. Afterwards, only the leftover transactions are submitted through AcceptMultipleTransactions().

    • Package feerate is defined as the total modified fees divided by the total virtual size of all transactions in the package after de-duplication and individual submission of each transaction. For a “package” of 1 transaction, this is equivalent to its modified feerate.

    • Package feerate, instead of individual feerate, is used to evaluate the transactions in the two feerate policies: the static Minimum Relay Feerate (minRelayTxFee) and dynamic mempool minimum feerate.

  • Incentive compatibility is the main concern of this PR. The following capabilities and restrictions encapsulate the behavior we are aiming for for users:

    1. A child can pay for both itself and its parent within a package. This is also known as Child Pays For Parent (CPFP).

    2. A child must pay for both itself and its parent within the package. The package feerate must meet the minimum requirements.

    3. Ensure we use modified feerate rather than base feerate. Modified fees include base fees and any fee delta from prioritisetransaction, which can be used by miners to give priority to their own transactions in their mempools.

    4. A transaction should not help the feerate of another transaction if it is not an ancestor (i.e. not necessary for the other transaction to be mined). Discourage “parent pays for child” and “sibling pays for sibling” behavior.

    5. A transaction should not harm the feerate of another transaction if it is not an ancestor (i.e. not necessary for the other transaction to be mined). A low-fee child in the package should not stop its parent(s) from being accepted if they have sufficient feerates individually.

    6. A transaction’s fees should never be “double counted.” Once a transaction has been submitted to the mempool, don’t include its fee or size in feerate calculations again.

  • Tests are added to represent various desirable and undesirable fee-related policies. The packages tested are illustrated in the diagrams below:

    image image image image


  1. Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK? What was your review approach?

  2. Define package feerate, in your own words.

  3. Quiz: Let’s say we have transactions {A, B, C, D} where {A, B, C} are parents of D. For simplification, transactions are 100vB. All {A, B, C} pay 0 fees, and D pays 30,000sat in fees. The user makes this series of calls:

    prioritisetransaction(B, +100,000sat)
    prioritisetransaction(C, +10,000sat)

    Now we submit package {A, B, C, D} and it succeeds. What is the package feerate returned?

    (A) 100sat/vB

    (B) 150sat/vB

    (C) 200sat/vB

    (D) 300sat/vB

    (E) 350sat/vB

    (F) no package feerate

  4. Why is submitting individually first and using this definition of package feerate sufficient to ensure we are assessing feerates properly? (Hint: we restrict package topology)

  5. Would this definition of package feerate work for all types of packages? For example, packages with three generations, a parent with multiple children, unrelated transactions, etc.

  6. Do you agree with the 6 incentive-related behaviors described in the notes above? Can you think of anything missing?

  7. Do the tests cover all 6 points? (It may help to go through each point and see which test covers each). Do you have any suggestions for more tests cases?

  8. If AcceptSingleTransaction() fails here due to a consensus rule, is there any reason we should do package validation later? Is there any reason we should continue to validate the other transactions in the package?

  9. Why do we need to create a new set of ATMPArgs here? Why can’t we just use args for individual submission? Why must we pass in args when creating single_args? Why don’t we just create a new ATMPArgs from scratch?

Meeting Log

  117:00 <glozow> #startmeeting
  217:00 <b10c> hi
  317:00 <lightlike> hi
  417:00 <michaelfolkson> hi
  517:00 <svav> Hi
  617:00 <bitcoin1o1> hi all
  717:00 <emzy> hi
  817:00 <theStack_> hi
  917:01 <glozow> Welcome to PR review club everyone! Today is package CPFP day: https://bitcoincore.reviews/24152
 1017:01 <glozow> did anyone get a chance to review the PR or read the notes? how about a y/n?
 1117:01 <effexzi> Hi every1
 1217:01 <emzy> n
 1317:01 <larryruane> gm!
 1417:01 <effexzi> N
 1517:01 <b10c> n
 1617:02 <svav> Read the notes
 1717:02 <bit-pleb-paul> Hi
 1817:02 <lightlike> y
 1917:02 <bitcoin1o1> n
 2017:02 <larryruane> 0.6y
 2117:02 <bit-pleb-paul> read the notes
 2217:02 <Ludwig> n
 2317:02 <theStack_> n
 2417:02 <michaelfolkson> y Read notes, built, run unit tests etc
 2517:02 <glozow> since a lot of us haven't reviewed the PR, could someone summarize what it does?
 2617:03 <glozow> (or take a guess?)
 2717:03 <svav> It is centered around creating an incentive-compatible policy for assessing package feerates.
 2817:03 <glozow> svav: yep!
 2917:03 <neha> hi
 3017:04 <glozow> can you give an example of what this policy enables?
 3117:04 <larryruane> when evaluating a package, instead of just checking the overall package feerate, look inside and check the individual tx first, accept any as individuals if possible, then check the (reduced) package (what's remaining0
 3217:04 <svav> Basically to make sure incentives for miners will work for packages.
 3317:04 <glozow> larryruane: yes, that's also a part of this pr!
 3417:05 <larryruane> and I think the part I mentioned has to do with exactly that, making things miner incentive-compatible
 3517:05 <svav> A basic question - what is the rationale behind wanting to have packages in Bitcoin at all? Is it just to help low fee transactions get processed?
 3617:06 <bit-pleb-paul> It’s for lightning - I’ll elaborate - I’ll be typing with my thumbs today
 3717:06 <glozow> svav: good question! here's a link for full motivation: https://gist.github.com/glozow/dc4e9d5c5b14ade7cdfac40f43adb18a#package-relay-and-package-mempool-accept
 3817:07 <larryruane> svav: that's a great question, maybe I can make an attempt... Suppose you have a tx that has SUCH a low fee that mempools won't even accept it at all ... and you don't have the ability to create a new tx, you only have that one ... so with package feature, you can combine it with a high fee child and get them both accepted
 3917:07 <glozow> good answer larryruane
 4017:07 <larryruane> (if the mempool won't accept it at all, then it's impossible to fee-bump it with CPFP)
 4117:07 <glozow> so one example is, if your mempool is full and the minimum feerate is higher than the feerate of an individual transaction, right now you won't be able to use CPFP to broadcast it, because we only consider transactions individually
 4217:08 <glozow> oops larryruane beat me to it :P
 4317:08 <larryruane> this was mentioned in the brink podcasts (the second one I believe), which everyone should listen to! https://brink.dev/podcast
 4417:09 <emzy> it's an chicken and egg problem.
 4517:10 <bit-pleb-paul> In regards to lightning - if a channel is force closed by one party, but the closing transaction doesn’t have a high enough fee, it  won’t even be accepted into the mempool, as described above. Thereby lightning force closures wouldn’t work, and I the outcome of that, I think, is that counterpartoes could steal funds
 4617:10 <glozow> bit-pleb-paul: indeed. commitment transactions also cannot replace each other via RBF, and if the feerate isn't enough to make it into the mempool, your hands are tied
 4717:10 <larryruane> You may be wondering why would someone create a tx with such a low feerate that it won't even be accepted into the mempool (much less mined)? I think the answer is that this tx may have been created months ago when fees were very low, and now fees are much higher, and we can't *change* the tx ... I think this happens with Lightning?
 4817:11 <glozow> larryruane: correct, fees are negotiated with your counterparty beforehand. not months ago, but yes you could have underestimated
 4917:11 <larryruane> (this idea that you HOLD a transaction, that you can't modify, for a long time, is common with L2 stuff)
 5017:12 <glozow> with package relay and the policy in this PR, you could actually just put 0 fees on the commitment transaction and adjust your fee-bump based on the current feerate market when you go to broadcast it
 5117:12 <glozow> that's further into the future though.
 5217:12 <glozow> emzy: what do you mean by chicken and egg problem?
 5317:12 <glozow> just curious
 5417:13 <ziggie> hi
 5517:13 <svav> With a Child Pays for Parent Transaction being used to bump another transaction, is the Child transaction always initiated by the same person that sent the parent transaction?
 5617:14 <emzy> glozow: you can't get the low fee transaction in and you can't get the CPFP transacion in.
 5717:14 <larryruane> glozow: very cool, I guess the tradeoff is, if you try to estimate a fee ahead of time, and it turns out to be enough, then you can save overall fees because you don't need that child ... whereas, if you create the tx with 0 fee, then it's guaranteed you'll need a child - 2 transactions, more expensive
 5817:14 <glozow> and anyone else, feel free to ask more questions about motivation, i'm happy to describe further. the PR isn't very exciting if we don't know why we're doing this
 5917:14 <glozow> svav: nope! if the parent is a payment, the child could be created by either the sender (from their change output) or recipient (from their payment output)
 6017:15 <glozow> emzy: oh yes i see what you mean now 🧠
 6117:16 <larryruane> svav: if the sender wants to change the fee, he or she has the option of RBF, which probably is a little cheaper overall (other things equal)
 6217:16 <glozow> larryruane: that's true. the counterargument is: if you estimate the fees ahead of time, you could end up overestimating and wasting money.
 6317:16 <ziggie> so this package is only a package for relaying, in the mempool the structure stays the same as of today ? And as soon as a tx is in the mempool other rules to accept it into a block apply ?
 6417:16 <larryruane> glozow: +1
 6517:16 <michaelfolkson> What sort of DoS vectors are there with CPFP (if any)? I'm in RBF mode and I'm trying to get into CPFP mode :)
 6617:17 <glozow> ziggie: correct, there would be no changes to the mempool data structure itself. our block template building logic has used CPFP since at least the past few years
 6717:17 <ziggie> glozow thanks
 6817:18 <theStack_> why is RBF not possible for lightning commitment transactions? is it because we would need a signature from the channel partner, which is possibly not able to cooperate (or not online at all)?
 6917:18 <larryruane> theStack_: maybe it's more a problem for the settlement tx?
 7017:18 <glozow> michaelfolkson: that's a good question, since DoS is a very front-and-center concern when looking at this area of the code. but i think incentive compatibility and preventing censorship are a more relevant concerns for package cpfp specifically.
 7117:19 <ziggie> theStack_as soon as you are using a lightning commitment tx your are "unilateraly closing channel" so no way to get a newer transaction (RBF) with more fees
 7217:19 <ziggie> because peer is unavailable
 7317:20 <glozow> theStack_: correct. for one commitment tx to replace the other, it would need to have pretty significantly higher fees. since you negotiated the fees ahead of time (and they would be the same for both you and your counterparty's tx) and you can't get them to sign a new one, it's not possible.
 7417:20 <theStack_> larryruane: hmm isn't the commitment transaction (also called refunding transaction afaik) what you mean by "settlement tx"?
 7517:20 <bit-pleb-paul> I really liked the incentive where a child with no fee ought not be able to ‘dragdown’ a parent from entering the mempool (by not increasing the fee while increasing the vybtes )
 7617:20 <bit-pleb-paul> This seems like an anti DoS incentive
 7717:20 <theStack_> ziggie: glozow: ok thanks, that makes sense
 7817:21 <larryruane> theStack_: you're probably correct, I'm lightning-challenged :)
 7917:22 <glozow> bit-pleb-paul: thanks! :) i wouldn't call that a DoS (it's not exhausting resources), but censorship. somebody shouldn't be able to get your transaction rejected just by adding a low-feerate child to it and sending them as a package.
 8017:22 <theStack_> larryruane: heh, me too, actually. i'm planning to read "mastering lightning" soon to hopelly change that :D
 8117:22 <theStack_> *hopefully
 8217:23 <michaelfolkson> There is a commitment transaction that can be broadcast to close a channel unilaterally and a justice transaction if your counterparty broadcasts an old commitment transaction
 8317:23 <glozow> i learned most of my lightning info from here: https://chaincode.gitbook.io/seminars/lightning-protocol-development
 8417:23 <ziggie> can somebody explain why we are removing transactions from the package which are valid without the package, it it not better for the package to be higher in feerate ?
 8517:24 <bit-pleb-paul> question - what are siblings? Any two txs in the same block?
 8617:24 <glozow> bit-pleb-paul: transactions that share a parent
 8717:24 <lightlike> why couldn't the original transaction be several months (or even years) old until it needed to be CPFP'ed? surely lighting channels could stay open that long?
 8817:24 <bit-pleb-paul> +1 ziggies question
 8917:25 <bit-pleb-paul> @light I believe they can be that old, they just generally aren’t
 9017:25 <glozow> lightlike: oh yes true, it could be months old if it's from a very old state e.g. when the counterparty had a much higher balance. i guess i didn't really imagine that happening normally
 9117:26 <larryruane> ziggie: i think "better" is a complicated concept here, if one thing is better, everything else is relatively slightly worse, so may not get mined (or possibly even get evicted from the mempool) ... but that's still a good question!
 9217:26 <theStack_> glozow: thx for the chaincode seminar link, that seems to be a good learning resource
 9317:27 <michaelfolkson> Can we do the questions from the notes? :)
 9417:27 <glozow> ziggie: very good question, it's one of the questions in the notes, so let's just move on to those
 9517:27 <larryruane> 1. concept ACK from me for sure, the code looks solid, i would need to review more
 9617:27 <glozow> first question: Define package feerate, in your own words.
 9717:28 <larryruane> total fees divided by total vbytes ... which may be very different from the individual tx feerates!
 9817:28 <glozow> larryruane: great answer
 9917:29 <glozow> let's have an example. Let’s say we have transactions {A, B, C, D} where {A, B, C} are parents of D. For simplification, transactions are 100vB. All {A, B, C} pay 0 fees, and D pays 30,000sat in fees.
10017:29 <glozow> what's the package feerate of package {A, B, C, D} ?
10117:30 <theStack_> first naive try: 30000 sats / 400 vb = 75 sats/vb
10217:30 <bitcoin1o1> glozow: 75 ?
10317:30 <glozow> theStack: bitcoin1o1: exactly
10417:30 <bit-pleb-paul> Without thr prioritise tx bumps?
10517:31 <larryruane> (I like the way you name those, A-D, very small suggestion, in the new packages.md text, they're the other way around, B is the parent and A is the child, maybe reverse those?)
10617:31 <glozow> very good. let's say, before submitting, we call `prioritisetransaction(D, +10,000sat)`. what's the package feerate now?
10717:32 <theStack_> 40000 sats / 400 vb = 100 sats/vb ?
10817:32 <glozow> theStack_: bingo
10917:33 <glozow> ok and what if, before submitting, we call `prioritisetransaction(A, +100,000sat)` ?
11017:33 <ziggie> We assume non of the transaction is in any mempool of node right now ?
11117:33 <bit-pleb-paul> 140,000/400
11217:33 <glozow> (in this scenario we didn't prioritise D)
11317:33 <bit-pleb-paul> Oh
11417:33 <glozow> ziggie: very good question, assume that none are in the mempool for this one
11517:33 <glozow> but followup question: what is the package feerate if A and B are already in the mempool and we submit {A, B, C, D} ?
11617:34 <bit-pleb-paul> They would remove A and B from thr package
11717:34 <bit-pleb-paul> So
11817:34 <ziggie> (in this scenario we didn't prioritise D) still 100
11917:35 <ziggie> A gets out of the package I guess
12017:35 <bit-pleb-paul> 100000sats/200vbytes?
12117:35 <glozow> ziggie: yes exactly! A is validated first, so we end up only using {B, C, D} in the package feerate calculation.
12217:36 <bit-pleb-paul> Why is A removed from the pckage while B remains?
12317:36 <ziggie> only A is valid, B has still 0 fees ?
12417:37 <glozow> yep, thank you ziggie. we validate all transactions individually first, and then try as a package. this ensures that ancestors don't pay for descendants
12517:37 <neha> i'm confused as well :) A and B have 0 fees -- how did they get in the mempool?
12617:37 <glozow> to illustrate this, look at this example: let's consider a package where the parent is high feerate and the child is low feerate: https://github.com/glozow/bitcoin-notes/blob/master/mempool_garden/package_rich_parent.png
12717:37 <glozow> A was prioritized - its fee was modified with +100,000 when we called `prioritisetransaction()`
12817:37 <bit-pleb-paul> I think in this case A had a setpriority from the miners
12917:38 <neha> ah, ok, missed that we were considering the prioritisetransaction for A. thanks!
13017:38 <glozow> neha: okay whew! glad that you clarified
13117:38 <neha> i suppose we assume the node operator just felt like including B?
13217:39 <ziggie> are we answering the why later for this "excluding A" from the package
13317:39 <glozow> B is submitted as a package with C and D. D pays 30,000sat fees, so the package feerate is 30000sat/300vB
13417:39 <theStack_> ok we only want child-pays-for-parent, but not parent-pays-for-child... that makes sense
13517:40 <bitcoin1o1> cause A is a parent?
13617:40 <ziggie> theStack_ good point
13717:41 <glozow> theStack_: yes exactly. in general, a transaction B should only "pay for" another transaction A if B requires A in order to be mined.
13817:41 <glozow> i hope that assertion makes sense to everyone
13917:42 <glozow> bitcoin1o1: sorry i'm not sure what you're referring to?
14017:42 <ziggie> yeah very clear now thanks glozow
14117:42 <svav> In this diagram https://github.com/glozow/bitcoin-notes/blob/master/mempool_garden/package_rich_parent.png .... P2 shows 0 sat. Does this mean 0 sat in fees? Isn't there a min fee rate that all transactions must adhere to?
14217:42 <glozow> svav: yes, P2 should be rejected.
14317:43 <bit-pleb-paul> If a user wanted to send a tip on chain, couldn't they piggyback onto a parent as a lower child? Seems like a legitimate usecase
14417:43 <svav> glozow In your examples are the fees you show the standard default fees set by the user rather than any discretionary prioritisation fees?
14517:43 <theStack_> isn't another other issue that bumping A would change its txid and therefore the outpoint for the child's input, needing to resign the child tx?
14617:43 <bit-pleb-paul> Like a tip at a bar* or something
14717:44 <glozow> svav: that is the simplest example for why we first validate individually, to eliminate the parents from the package if they don't need to be paid for.
14817:44 <michaelfolkson> [17:41:13] <glozow> theStack_: yes exactly. in general, a transaction B should only "pay for" another transaction A if B requires A in order to be mined.
14917:44 <glozow> and yes they are base fees, with no modifications unless specified
15017:45 <larryruane> bit-pleb-paul: there would be no need for the tip to be a child, right? it could be just a separate tx?
15117:45 <lightlike> bit-pleb-paul: but they need to be able to build a package with the parent, so they need to be the sender or receiver. they can't just tip random transactions they like.
15217:45 <michaelfolkson> The point is we only try the package logic after trying the individual tx logic right?
15317:45 <michaelfolkson> If the individual tx has enough fee it gets into mempool on its own
15417:45 <glozow> right. there's no reason why the tip should be a child. it could be a separate transaction or, better yet, the sender should just create a replacement with the new payment value.
15517:45 <emzy> bit-pleb-paul: the tip example still works. But the package to get in the mempool is not needed.
15617:46 <bit-pleb-paul> glozow the replacement tx is a better idea, you're right
15717:46 <glozow> michaelfolkson: yes. the only reason to consider package feerate together is for descendants to pay for ancestors. the nice thing about only allowing child-with-parents packages is that simply validating them individually first achieves this.
15817:46 <emzy> bit-pleb-paul: the parent can go in the mempool alone.
15917:47 <neha> bit-pleb-paul: the intuition i use is that running a node incurs a cost, and you want people proposing txns to pay for the resources they use. the only way we can do that before txn confirmation is to only really consider things that have a chance of getting confirmed on chain, where the txn proposer will have to pay a fee. otherwise, an attacker could create a lot of work for a node "for
16017:47 <neha> free"
16117:48 <neha> so consider your tips in that analogy -- we don't want there to be a way for a transaction proposer to create a bunch of low-feerate txns without the required cost of paying something higher fee at some point. otherwise, people could spam nodes with low-feerate tips.
16217:48 <bit-pleb-paul> neha thx
16317:49 <larryruane> michaelfolkson: "... only allowing child-with-parents packages ..." You bring up a point I wanted to confirm, even with this PR (and the package relay ones that came previously), the full DAG package isn't supported, right? It's restricted to just 2 levels, and more than that, a single child with possibly many parents?
16417:50 <glozow> the biggest reason why this works is because we only allow child-with-parents packages. "validate each individually first" is equivalent "validate all ancestors individually first," which eliminates the problem of ancestor-paying-for-descendant, and we can just use a package feerate
16517:50 <larryruane> but later we'll allow the more general DAG?
16617:50 <svav> Can we whizz through the rest of the questions?
16717:50 <glozow> equivalent to*
16817:50 <glozow> ok sure. Would this definition of package feerate work for all types of packages? For example, packages with three generations, a parent with multiple children, unrelated transactions, etc.
16917:51 <larryruane> glozow: i *think* so
17017:52 <svav> My guess is yes because it has to cover all eventualities???
17117:52 <lightlike> no. it might be incentive-compatible to just accept the parent and a child of the first generation, but discard all children later generations, so it would need a more sophisticated definition.
17217:52 <glozow> no, it doesn't haha
17317:52 <ziggie> I think the one with more than one generation is the problem
17417:52 <glozow> lightlike: exactly. package feerate in this definition is ignorant of topology, which is crucial when we're thinking about incentive compatibility of multiple transactions.
17517:52 <larryruane> lightlike: +1 good point!
17617:53 <neha> is there a need to accept packages with more complex topologies?
17717:53 <glozow> like we've talked about multiple times, unless there is a dependency relationship, there's no reason why another transaction's fees should be relevant in validation
17817:53 <neha> or do the current child/parent packages handle all the, for example, L2 use cases you'e heard of?
17917:54 <glozow> neha: good question. luckily no, according to everyone i've talked to
18017:54 <neha> that's great!
18117:54 <bit-pleb-paul> What would happen if a child's fee were still to low, due to user error, e.g. lowballing? The child would simply be replaced by another?
18217:54 <glozow> mailing list discussion thread here: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-September/019464.html
18317:55 <glozow> next question is
18417:55 <glozow> Do you agree with the 6 incentive-related behaviors described in the notes above (https://bitcoincore.reviews/24152)? Can you think of anything missing?
18517:56 <michaelfolkson> "Discourage “parent pays for child” and “sibling pays for sibling” behavior." sounds strong from previous discussion
18617:56 <glozow> great!
18717:57 <michaelfolkson> It isn't discourage, it is only check this once you've checked individually
18817:57 <michaelfolkson> Oh I mean too strong lol
18917:57 <michaelfolkson> Right?
19017:57 <glozow> er, what do you mean by too strong?
19117:58 <michaelfolkson> I guess its a nit on the word "discourage"
19217:58 <neha> can't a txn's feerate be harmed by a non-ancestor if it conflicts out one of their ancestors? or children?
19317:58 <michaelfolkson> If it is just handled by the code logic the user isn't being discouraged, it is just a code logic ordering issue
19417:58 <neha> or maybe i'm misunderstanding "harm"
19517:59 <neha> this is wrt #5
19617:59 <glozow> neha: ah yes that's a very good observation! we'll get to that with package RBF. but at this point, no conflicts are allowed in package validation.
19718:00 <glozow> michaelfolkson: the ordering of the code here affects the outcome.
19818:00 <svav> How come this PR has no reviewers yet? Is it too new or are people not interested in reviewing?
19918:00 <larryruane> hey let's all review it!!
20018:01 <glozow> svav: afraid i am unable to answer that question :'(
20118:01 <ziggie> larryruane +1
20218:01 <michaelfolkson> glozow: Hmm ok, I think I understand. It is in the user's interest to try to get the individual tx in the mempool first?
20318:01 <neha> thank you glozow!
20418:01 <glozow> feel free to also ask questions on the PR, those would be valuable to both me and other future reviewers who might have the same question
20518:01 <glozow> #endmeeting