Add getprioritisationmap, delete a mapDeltas entry when delta==0 (rpc/rest/zmq, mempool)

https://github.com/bitcoin/bitcoin/pull/27501

Host: glozow  -  PR author: glozow

Notes

  • Miners can call the prioritisetransaction RPC to modify the fees of mempool entries (e.g. to account for out-of-band fees or prioritise their own transactions). The difference “delta” is an absolute fee amount which may be positive or negative. The miner can later “cancel” the priority by calling the RPC again with the inverse value (e.g. 100sat and -100sat).

  • In CTxMemPool, the delta is stored in 2 places.
  • mapDeltas entries are never removed from mapDeltas except when the tx is mined in a block or conflicted (since PR #6464).
    • Mostly it is a feature to allow prioritisetransaction for a tx that isn’t in the mempool {yet, anymore}. When a tx is resubmitted it retains its priority, or marked as “definitely accept” before it has ever been seen.
    • Since PR #8448, mapDeltas is persisted to mempool.dat and loaded on restart.
    • Note the removal due to block/conflict is only done when CTxMemPool::removeForBlock is called, i.e. when the block is received. If you load a mempool.dat containing mapDeltas with transactions that were mined already (e.g. the file was saved prior to the last few blocks), you don’t delete them.
  • There is no way to query the node for not-in-mempool mapDeltas. If you add a priority and forget what the value was, the only way to get that information is to inspect mempool.dat.
  • Calling prioritisetransaction with an inverse value does not remove it from mapDeltas, it just sets the value to 0. It disappears on a restart (LoadMempool checks if delta is 0), but that might not happen for a while.

  • PR #27501 adds an RPC to get prioritised transactions (also tells you whether the tx is in mempool or not), helping users to clean up mapDeltas manually. It also changes CTxMemPool::PrioritiseTransaction so that when a delta is set to 0, it removes the entry from mapDeltas entirely.

Questions

  1. Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK? What was your review approach?

  2. What is mapDeltas? Why does it exist?

  3. How is an entry added to mapDeltas? When is it removed?

  4. Why shouldn’t we delete a transaction’s entry from mapDeltas when it leaves the mempool?

  5. Why should we allow prioritising a transaction that isn’t present in the mempool?

  6. What problem is this PR trying to solve?

  7. What is the PR’s approach to solving this problem? Can you think of any alternatives?

Meeting Log

  117:00 <glozow> #startmeeting
  217:00 <kevkevin> hey
  317:00 <glozow> Hi everyone! Welcome to PR Review Club. Feel free to say hi so we know you're here. Any first-timers?
  417:00 <abubakarsadiq> hi
  517:01 <glozow> We're looking at #27501, add getprioritisationmap, delete a mapDeltas entry when delta==0 today
  617:01 <glozow> Notes and questions in the usual place: add getprioritisationmap, delete a mapDeltas entry when delta==0
  717:01 <LarryRuane> hi
  817:01 <glozow> oops bad copypaste. https://bitcoincore.reviews/27501
  917:02 <glozow> Did y'all get a chance to review the PR and/or look at the notes? How about a y/n
 1017:02 <LarryRuane> y notes and questions, n for review, but I'd love to review this
 1117:02 <kevkevin> very briefly will be mostly spectating today
 1217:03 <sebastianvanstaa> y notes and questions
 1317:03 <abubakarsadiq> yes i read the notes
 1417:03 <yashraj> notes & qns
 1517:04 <LarryRuane> concept and approach ACK
 1617:04 <glozow> Great to hear! Can somebody summarize what this PR does?
 1717:06 <abubakarsadiq> added an rpc to get entries of mapDeltas and also remove a transaction from the mapDelta entries when it's delta fee is 0
 1817:06 <LarryRuane> It keeps `mempool.dat` cleaner by removing unnecessary entries (i.e. ones that have zero delta)
 1917:06 <glozow> abubakarsadiq: yes!
 2017:06 <kevkevin> Creates a new rpc to get mapdeltas and removes any entry with delta==0
 2117:07 <glozow> LarryRuane: yes that is the overarching goal, to clean stuff up
 2217:07 <glozow> kevkevin: yep!
 2317:07 <kevkevin> question? what are map deltas exactly?
 2417:07 <glozow> That's the first question!
 2517:07 <theStack> the pr enables users of the prioritisetransaction RPC (i.e. miners) the possibility to 1) inspect the current map of prioritised txs with a new RPC and 2) also delete them from the map if delta is zero again
 2617:07 <glozow> What is mapDeltas? Why does it exist?
 2717:07 <glozow> Here is where it is declared in the code: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/txmempool.h#L450
 2817:08 <glozow> theStack: yes exactly
 2917:08 <sebastianvanstaa> for a miner to prioritize transactions without changing the (on-chain) financial incentive (i.e fee)
 3017:09 <LarryRuane> mapDeltas allows the node user to artificially modify a transaction's fee which would affect fee estimation, eviction decisions, mining decisions and probably a few other things
 3117:09 <glozow> sebastianvanstaa: LarryRuane: yes! except prioritisetransaction does not affect the fee estimator.
 3217:10 <LarryRuane> theStack: yes, and IIUC, this new RPC won't tell us anything we *couldn't* already know, if we kept track somehow separately, but since the node already knows it, it's helpful for it to be able to provide it
 3317:10 <LarryRuane> glozow: ah +1 thanks
 3417:11 <glozow> How is an entry added to `mapDeltas`? When is it removed?
 3517:11 <sebastianvanstaa> added with prioritizetransaction RPC
 3617:11 <LarryRuane> and just to confirm... when we *relay* a tx, we don't relay it with its different fee... because that would change the txid!
 3717:11 <abubakarsadiq> the entries of mapDelta is transactionID with the current fee right?
 3817:11 <kevkevin> is it added by using prioritisetransaction? and removed when delta==0
 3917:12 <LarryRuane> I think it's added here: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/txmempool.cpp#L854
 4017:12 <yashraj> prioritisetransaction RPC call? removed on node restart?
 4117:12 <sebastianvanstaa> removed when tx mined in a block or conflicted
 4217:12 <LarryRuane> taking advantage of the behavior of `std::map` that if you access a key that doesn't exist, it gets created as a side-effect -- is that right?
 4317:12 <sipa> LarryRuane: You should think of the RPC as informing bitcoind of the fact that if somehow that transaction gets mined, *you* (the node owner) gets paid out of band.
 4417:12 <sebastianvanstaa> kevkevin i think it is not removed when delta==0. Hence the new PR
 4517:13 <kevkevin> sebastianvanstaa ohh sorry question was for current functionality
 4617:13 <glozow> Yeah the transaction itself is *not* modified. If miners could do that and add fees to transactions, I think Bitcoin would be broken.
 4717:13 <LarryRuane> sipa: +1 thanks
 4817:14 <abubakarsadiq> sebastianvanstaa: what does conflicted means, an identical transaction with higher fee rate?
 4917:14 <glozow> abubakarsadiq: no, the fee is a delta, not the current fee. It's what's added on top.
 5017:14 <LarryRuane> (or subtracted)
 5117:14 <LarryRuane> well sorry, it's always added, but could be < 0 haha
 5217:15 <glozow> sebastianvanstaa: correct, it is only removed when the tx is confirmed or conflicted by a block. Another case is if its value is 0 and you restart.
 5317:15 <sebastianvanstaa> abubakarsadiq not sure what 'conflicted' encompasses. Maybe gloria can shed some light
 5417:16 <glozow> and yes everybody is correct, the way to add a delta is to call prioritisetransaction or to load a mempool.dat that contains deltas in it
 5517:16 <kevkevin> does conflicted mean that the uxto was spent already? and thus invalid
 5617:16 <glozow> LarryRuane: right! the delta can be negative. So theoretically miners can use this to block transactions from their mempools
 5717:17 <glozow> kevkevin: yes, "conflicted by a block" means a conflicting transaction (i.e. spending one or more of the same UTXOs) has been confirmed.
 5817:17 <sebastianvanstaa> kevkevin (y)
 5917:17 <sebastianvanstaa> LarryRuane: can they? not sure
 6017:18 <LarryRuane> glozow: that's interesting! they don't really have a way to remove a tx from the mempool, if they want to censor it, so this gives them a means to the same end (not that we like censoring)
 6117:18 <sebastianvanstaa> I think it would stay in the mempool, just not included in a block
 6217:18 <sebastianvanstaa> So, censoring, yes
 6317:19 <abubakarsadiq> well if its deducted to zero will be removed
 6417:19 <glozow> Yeah they would stay in the mempool unless it fills up and we start evicting the low-feerate transactions. The modified feerate is used in that algo, so we'll evict those "de-prioritised" ones.
 6517:19 <LarryRuane> abubakarsadiq: oh you're right
 6617:19 <yashraj> is this rpc for miners?
 6717:19 <sebastianvanstaa> abubakarsadiq the mapDelta entry, not the transaction would be removes
 6817:20 <sebastianvanstaa> *removed
 6917:20 <glozow> yashraj: yes. I can't think of a use case for non-mining nodes to use this.
 7017:20 <sipa> Apart from testing the mining code, perhaps.
 7117:20 <sipa> But yes, I'd say it's intended for miners.
 7217:20 <glozow> haha, yes
 7317:21 <yashraj> thank you
 7417:21 <glozow> Can `mapDeltas` contain an entry for a transaction that isn't in the mempool?
 7517:21 <LarryRuane> in general, we'd like to not encourage miners to get payments out of band (even though no way to prevent it, of course), IIUC
 7617:21 <sebastianvanstaa> glozow yes it can
 7717:21 <LarryRuane> glozow: yes! that's why the delta isn't stored in the actual mempool entry
 7817:21 <sipa> Of course, ideally miners don't get paid out of band at all for transactions; it removes the ability for the public P2P network to e.g. estimate fees.
 7917:22 <glozow> LarryRuane: Yes, I agree. Ideally the public p2p network is a fair, fee-based auction in which everybody knows what the "going rate" is and has the ability to bid.
 8017:22 <sipa> But it was at some point obvious that miners were accepting out of band payments anyway.
 8117:22 <glozow> sebastianvanstaa: LarryRuane: correct
 8217:22 <LarryRuane> sipa: so for example if our standardness rules are too tight, that might be a bad thing if it encourages more out-of-band transactions.. but if they're too loose, then there's a possible DoS vector!
 8317:23 <sipa> Exactly.
 8417:23 <glozow> Why shouldn't we delete a transaction's entry from `mapDeltas` when it is replaced or expired?
 8517:23 <LarryRuane> balancing act :)
 8617:23 <abubakarsadiq> so that if it appears again it will maintain its priority
 8717:24 <LarryRuane> glozow: because it could come back again.. if we didn't retain it in `mapDeltas`, the user would have to prioritize it again
 8817:24 <glozow> abubakarsadiq: LarryRuane: right!
 8917:24 <glozow> Why should we allow prioritising a transaction that isn’t present in the mempool?
 9017:25 <LarryRuane> but i do have a question on this point... if a tx is removed due to it appearing in a block, but then that block is reorged out (rollback), then we would forget its prioritization, right? (but maybe that's rare enough we don't care)
 9117:25 <kevkevin> because it may not be present in our mempool but may be in someone else's?
 9217:25 <kevkevin> and thus might appear again
 9317:25 <sebastianvanstaa> why not? It could show up any time
 9417:25 <theStack> LarryRuane: heh, wanted to ask the same
 9517:25 <LarryRuane> glozow: because it's not in the mempool YET
 9617:25 <glozow> LarryRuane: yes I think we're expecting that to be really really rare.
 9717:26 <sipa> If someone paid to get their transaction prioritized, and the block is then reorged out, they should pay for prioritization again!
 9817:26 <LarryRuane> i almost wonder if it could be, after the tx is 6 or some number of blocks deep, only THEN remove it from mapDeltas?
 9917:26 <abubakarsadiq> glozow: but when that happen it losses it's priority
10017:26 <glozow> Here's the PR that added the removal logic: https://github.com/bitcoin/bitcoin/pull/6464. There is some discussion about when we should/shouldn't remove an entry there.
10117:26 <LarryRuane> sipa: good point!
10217:26 <glozow> I imagine at the time, this was sufficient to make sure mapDeltas was always cleaned up (?). We didn't have RBF at the time.
10317:28 <LarryRuane> Oh Luke makes a good point on the PR you just linked to, that we need a separate map because if the tx isn't already in the mempool, it may not be able to *enter* on its own fee
10417:28 <sebastianvanstaa> LarryRuane(y)
10517:29 <LarryRuane> sipa: although I guess it depends on how they were paid (out of band) ... if a Visa payment, then may not need to be redone
10617:29 <sebastianvanstaa> (y)  I meant
10717:30 <glozow> What is the problem if we don't clean up `mapDeltas`?
10817:31 <kevkevin> we will have a bunch of useless mapDeltas because the transactions are already confirmed or conflicted
10917:31 <LarryRuane> excess memory usage? (but that doesn't seem like it would be a major problem)
11017:31 <sebastianvanstaa> every byte counts!
11117:31 <sebastianvanstaa> to keep HW requirements as low as possible
11217:31 <abubakarsadiq> transactions with 0 delta will be sitting there which is of no use, since they are no longer a priority.
11317:32 <glozow> Yeah, essentially it's a waste of memory
11417:32 <LarryRuane> i had another question, how often is `mapDeltas` written out to `mempool.dat`? is it only on clean shutdown? Or periodically?
11517:33 <LarryRuane> if only on clean shutdown, then the prioritising would have to be redone after restart i guess
11617:33 <LarryRuane> sorry i mean, after non-clean restart
11717:33 <glozow> 2 places. On shutdown (unless you configure otherwise) and if you call the savemempool RPC
11817:33 <LarryRuane> glozow: thanks
11917:34 <glozow> You can grep for `DumpMempool`: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/kernel/mempool_persist.h#L16
12017:34 <LarryRuane> perhaps miners run a cron job to run `savemempool` ... or maybe after each prioritization
12117:35 <glozow> Why?
12217:35 <kevkevin> what happens when `mapDeltas` are written out to `mempool.dat` like what is changed in `mempool.dat`?
12317:36 <glozow> kevkevin: sorry, I don't follow what the question is?
12417:36 <abubakarsadiq> can we set a negative delta for a transaction
12517:36 <glozow> yes
12617:37 <LarryRuane> (was that question to me?) well because if the node shuts down uncleanly, the user won't have to remember all the prioritizations that were done ... but i know non-clean shutdown is very rare
12717:37 <glozow> Oh I see. That's a good question to think about: today, if you're a miner, how do you clean up `mapDeltas`?
12817:37 <kevkevin> glozow I just saw that mapDeltas are written out to `mempool.dat` I thought they were separate. Do the `mapDeltas` change `mempool.dat` in anyway?
12917:38 <glozow> (My guess is the answer is "you don't")
13017:38 <glozow> kevkevin: oh I see. Yes, they are part of mempool.dat!
13117:38 <abubakarsadiq> 0 fee delta will remove it from mapDeltas is it the same with negative?
13217:38 <kevkevin> glozow ohh ok thanks
13317:39 <kevkevin> abubakarsadiq wouldn't negative still be valid if we want to deprioritize a transaction?
13417:40 <glozow> abubakarsadiq: 0 fee delta only removes it from mapDeltas when you're loading: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/kernel/mempool_persist.cpp#L76-L78
13517:40 <abubakarsadiq> What is mapDeltas? Why does it exist?
13617:40 <glozow> negative delta would be something you still want to keep, i don't think it makes sense to just delete it if it's negative
13717:40 <abubakarsadiq> sorry copy paste, not a question
13817:41 <LarryRuane> back to question 3 for a second, another place entries are added to `mapDeltas` is on restart: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/txmempool.cpp#L854
13917:41 <abubakarsadiq> thanks glowzow, kevkevin
14017:41 <glozow> kevkevin: you can see mapDeltas being written to the file here: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/kernel/mempool_persist.cpp#L166
14117:42 <theStack> interesting that the mempool serialization was never changed to not serialize 0 fee delta entries in the first place (but only ignore them on loading)
14217:43 <glozow> Anybody have any ideas on the last question? If you're a miner today, how might you clean up mapDelats?
14317:43 <kevkevin> glozow gotcha and DumpMempool is only used when we call savemempool or on shutdown
14417:43 <glozow> kevkevin: yep!
14517:43 <LarryRuane> glozow: that code you just linked to, interesting that it's only writing out `mapDelta` entries that are NOT in the mempool?
14617:43 <LarryRuane> see 4 lines above that line, the call to `mapDeltas.erase()`
14717:44 <glozow> LarryRuane: for entries in the mempool, there is an `nFeeDelta` field serialized alongside the transaction https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/kernel/mempool_persist.cpp#L162
14817:44 <kevkevin> glozow no idea either do nothing or delete all mapDeltas?
14917:44 <glozow> LarryRuane: also note that, in that function, `mapDeltas` is a local copy of the mempool's mapDeltas and goes out of scope at the end of the function
15017:44 <sebastianvanstaa> glozow restart the client now and then? (bad idea , I know)
15117:45 <LarryRuane> glozow: +1 thanks
15217:45 <theStack> sebastianvanstaa: +1, that would be also my answer
15317:46 <theStack> (or well, restart bitcoind, not the client)
15417:46 <glozow> kevkevin: sebastianvanstaa: kind of. I think the only way to do it is to keep track of every time you've called prioritisetransaction and when the entries would be deleted, then "cancel" the stale ones by setting the delta to 0 and then restarting your node.
15517:46 <LarryRuane> yes so this is a very useful PR!
15617:46 <sebastianvanstaa> but restarting your node would certainly lead to loss of mining profits, right?
15717:47 <LarryRuane> sebastianvanstaa: really good point!
15817:47 <glozow> yeah, so my guess is it's just not cleaned up lol
15917:47 <LarryRuane> probably miners are using beefy machines with tons of memory so isn't a practical problem
16017:47 <sebastianvanstaa> theStack yes I meant bitcoind
16117:48 <glozow> well, if mapDeltas is huge, it cuts into your 300MB max mempool
16217:48 <sebastianvanstaa> just increase it's size then :)
16317:49 <glozow> Ok so after this PR, if you're a miner, how would you clean up your mapDeltas?
16417:49 <LarryRuane> glozow: oh, interesting, that mempool limit applies to this map as well, didn't know that
16517:50 <glozow> see `CTxMemPool::DynamicMemoryUsage`: https://github.com/bitcoin/bitcoin/blob/1d7f1ada48083d27adda47e04fd0311cc3cd6816/src/txmempool.cpp#L959
16617:50 <kevkevin> would we just call prioritisetransaction and set the delta to 0
16717:50 <kevkevin> because of this bit of code
16817:50 <kevkevin> https://github.com/bitcoin/bitcoin/pull/27501/files#diff-c065d4cd2398ad0dbcef393c5dfc53f465bf44723348892395fffd2fb3bac522R873-R878
16917:50 <LarryRuane> glozow: +1 thanks
17017:51 <glozow> kevkevin: yes and how would you know what arguments to call prioritisetransaction with?
17117:51 <theStack> i'd use the new fancy RPC call to see what the current delta for a given tx is, and then call prioritisetransaction with that value negated to set it to zero again... aaand it's gone
17217:51 <glozow> theStack: bingo
17317:51 <sebastianvanstaa> glozow first use the RPC added by this PR to get a list of all priritized transactions, of course :)
17417:51 <kevkevin> you would first call getpriotisationmap to know how much to move the delta?
17517:53 <glozow> yep yep
17617:53 <glozow> Ok that's all the questions I prepared. Sounds like everybody understands the PR so I'll be expecting your reviews soon ;)
17717:53 <kevkevin> would it not make sense to also be able to just remove a transaction from the mapDelta and not have to call two rpc's to achive that, not sure if I'm missing why not have the ability to do so
17817:54 <glozow> kevkevin: are you suggesting a `clearallprioritisation` RPC?
17917:54 <sebastianvanstaa> new PR coming up !
18017:55 <kevkevin> glozow: yea that or to be able to clear a specific prioritisation
18117:55 <glozow> kevkevin: how do you know which specific transactions to clear?
18217:55 <LarryRuane> txid?
18317:56 <kevkevin> glozow: the same way we call prioritisetransaction
18417:56 <glozow> then you have to remember that it's in your mapDeltas?
18517:57 <kevkevin> well I guess the other way would be to call getprioritisationmap but we already do that with this PR haha
18617:57 <glozow> jnewbery suggested to me offline that perhaps we could implicitly interpret `prioritisetransaction(txid, 0, delta=0)` as "clear prioritisation." I think that could be an extra simplification.
18717:58 <kevkevin> glozow: ya I think that would be better than a whole new rpc
18817:58 <sebastianvanstaa> glozow yes that would make sense
18917:59 <glozow> Anyway, maybe for a followup. It could be a "userspace break" so I won't throw it in this PR
19017:59 <LarryRuane> someone earlier asked if probably only miners would use `"prioritisetransaction` .. i noticed that that PRC is in `rpc/mining.cpp` so that tell you something!
19117:59 <abubakarsadiq> +1 i think it will will be better to remove a transaction in one call
19217:59 <glozow> Thanks for coming everyone! See you next week!
19317:59 <theStack> i think both the new RPC and the implicit "clear prioritisation" idea have value
19417:59 <glozow> #endmeeting