Stop relaying non-mempool txs (p2p)

https://github.com/bitcoin/bitcoin/pull/27625

Host: mzumsande  -  PR author: MarcoFalke

The PR branch HEAD was faa2976a56ea7cdfd77ce2580a89ce493b57b5d4 at the time of this review club meeting.

Notes

  • mapRelay is a map containing all transactions that we’ve relayed to any peer recently. It is accompanied by g_relay_expiration, a sorted list of expiration times for mapRelay entries. Entries stay in mapRelay and g_relay_expiration for 15 minutes.

  • When a peer asks for a transaction by sending a getdata message but the transaction is no longer in the mempool, it can be served from mapRelay.

  • mapRelay has been around for a long time, it was already present in the first github commit. While it was essential back then, its scope has been reduced over time: For example, Bitcoin Core now first tries to fetch transactions directly from the mempool. There are other reasons why mapRelay wasn’t removed earlier (see this comment for an overview), but most of these have been made obsolete by other improvements.

  • This PR removes mapRelay and instead introduces m_most_recent_block_txs to keep track of only the transactions from the most recent block.

Questions

  1. Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK? What was your review approach?

  2. What are the reasons to remove mapRelay?

  3. Why is the memory usage of mapRelay hard to determine? (Hint: see this comment)

  4. What problem is solved by introducing m_most_recent_block_txs? Do you think it is necessary to introduce it (as opposed to just removing mapRelay without any replacement)?

  5. What are the memory requirements for m_most_recent_block_txs compared to mapRelay?

  6. Are there scenarios in which transactions would be made available for a shorter or longer time than before as a result of this change?

  7. Can you think of any other possible downsides of removing mapRelay?

Meeting Log

  117:00 <lightlike> #startmeeting
  217:00 <lightlike> hi!
  317:00 <stickies-v> hi
  417:01 <abubakarsadiq> hello
  517:01 <effexzi> Hi every1
  617:01 <lightlike> welcome to this week's review club!
  717:01 <kevkevin> hi
  817:01 <emzy> hi
  917:01 <lightlike> Today's PR is https://github.com/bitcoin/bitcoin/pull/27625 (Stop relaying non-mempool txs). Notes are at https://bitcoincore.reviews/27625
 1017:02 <lightlike> the PR already got merged earlier this week!
 1117:02 <lightlike> Did you review the PR? Concept ACK, approach ACK, tested ACK, or NACK?
 1217:02 <kevkevin> Concept ACK
 1317:03 <emzy> n
 1417:03 <glozow> woot!
 1517:03 <mutex> Concept ACK
 1617:03 <abubakarsadiq> tested ACK
 1717:03 <stickies-v> i'm hoping to get to Concept ACK after this meeting - looked at the code (and notes) but haven't fully wrapped my head around it yet
 1817:04 <lightlike> great! then let's jump into the questions.
 1917:04 <lightlike> What are the reasons to remove mapRelay?
 2017:05 <brunoerg> hi
 2117:05 <emzy> memory consumption?
 2217:06 <stickies-v> unpredictable memory consumption (since we don't know which % of mapRelay is also in mempool)?
 2317:06 <schmidty> hi
 2417:06 <abubakarsadiq> because their is no utility for the transactions in mapRelay, all transaction dropped from mempool except for BLOCK reason does not need to be relayed anymore?
 2517:07 <stickies-v> also, side question: this PR should affect GETDATA behaviour, but not INV behaviour, right?
 2617:07 <lightlike> emzy, stickies-v: yes, exactly. I'd say it just gives an uneasy feeling having an unbounded structure, even if the actual memory consumption may not be that high.
 2717:08 <instagibbs> abubakarsadiq and REPLACED
 2817:08 <instagibbs> (nice table in the PR)
 2917:08 <lightlike> abubakarsadiq: yes, that too. I think it's kind of a relic from the past when it was the main mechanism for tx transaction and we woldn't look up txns from the mempool.
 3017:10 <lightlike> stickies-v: yes, i agree, INV shouldn't be affected
 3117:11 <abubakarsadiq> instagibs:I thought REPLACED also is not needed https://github.com/bitcoin/bitcoin/pull/27625#issuecomment-1548505174 , why would we want to relay REPLACED transactions?
 3217:11 <stickies-v> thanks, makes sense, we're just not keeping data for transactions that we believe our peers shouldn't care about anymore either
 3317:12 <lightlike> next question: Why is the memory usage of mapRelay hard to determine?
 3417:12 <mutex> it depends on the number/rate of incoming transactions?
 3517:13 <abubakarsadiq> there is no maximum memory?
 3617:13 <lightlike> mutex: that's one aspect, but not all there is to it.
 3717:15 <lightlike> do we need the same amount of memory for all entries of mapRelay (given the txns are the same size), or does it depend on other factors?
 3817:16 <stickies-v> mapRelay and mempool share the same `std::shared_ptr<const CTransaction>` pointers, so the memory overhead of having an extra transaction in mapRelay depends on whether or not the transaction is still in mempool
 3917:16 <mutex> ah I see the detailed comment now! Looks like mapRelay <-> Mempool eviction relationship is not tracked
 4017:18 <lightlike> stickies-v: exactly! Due to the use of a shared_ptr, if we have the tx stored in the mempool, the overhead of adding it also to mapRelay is small. If mapRelay is the only place the tx is stored, the additional size is much larger.
 4117:19 <lightlike> so the size doesn't only depend on the number of transactions, but also on how many of its txns are no longer in the mempool (which is hard to predict)
 4217:21 <stickies-v> come to think about it, I feel a bit weird about calling that unpredictability a downside
 4317:21 <stickies-v> it's obviously much preferable over storing the entire tx in mapRelay, regardless of whether or not it already exists in mempool
 4417:23 <lightlike> stickies-v: yes, that's true,, it would be much worse if we did that. But what is not nice is that if for some reason there is a situation with a lot of mempool turnover, mapRelay could become very large in memory consumption.
 4517:24 <lightlike> next q: What problem is solved by introducing m_most_recent_block_txs?
 4617:24 <stickies-v> but then I think the real problem is that mapRelay was unbounded in size?
 4717:25 <mutex> that's what I read from the comments, having an unbounded datastructure is a concern
 4817:25 <stickies-v> (regardless: it highlights the limitations that mapRelay had, e.g. it could easily be abused with high RBF turnover, so perhaps I'm just arguing semantics)
 4917:26 <lightlike> stickies-v: yes, just bounding it in size (e.g. by evicting earlier when it gets too large) would've been an alternative possibility, but in my opionion that would only make sense if mapRelay had an important use case in the first place (which doesn't seem to be the case anymore)
 5017:27 <stickies-v> lightlike: yup, fully agreed
 5117:28 <abubakarsadiq> I think to be able to relay transactions that was mined in the most recent block, because it will be dropped from the mempool, hence m_most_recent_block_txs will help
 5217:28 <stickies-v> lightlike: when a new block comes in and we validate it, we remove those transactions from our mempool, but we still want to be able to relay those recent txs to our peers, because they're very relevant
 5317:29 <lightlike> abubakarsadiq, stickies-v: correct!
 5417:29 <lightlike> Follow-up q: Do you think it is necessary to introduce it (as opposed to just removing mapRelay without any replacement)?
 5517:29 <lightlike> (I added this because im not completely convinced)
 5617:31 <stickies-v> isn't that helpful for block propagation speed? if you receive a block but don't yet have all txs, you'd want to have as many peers as possible that can relay you that tx?
 5717:31 <lightlike> I wonder about the timing here. When we have received and connected a new block, wouldn't we immediately announce it to our peers, so they'd get the transactions from us anyway during compact block download?
 5817:31 <lightlike> so they wouldn't need to request them via GETDATA anyway?
 5917:32 <stickies-v> what about peers that don't use compact blocks?
 6017:33 <lightlike> stickies-v: those peers would ask us for the entire BLOCK msg, it doesn't help them at all if we sent them the txns via GETDATA.
 6117:33 <stickies-v> oh right
 6217:35 <stickies-v> hmmm. would this be helpful during a chain split? a peer on a different tip would still benefit from being able to get the tx, and potentially won't have received it as part of a block?
 6317:37 <lightlike> stickies-v: yes, that might be the reason.
 6417:37 <lightlike> I think I'll try to add some extra logging on my node to see how often we'd actually answer getdata requests from m_most_recent_block_txs.
 6517:38 <lightlike> next question: What are the memory requirements for m_most_recent_block_txs compared to mapRelay?
 6617:41 <stickies-v> i don't have the numbers, but it's bounded to the (in-memory) size of the `CTransaction`s in the previous block, plus a bit of overhead for the index etc?
 6717:43 <abubakarsadiq> stickies-v +1: unlike mapRelay which is unpredicatable
 6817:44 <lightlike> I think it's even less. We save the last block anyway in m_most_recent_block, which includes CTransactionRef (the shared pointer) to all its txns. So even after the txns are removed from the mempool, I think there won't be extra storage requirement for the actual transaction data.
 6917:46 <stickies-v> ooohh I didn't see that, that's neat, yeah I think you're right
 7017:46 <lightlike> next q: Are there scenarios in which transactions would be made available for a shorter or longer time than before as a result of this change?
 7117:47 <Anton> depends on the mining speed?
 7217:49 <stickies-v> with `RELAY_TX_CACHE_TIME = 15min`, it looks like txs were expired from mapRelay after 15 mins, so I suppose they're available longer whenever there is more than 15 mins since the last block?
 7317:49 <Anton> if mapRelay stored 15 minutes of blocks, then it could store more than a single block at certain times
 7417:51 <lightlike> yes - if blocks take longer, we'd keep some txns around for longer now.
 7517:51 <stickies-v> and I think shorter whenever last block time <15 mins?
 7617:51 <lightlike> which makes perfect sense to me.
 7717:52 <lightlike> yes - if there are e.g. two blocks in <15 mins, the txns from the earlier wouldn't be kept around anymore
 7817:52 <lightlike> this seems ok, after all the 15 minute limit was rather arbitrary
 7917:53 <stickies-v> block height is the only true time <3
 8017:54 <lightlike> and the peers should rather catch up with the chain instead of asking for old transactions, so I don't really see a downside.
 8117:55 <lightlike> but it might decrease availability of txns in case of chainsplits >1 maybe (which are extremely rare)
 8217:55 <lightlike> last question: Can you think of any other possible downsides of removing mapRelay?
 8317:56 <lightlike> I couldn't think of any, so I don't really expect any answers here :)
 8417:56 <stickies-v> I think in case of reorg, having the `CONFLICT` transactions still in your mapRelay gives you a slight speed advantage?
 8517:57 <instagibbs> only I think in the case of a reorg race? if you've marked it conflicted, you're now on a heavier chaintip, and you want peers to get stuff on that heavier chaintip
 8617:57 <instagibbs> if there's a back and forth reorg, maybe
 8717:57 <instagibbs> but that's no bueno
 8817:59 <stickies-v> instagibbs: I don't actually mean from a relay pov, just for yourself to more quickly be able to validate the new chaintip? since you still have the tx you don't need to fetch it from a peer?
 8917:59 <stickies-v> you're right that you wouldn't want to relay it to your peers
 9017:59 <lightlike> stickies-v: but would you actually try to fetch it from mapRelay in that scenario?
 9118:00 <stickies-v> as in - does if the code currently does that? or if it's desirable? the former no idea, the latter I think so?
 9218:00 <lightlike> I meant the former.
 9318:01 <stickies-v> yeah sorry no idea :-D also this is quite edge case and probably (definitely?) not worth optimizing for, just the only thing i could come up with
 9418:01 <lightlike> in any case, time's up.
 9518:01 <lightlike> Thanks everyone!
 9618:01 <abubakar> thanks for hosting lightlike
 9718:01 <stickies-v> thank you lightlike! and MacroFake for authoring
 9818:01 <emzy> Thanks lightlike and everyone else!
 9918:01 <instagibbs> spending time avoiding reorgs through faster prop seems better value than handling them slightly faster... gut feeling :) thanks lightlike
10018:02 <stickies-v> yeah i agree
10118:03 <stickies-v> #endmeeting