Erlay: bandwidth-efficient transaction relay protocol (
p2p) Feb 10, 2021
The PR branch HEAD was c452e5d at the time of this review club meeting.
In this PR Review Club meeting, we’ll discuss
and the high-level concepts behind Erlay.
We’ll also look at the first few net processing commits in more detail:
Erlay is a proposal for a new method of transaction relay based on a
combination of flooding and reconciliation (the current transaction relay is
flooding-only). The idea was presented in a 2019 paper,
, and is
Transaction Relay for Bitcoin BIP330.
Reconciliation works asymmetrically in Erlay, depending on the direction of
the connection. When we interact with an outbound peer (i.e. we initiated the
connection), we are the requester of a reconciliation, and our peer is the
responder (vice versa when interacting with an inbound peer).
In the simplest scenario, being a requestor means that we send a
reconciliation (message REQRECONCIL), receive an answer back (SKETCH), and
reply with the reconciliation differences (RECONCILDIFF) that we can extract
by comparing our sketch with the received one. At this point, both parties
know the set of transactions their peers might require that are known to both.
A sketch is a representation of a set of transactions (or their short IDs)
that is optimized for a specified space. Erlay implements an existing set
reconciliation algorithm called PinSketch.
Erlay does not completely abandon flooding but uses it much more sparingly.
Nodes will still flood certain transactions to a limited number of peers; the
goal is that only well-connected publicly reachable nodes flood
transactions to other publicly reachable nodes via outbound connections.
Did you review the PR?
Concept ACK, approach ACK, tested ACK, or
What advantages does a reconciliation-based approach to transaction relay
have over the current flooding method. What are the trade-offs?
Which of the existing message types participating in transaction relay will
be sent less often with Erlay? Which ones will be unaffected?
Can you explain
why a reconciliation-based approach scales better than flooding
(after all, the reconciliation messages also consume regular traffic)?
How do two peers reach agreement about whether to use reconciliation?
If two peers have agreed to use reconciliation, does that mean there will
be no flooding on this connection?
MAX_OUTBOUND_FLOOD_TO is set to 8. Considering that 8 is also the
maximum number of outbound connections participating in transaction relay,
why do you think this value was chosen?
Can you think of possible new attack vectors that would specifically apply
1 18:00 <jnewbery> #startmeeting
2 18:00 <jnewbery> hi folks! Welcome to PR Review Club. Feel free to say hi to let everyone know you're here.
6 18:00 <jnewbery> (or not. lurkers are also welcome at this club)
8 18:00 <joelklabo> Hey everyone 👋
14 18:00 <michaelfolkson> hi
17 18:00 <AnthonyRonning> hi
18 18:00 <NelsonGaldeman56> Hi!
20 18:00 <gleb> hi! Thank you for coming.
21 18:00 <jnewbery> Thank you to lightlike for hosting and to gleb for writing the PR!
23 18:01 <jnewbery> is anyone here for the first time?
28 18:01 <dergoegge> first time not lurking
30 18:01 <OliP> First time here!
31 18:01 <jonatack> hi (lurking while reviewing 18017 ;)
33 18:01 <DylanM> Hi first time, lurking mostly
34 18:01 <gleb> A lot of new names since I attended last time, great :)
35 18:01 <jnewbery> dergoegge: lovely. Thanks for joining us!
37 18:01 <jnewbery> OliP, DylanM: welcome, welcome
39 18:01 <OliP> jnewbery: Many thanks
40 18:02 <jnewbery> couple of reminders. We're all here to learn. Lightlike will guide the discussion but feel free to ask questions at any time
41 18:02 <jnewbery> Also, you don't have to ask if you can ask a question. Just ask!
45 18:02 <willcl_ark> great turnout!
46 18:02 <jnewbery> And one more reminder before I hand over: I'm always looking for hosts for review clubs. If you think that's something you'd like to have a go at, please message me
47 18:02 <jnewbery> ok, enough of me. Over to lightlike
48 18:03 <lightlike> Ok, thanks jnewbery - so today we'll talk about Erlay!
49 18:03 <lightlike> First, since this a really large PR and there is also a BIP and a paper with lots of information, we won't get deep into too much of the code today.
50 18:03 <lightlike> (that would be a possibility for a follow-up)
51 18:03 <lightlike> But let's start with first things first - Who had the time to review the PR this week? (y/n)
60 18:04 <AnthonyRonning> n
62 18:04 <jnewbery> y (just the four commits)
65 18:04 <emzy> y (without the code)
66 18:04 <michaelfolkson> y
67 18:04 <dhruvm> n(ot yet)
68 18:04 <glozow> looked at bip, 0.2y for the PR
69 18:04 <joelklabo> n (read about erlay)
70 18:05 <lightlike> Great! So what is your initial impression? (Concept ACK or NACK)
71 18:05 <emzy> Concept ACK
72 18:05 <rich> Concept ACK
73 18:05 <dergoegge> Concept ACK
74 18:05 <pinheadmz> concept ACK
75 18:05 <joelklabo> concept ACK
76 18:05 <lightlike> seems still a bit early for code ACKs :-)
77 18:05 <OliP> Concept ACK
79 18:05 <fodediop> Concept ACK
80 18:05 <dhruvm> Concept ACK - the gains seem incredible!
81 18:05 <AnthonyRonning> concept ack
82 18:05 <sergei-t> concept ack (based on prior knowledge about erlay)
83 18:06 <svav> Concept ack
84 18:06 <lightlike> Ok - that's a lot of concept ACKS!
85 18:06 <lightlike> So, let's dive into the questions:
86 18:07 <lightlike> What advantages does a reconciliation-based approach to transaction relay have over the current flooding method. What are the trade-offs?
87 18:07 <emzy> Less bandwidth usage at the cost of slower propagation of TX through the network.
88 18:07 <dhruvm> a reconciliation-based approach minimizes redundant INV messages sent to nodes which already have the transaction. 224/524 ~ 42.7% of all bytes exchanged for tx relay are redundant if flooding is used in contrast.
89 18:07 <ecola> savings in bandwith
90 18:07 <pinheadmz> less bandwidth required
91 18:07 <AnthonyRonning> Significantly reduces bandwidth and allows for better connectivity. It increases the time it takes to propagate a tx to all nodes.
92 18:07 <dergoegge> less bandwidth usage, increased connectivity and privacy benefits vs. slightly slower tx propagation.
93 18:07 <pinheadmz> obfuscation of the network graph perhaps ?
94 18:07 <tuition_> lower transaction bandwidth use at the cost of marginal increases in txn propagation delays
95 18:08 <fodediop> It allows for susbtantial bandwidth savings on trasaction propagation
96 18:08 <sergei-t> Con: more complex protocol logic, the notion of "well-connected publicly reachable" nodes (centralization risks?)
97 18:08 <dhruvm> also the growth in bandwidth is sub-linear to number of peer connections per node.
98 18:08 <pinheadmz> trade offs might be the extra round trips if txs are missing
99 18:08 <willcl_ark> It will allow nodes to make more connections due to lowering bandwidth usage, and therefore make eclipse attacks more costly/difficult
100 18:08 <joelklabo> less bandwidth, more code complexity
101 18:09 <sergei-t> Privacy implications: do peers now know more about which txs I have and don't have?
102 18:09 <sipa> pinheadmz: that's already accounted for in the model
103 18:09 <tuition_> willcl_ark nice point about decreasing bandwidth constraints allowing more peering to mitigate eclipse attacks
104 18:09 <dhruvm> sergei-t: peers already knew that with flooding i think
105 18:09 <ajonas> +1 willcl_ark and that plays nicely with dhruvm's comment that things only gets better with more conns
106 18:10 <pinheadmz> tradeoff then might be computational resources
107 18:10 <gleb> sergei-t: Erlay attempts to leak no more than already leaks through flooding. In practice, reviewers should confirm the defences indeed work.
108 18:10 <pinheadmz> extra math per-peer
109 18:10 <lightlike> lots of great answers here! I think especially the scaling for more connections are important.
110 18:10 <sipa> tuition_: yeah, that's the big one; the savings are modest for current average connection counts, but with increased connections the benefits grow
111 18:11 <rich> seems to be some implication that tx origination could be obscured since it's not pushed out preemptively via inv (amiti?)
112 18:12 <lightlike> Not sure about this, but at the current connectivity (8 OB), probably a similar bandwidht saving could be gained by only using short TX Ids with Salting (as is part of Erlay) but no reconciliation
113 18:12 <lightlike> sipa, gleb: would you agree to this?
114 18:12 <amiti> rich: good question, I don't know the answer :)
115 18:13 <sipa> lightlike: yeah, with salting
116 18:13 ℹ Dulce is now known as Guest73056
117 18:13 <gleb> lightlike sipa: but then it's hard to deduplicate.
118 18:13 <dhruvm> rich: origination seems to be done with reconciliation and not flooding, so originator gets privacy iiuc
119 18:13 <gleb> I would say just short ids are not so easy to get done right as well
120 18:14 <sipa> gleb: right
121 18:14 <lightlike> ok, moving on: Which of the existing message types participating in transaction relay will be sent less often with Erlay? Which ones will be unaffected?
122 18:14 <pinheadmz> a lot less INV
123 18:14 <dhruvm> INV will be sent less often. TX messages will be unaffected.
124 18:14 <emzy> INVs will be send less often. The missing TX itself still needs to be send.
125 18:14 <sipa> dhruvm: both recon-based announcement and inv announcement leak which txids the sender has; the point is that the recon-based ones are less frequent (on a per peer basis)
126 18:15 <pinheadmz> and a little less GETDATA as well i presume?
127 18:15 <dergoegge> unaffected: tx, getdata less: inv
128 18:15 <dhruvm> sipa: ah, that makes sense.
129 18:15 <lightlike> pinheadmz: why would there be less GETDATA?
130 18:15 <gleb> pinheadmz: why do you think less getdata? I might be forgetting somethign at this point :)
131 18:15 <dhruvm> pinheadmz: there should be less GETDATA if the missing txes are known at set-difference time?
132 18:16 <pinheadmz> could be wrong but curent flood goes INV-> then GETDATA<- then TX-> right?
133 18:16 <gleb> number of getdata = number of txs, logically, right?
134 18:16 <felixweis> i wonder if sending a sketch of transactions that are dependent and required together to meet minrelayfee requirements of a nodes mempool can help with package relay?
135 18:17 <willcl_ark> sipa: Does it not make some timing analysis attacks to detect origination more difficult or, less accurate?
136 18:17 <gleb> felixweis: Hard to talk about optimizations while we have no basic design for package relay...
137 18:17 <sipa> pinheadmz: every node should GETDATA every transaction exactly once
138 18:17 <sipa> pinheadmz: both before and after erlay
139 18:17 <dhruvm> pinheadmz: I think that's right and am wondering why we'd need as many GETDATA as well. At set difference, the other node could just send the TX messages.
140 18:17 <lightlike> I think it's important thet the current flow with INV->GETDATA->TX will be unchanged - it's just that the initial INV is sent only for those transaction we are reasonably sure our peer actually needs.
141 18:17 <glozow> right now I think you'd only request 1 getdata at a time, unless you got it by txid and there's different wtxids you don't know about
142 18:17 <pinheadmz> sipa a ha right
143 18:18 <pinheadmz> but do we still use GETDATA to reconcile if a tx is missing?
144 18:18 <pinheadmz> i thought bip330 introduced other messgaes
145 18:18 <glozow> yeah, gettx
146 18:18 <gleb> pinheadmz: you're probably referring to an older version of the bip
147 18:18 <glozow> oh, i guess if you're sending gettxes then you're sending fewer getdatas
149 18:19 <gleb> pinheadmz: yeah, no gettx there, right?
150 18:19 <gleb> The latest version is linked in the PR.
151 18:19 <rich> willcl_ark: that is also how I understood "the point is that the recon-based ones are less frequent (on a per peer basis)"
152 18:19 <pinheadmz> oh I see `reconcildiff` triggers the usual inv->getdata->tx
153 18:20 <gleb> pinheadmz: exactly!
154 18:20 <lightlike> pinheadmz: yes!
155 18:20 <pinheadmz> i guess i thought it was more like getblocktxn from compact blocks
156 18:20 <ariard> felixweis: thought about it either adding another snapshot for package ids only or eating the bullet of wasted bandwidth in case of redundancy
157 18:20 <gleb> dhruvm: in short, sketches operate over short ids, but we can't request by short ids for reasons.
158 18:20 <felixweis> whats the rationale for the last point in Short ID computation? 1 + s mod 0xFFFFFFFF. why not use s already?
159 18:21 <sipa> felixweis: the pinsketch algorithm needs item IDs that are nonzero
160 18:21 <lightlike> next question:
161 18:21 <dhruvm> gleb: i see. are the reasons documented somewhere?
162 18:21 <felixweis> ah! makes sense
163 18:21 <ma33> Is their an analysis of when Erlay becomes too computationally expensive? I remember seeing a comparison with another set-recon scheme in the paper with up to 50 differences. What if the number of differences increases to, say, 500?
164 18:22 <lightlike> Can you explain why a reconciliation-based approach scales better than flooding (after all, the reconciliation messages also consume regular traffic)?
165 18:22 <felixweis> re-reading the next sentence would have helped already lol
166 18:23 <pinheadmz> lots of redundancy sending 255 byte transactions to all nodes all the time
167 18:23 <rich> It doesn't if you only have one peer, but with more peers sending the same tx inv, there is deduplication savings.
168 18:23 <lightlike> ma33: I think in the paper there is a suggestion to make larger set differences computationally viable via bisection - which is not needed for bitcoin though becuse typical differences are smaller.
169 18:23 <ariard> dhruvm: 32-bit short ids means you can have tx-relay jamming based on malicious collisions
170 18:23 <sipa> pinheadmz: again, the transactions are only sent once already
171 18:23 <willcl_ark> instead of receiving every tx from every peer, you just receive it from one
172 18:23 <gleb> dhruvm: that's one of the design choices we made while working on the implementation, I think you get it naturally when you review the protocol closely. Not sure all of them should be documented. But we can discuss this as well
173 18:23 <sipa> willcl_ark: transactions are only sent once already
174 18:24 <pinheadmz> sipa we dont re-broadcast entire TX to every peer besides the one who sent it to us ?
175 18:24 <pinheadmz> (currently, before PR)
176 18:24 <dhruvm> ariard: gleb: thanks.
177 18:24 <gleb> ma33: yeah, check graphs in the minisketch repo. It has nice data on the time it takes to compute
178 18:24 <sipa> pinheadmz: we _announce_ it to every other peer, by txid; not the full tx, obviously
179 18:24 <pinheadmz> ahaahahaha yes
180 18:24 <sipa> the peer requests the actual transaction from one node that announced it
182 18:24 <pinheadmz> so what were saving is 32 byte txid > 32 bit short txid
183 18:25 <sipa> nope, far more than that
184 18:25 <sipa> because erlay doesn't send short ids; it sends sketches
185 18:25 <pinheadmz> oh right
186 18:25 <sipa> and the size of sketches scales with the bandwidth of actual differences
187 18:25 <rich> I wonder what the crossover point for number of peers where reconciliation starts to pay off?
189 18:26 <rich> I suppose it also has to due with how interconnected your peers are too.
190 18:26 <lightlike> yes, that is the core of it. If we just sent short txids over each link as part of the recon, the scaling wouldn't be better than now.
191 18:26 <sipa> rich: yeah, and on the exact parametrization; i believe gleb experimented with various configurations
192 18:26 <gleb> yeah, the data is in the paper.
193 18:26 <maqusat> only diff of tx ids sent instead of the whole set
194 18:27 <gleb> rich: I think 9 peers actually. Erlay would cost 8x, short ids would pay 9x, current flooding is 32x (hence full txid)
195 18:28 <gleb> or not.... sorry, disregard for now.
196 18:28 <rich> gleb: do we have some way to model how interconnected typical peers are?
197 18:28 <lightlike> sipa, gleb: how would Erlay deal with it if someone dumped thousands of transactions at the same time into the network (e.g. for utxo consolidation)? would sketches be able to deal with that?
198 18:29 <ma33> rich TxProbe paper had some stats on the Bitcoin graph, albeit only on testnet
199 18:29 <gleb> rich: my simulator generated a topology similar to bitcoin today: 10k reachable nodes + 50k non-reachable nodes. And then random distribution. That's it
200 18:29 <rich> gleb: +1 seems reasonable
201 18:30 <gleb> lightlike: i haven't tried this scenario. Worst case we waste a limited amount of bandwidth and fallback to flooding.
202 18:30 <dhruvm> lightlike: while that might make the sketch diffs larger it should still be atleast as good as flooding right (and then some due to short txids)?
203 18:30 <sipa> lightlike: if you dump 1000s of transactions on any given 0.21 node, the ones above 5000 will just be ignored; the rest would be requested with some delay... then that peer would start propagating them, but at a limited rate
205 18:31 <sipa> erlay doesn't really come into play much, i think
206 18:31 <rich> ariard: thx!
207 18:31 <lightlike> ok, thanks!
208 18:31 <dergoegge> why is the reconil. frequency fixed at 1 every 2 seconds? could it make sense to have the frequency change based on how many new txs the node has seen in the last x hours/minutes? (i.e. reconcil more frequently if there are more txs coming through)
209 18:32 <gleb> dergoegge: that's possibly, I kind of assumed a "normal" operation of the network in the window of 3..14 txs per second appearing randomly in the network.
210 18:33 <lightlike> moving on with the q (getting into the actual commits):
211 18:33 <lightlike> How do two peers reach agreement about whether to use reconciliation?
212 18:33 <dhruvm> When a node receives a WTXIDRELAY message from a peer that supports tx-relay, it announces a SENDRECON message and requests to act as a reconciliation-requestor for outbound connections and a reconciliation-responder for inbound connections.
213 18:33 <sipa> gleb: just because the initiating node hasn't received many transactions doesn't mean there aren't many; it may just not have heard about them>
214 18:34 <glozow> do i have this correct? :
215 18:34 <glozow> both send `SENDRECON`, both must not have txrelay off, both must use wtxid relay, peer who initiates outbound must have requestor=on/responder=off, peer who receives inbound connection must have requestor=off/responder=on
216 18:34 <sipa> dergoegge: more generally, i think it's a bit scary to make the frequency of propagation depend on seen transactions (e.g. i could imagine some scary fix point where all nodes end up deciding there are no transactions, and stop reconciling entirely...)
217 18:35 <lightlike> dhruvm, glozow: exactly! both nodes send a SENDRECON message.
218 18:35 <ariard> glozow: I think they should be mauve peers to be correct
219 18:35 <glozow> ariard: mauve?
220 18:35 <lightlike> so everything is happening during VERSION negotiation and fixed for the lifetime of the connection - no later changes.
221 18:35 <pinheadmz> as part of the version handshake, after WTXID is set
222 18:36 <ajonas> Well we would fall back to flooding on a conn that supports reconciliation if we don’t have enough outbound flood conns, right?
223 18:36 <jnewbery> Is there a reason that both parties add salt?
224 18:37 <glozow> so that nobody can get nodes to use duplicate salts?
225 18:37 <pinheadmz> jnewbery so attackers cant try to find coliding short ids
226 18:37 <pinheadmz> or if they do itll only affect one pair of nodes ;-)
227 18:37 <dergoegge> gleb, sipa: i see. although there could be a limit say "reconciliate at least every ~n seconds" to avoid stopping entirely
228 18:38 <sipa> dergoegge: yeah, not saying that doesn't make sense, just that it does bring in some possibly scary considerations
229 18:38 <gleb> two salts may be redundant actually, what if the salt was always picked by the "connection initiator" side randomly per peer, and they use that?
230 18:39 <gleb> what do you think guys? sipa?
231 18:39 <sipa> gleb: i think that's probably ok, but i find it easier to reason about if both contribute
233 18:39 <dergoegge> sipa thats true, thanks for the answer
234 18:39 <ma33> If I understand it correctly, a node can request a sketch from only a maximum of 8 of its (outbound) peers, correct? Or do peers in blocks-only-mode also participate in recon?
235 18:39 <jnewbery> I don't understand the attack. I could make more than one of my peers have the same salt?
236 18:39 <gleb> yeah, it's indeed "safer" when both contribute
237 18:40 <jnewbery> but I can't force the same salt transitively to other connections
238 18:40 <sipa> jnewbery: the biggest reason for the salt is that not all connections use the same salt; if there were, an attacker could easily grind transactions that propagate very badly over all connections
239 18:40 <jnewbery> certainly I understand the reason for a salt per connection, just not the need for both the requestor and responder to contribute
240 18:41 <sipa> so if both contribute entropy, you can just say "assume not both parties are the grinding attacker", which is obviously true - the attacker doesn't need to relay to themselves
241 18:41 <gleb> ma33: not blocks-only, no, but all tx-relaying peers can be requested for a sketch (if they indicated support). Yes, this is *currently* limited to 8.
242 18:41 <dhruvm> ma33: recon seems like a tx-relay thing, not sure blocks-only nodes would get the network characteristics they desire with recon. but erlay perhaps reduces the need to be blocks-only?
243 18:41 <lightlike> ma33: It would request sketches from all of its peers, its just that the current limit for outbounds is 8.
244 18:41 <sipa> blocks-only connections definitely should not do erlay; it reveals information about their mempool
245 18:42 <lightlike> ajonas: your question wrt fallback to flooding kind of touches my next question
246 18:42 <jnewbery> the party that sends the SENDRECON message second can grind the salt value
247 18:42 <lightlike> If two peers have agreed to use reconciliation, does that mean there will be no flooding on this connection?
248 18:42 <rich> if I understand correctly, blocks-only would still be lower bandwidth than erlay, but with the limitation of no mempool
249 18:42 <dhruvm> I am not certain but it seems flooding is between listening nodes. Reconciliation is between all pairs of peers?
250 18:43 <rich> (and blocks come too infrequently to worry about inv message deduplication for them)
251 18:43 <sipa> jnewbery: yes, but they can't grind it so that the same combined salt is used on all their connections
252 18:44 <sdaftuar> sipa: does it matter if a peer chooses to make relaying with themselves not work?
253 18:44 <sdaftuar> ie, they could just not relay transactions anyway
254 18:44 <gleb> dhruvm: what you're referring to is an *intuition* behind how erlay achieves it's properties. In practice, the relay policy is a bit more specific. This is because we have no notion of reachable node in the codebase (a node can't for sure know if it's reachable or not )
255 18:44 <dergoegge> There will be at least 8 flooding connections even if they also support reconciliation. If the reconciliation set is full we also fall back to flooding i think.
256 18:44 <pinheadmz> and what is the worst case scenario for a succesful shortid collision? just one peer doesnt get a tx message? theyll see it in a block at least
257 18:44 <lightlike> rich: yes, blocks-only is strictly lower bandwidth - but obviously tx relay is necessary for the network, so you dont contribute that by being blocks-only.
258 18:44 <ariard> dhruvm: beyond the bandwidth saving, blocks-only are also (hopefully!) hidden peers, making it harder to partition a victim node
259 18:44 <sipa> sdaftuar: generally in cryptographic analysis you always assume that at least one of the involved parties is honest; if everyone is an attacker, who cares?
260 18:44 <sipa> i think the same applies here
262 18:45 <ma33> gleb got it, thanks
263 18:45 <ma33> dhruvm nice follow up question. any ideas?
264 18:45 <sdaftuar> sipa: not sure i follow -- the idea is that one of the peers can break relay on their own link. that doesn't generalize to any other connections on the network that don't involve them
265 18:45 <dhruvm> gleb: ah, i should have read more than the paper :)
266 18:45 <sdaftuar> i'm just saying they could do that already by not relaying to begin with
267 18:46 <sipa> sdaftuar: ah yes
268 18:46 <dhruvm> ariard: not sure i follow. how are block-only peers hidden?
269 18:46 <sipa> dhruvm: connectivity graph is much harder to infer for blocks-only connections (because of no addr and no tx relay)
270 18:46 <sdaftuar> i don't know that it matters much either way, no downside either to both sides contributing entropy
271 18:47 <lightlike> dergoegge: yes, but those connections won't be flooding only. We would flood TO up to 8 peers if we are a reachable note, but still get txes FROM these peers via reconciliation
272 18:47 <jnewbery> sdaftuar: yes, this is what I was trying to ask earlier. The worst I could do would be have the same salt for all my peers, which isn't attacking anyone else
273 18:47 <sdaftuar> jnewbery: yeah that makes sense to me
274 18:47 <gleb> jnewbery: this is my understanding as well.
275 18:48 <sipa> right, that's a fair point
276 18:48 <dhruvm> sipa: i see. so two incentives to be blocks-only: protection from graph inference and bandwidth reduction.
277 18:48 <dergoegge> lightlike: ah yes, thanks for clarifying
278 18:48 <gleb> In some older design, we needed 2 salts, but not really anymore
279 18:48 <sipa> dhruvm: indeed
280 18:48 <sdaftuar> one thing that occurred to me is if having a bad salt incurred cpu cost on your counterparty
281 18:48 <sdaftuar> in which case that might be a motivation to just avoid the influence of an attacker in that regard, but no idea if that might be true
282 18:49 <lightlike> next q:
283 18:49 <lightlike> In the Limit transaction flooding commit, MAX_OUTBOUND_FLOOD_TO is set to 8. Considering that 8 is also the maximum number of outbound connections participating in transaction relay, why do you think this value was chosen?
284 18:49 <lightlike> (already partly answered I think)
285 18:49 <ariard> dhruvm: if you're picked up as a outbound block-relay-only the protection for graph inference is also benefiting your peer
286 18:50 <ariard> it's a shared incentive
287 18:50 <sipa> sdaftuar: i believe siphash does assume that keys are random; if the exact key can be chosen exactly, there may be attacks beyond the ability to grind colliding transactions (e.g. i could imagine that some pathological keys exist that result in strictly more collisions that average keys); the actual key is computed with SHA256, but with too little entropy, that might be grindable too
288 18:50 <dhruvm> ma33: as sipa and ariard mentioned, the other incentive for block-only is security related.
289 18:50 <dhruvm> ariard: yeah that makes sense because the inference ability is on the link.
290 18:51 <gleb> I think this q is a hard one. MAX_OUTBOUND_FLOOD_TO=8 is based on my simultions: it provides low enough latency while not flooding too much (assuming we have more max conns)
291 18:52 <gleb> (low latency achieved in a combination of "diffusion" interval we wait before flooding out a tx, which was another parameter)
292 18:53 <lightlike> gleb: Ok - I thought part of the motivation was to not flood more than right now when, as a next step, the number ob MAX outbound connections is adjusted upwards.
293 18:53 <dhruvm> gleb: so it was the empirical sweet-spot between all-flooding(bandwidth-intensive) and all-reconciliation(latency-intensive)?
294 18:53 <gleb> lightlike: yeah, that's a way to think about it.
295 18:53 <rich> gleb: so can we think about it as the default outbound relay connections are added to the flood connections?
296 18:53 <gleb> dhruvm: yeah, how often are reconciliations vs how often we flood
297 18:54 <lightlike> Final question: Can you think of possible new attack vectors that would specifically apply to Erlay?
298 18:54 <gleb> rich: sorry, i don't understand your question
299 18:54 <rich> gleb: nevermind
300 18:55 <sdaftuar> lightlike: i think that's a great question that i hope reviewers give a lot of thought to!
301 18:55 <rich> perhaps new attack vectors related to latency if enough nodes reduce/exchange flood connections and you can somehow synchronize when reconciliations happen
302 18:56 <glozow> fingerprinting-type attacks? (which is why there are delays in responses)?
303 18:56 <tuition_> /me wonders about any sort of race conditions
304 18:56 <lightlike> gleb: in your simulations, did you simulate the mixed scenario of Erlay and legacy nodes? I am a bit concerned that, since flooding is faster, TX relay might be centralised towards the legacy nodes, and the new Erlay nodes only get the "leftovers" and don't really have anything to reconcile.
305 18:56 <gleb> glozow: yeah, I only accounted for a "first-spy estimator", and also my general understanding of spy heuristics. Def a lot of room for research
306 18:56 <ariard> targeting the limited size of the local reconcliation set?
307 18:57 <ariard> broadcasting hard to compute sketch?
308 18:57 <dhruvm> Reducing redundancy can potentially open up tx censorship, but I can't actually think of a specific attack.
309 18:57 <glozow> what are the upper bounds of how hard it could be to compute a sketch?
310 18:57 <gleb> lightlike: I think no, but that would mean erlay nodes just won't get the benefit in early days? That's possible.
311 18:57 <willcl_ark> what happens if the flood nodes censor certain transactions
312 18:58 <lightlike> gleb: it might also mean more bandwidth for legacy nodes.
313 18:58 <gleb> willcl_ark: every node gets every transaction anyway. Censorship can happen the same way it can happen today (if maaany reachable nodes censor a tx)
314 18:58 <ariard> willcl_ark: you damage their latency but they should propagate through reconciliation?
315 18:58 <tuition_> Do we expect miners will run with erlay and non erlay nodes? it seems they'll want to be maximally aware of all txns (hence non erlay) but will also want to guard against eclipse attacks (and maybe run block only hence no erlay)
316 18:58 <gleb> lightlike: need to think about this one
317 18:58 <rich> I guess you are creating/storing a sketch and holding onto it for some response, you'd want to be careful someone couldn't blow up your memory.
318 18:58 <lightlike> becasue they send all the TXes now before Erlay nodes get a chance
319 18:59 <gleb> tuition: if they run reachable nodes, even erlay-enabled, their latency is gonna be lower than what we have today!
320 18:59 <gleb> because in erlay, flooding artificial delays are reduced.
321 19:00 <ariard> lightlike: depend if you assume that reachable nodes are going to be deploy edfaster than non-listening ones?
322 19:00 <lightlike> ok, great discussion, time is up already!
323 19:00 <lightlike> !endmeeting
324 19:00 <glozow> thank you lightlike!
325 19:00 <willcl_ark> thanks lightlike, very interesting!
326 19:00 <ajonas> thanks lightlike
327 19:00 <ecola> thank you lightlike, very interesting !
328 19:00 <dhruvm> Thanks lightlike, gleb, sipa! This was 🔥!
329 19:00 <rich> thanks lightlike!
330 19:00 <dergoegge> thank you, this was fun!
331 19:00 <maqusat> thank you!
332 19:01 <shafiunmiraz0> Thank you lightlike
333 19:01 <jonatack> thanks gleb!
334 19:01 <OliP> Thank you!
335 19:01 <tuition_> thanks lightlike gleb sipa
336 19:01 <jonatack> thanks lightlike!
337 19:01 <gleb> I hope you guys stick around for the PR until it's done :)
338 19:01 <felixweis> thanks lightlike!
339 19:01 <fodediop> Thank you lightlike 🙏🏿
340 19:01 <effexzi> Thanks.
341 19:01 <jnewbery> thank you lightlike!