How does a node signal to the network that it can serve compact block filters?
What is the behavior of a node that has not finished generating and indexing
compact filters for every block in the chain?
Does this PR expose any new denial-of-service attack vectors? If so, how are
they being handled? Could this be improved?
What is the purpose of the cfcheckpt message?
Can Bitcoin Core be used to retrieve compact block filters from another peer?
How is the test coverage in this PR? Do you think it could be improved, and how?
Are there other ways to test this PR outside of the usual testing framework?
<jnewbery> nobody123: not really a class. Look in net.cpp for the low-level peers and connections management, and then in net_processing.cpp for the application-level logic
<pinheadmz> Ok, so on the topic of the service bit -- Did anyone notice the sort of open-ended question about this service bit as it relates to this PR?
<andrewtoth> From jimpo in the PR: "And if I understand correctly, we re-advertise our local address with service bits to each peer every day on average, so the network should see the change eventually without the node operator having to do anything." Where does this happen?
<headway-translat> pinheadmz: purpose of compact filters is to create a space-efficient proof of txns non-inclusion in a block so that light clients have cheaper access to truth. e.g. lightning light clients can space / bandwidth efficiently certify that channel closure txns etc aren't in blocks
<jnewbery> so if I start by telling my peers that I don't support a feature, that'll get gossipped around the network for some time after I change my service bit flags
<pinheadmz> I suppose the question for light clients would be: if you request a cfilter and the node respnds with notfound or null -- is that reason to disconnect or ban? Because that node could still just be syncing
<pinheadmz> andrewtoth: good question - until the cfilters are committed in the block somehow (and therefore verified by all network nodes and ensured by proof of work) - there is no way to know without checking the block yourself
<michaelfolkson> You could have two flags. One saying that you intend to provide them and one saying you're not ready to provide them? After a certain period of time the gossip would ignore the second flag?
<jimpo> jnewbery: Yes, that is my concern. That allowing a valid case where a node advertises that it can serve filters and is unable to would complicate client logic.
<jimpo> pinheadmz: There was a change to the BIP last year that made them contextual. Specifically, the filters include the prevout output scripts, which are not included in blocks, only referenced by inputs.
<jnewbery> I think you need that logic anyway. In an untrusted p2p network, you don't know who is advertising that they can serve the filters: malicious nodes, buggy implementations, stalling node, unreliable connections, ... A BIP 157 client has to be able to handle cases where it gets invalid or missng reponses
<andrewtoth> michaelfolkson: what would be the difference to just signaling the first bit and then disconnecting after the ignore time time if they don't start providing?
<Talkless> jimpo: but that's only about tx'es inside actual block, transactions queued in mempool will not be delivered through this mechanism to the light wallets?
<sanoj> jimpo: what about a state that returns some sort of sync message. I wouldn't imagine it'd happen that often but it would avoid the disconnect for proper behavior and maintain the validity of the capabilities service bit
<jnewbery> A bit more on the contextual/non-contextual stuff: the new scripts created in the block can obviously be constructed from the block. The scripts spent in the block require a node that has the UTXO set or the 'undo data' for the block
<michaelfolkson> andrewtoth: I suppose it depends whether you are concerned about maintaining connections or not. If you're not then sure advertise that you offer something and get disconnected when you disappoint
<pinheadmz> jimpo: I was wondering about the tiny amount of time it would take a fully synced node to produce the cfilter... there could be a moment where even a synced node can't provide the filter for the tip ?
<jnewbery> I personally don't like it for a couple of reasons: service bits are handled by our net layer, and I don't like coupling together net with the compact blocks index. second: I think there's a general assumption that service bits don't change and I'd prefer not to break that
<headway-translat> Talkless CMIIR but i believe how the encoding works, to have higher degree of soundness, client would need to request filters several blocks before their bday
<michaelfolkson> andrewtoth: I think jnewbery answered this earlier. The incorrect service bit would percolate around the gossip network and be a pain to replace
<jnewbery> andrewtoth: when peer addresses are advertised in ADDR messages, that peer's service bits are included, so you can think of other nodes caching the peer's service capabilities
<jnewbery> sanoj: 'checkpoints' might be a bit confusing terminology here because we also have checkpointed blocks in Bitcoin. Here, cfcheckpoints are just an optimization for downloading the full set of compact block filters for the chain
<michaelfolkson> What was implementing Neutrino on Bcoin like pinheadmz? Did you consider similar issues? Anything you will revisit now it is (almost) fully implemented on Core?
<jkczyz> jnewbery, jimpo: Peers can send notfound in response to getdata. Would something analogous be suitable for when the filters are not ready? Is there some general error mechanism in Core's P2P layer?
<jnewbery> they do need to keep track of confirmations, so they'd need to work out whether any transactions that they're interested in had been removed from the block chain during the reorg. That doesn't require undo data
<tuxcanfly> michaelfol: to answer your bcoin question - it was pretty straightforward... however there were some bip updates which needed to be synced... the test vectors were helpful in that regard, to make sure we're core compatible for every edge case
<tuxcanfly> michaelfol: the p2p cfilter serve was part of the PR but didn't enough review so it isn't merged just yet... we will be getting that one soon
<tuxcanfly> ah well... I was just curious if more thought was given to committing cfilters to blocks via coinbase (something similar was mentioned in the btcd/bitcoind comments)
<fjahr> I thought for a moment the filters in the test might be all 0's because there are no transactions in the blocks but forgot about the coinbase of course...
<jnewbery> Talkless: either is fine. If you generate a new private key randomly, you know for sure that there haven't been any sends to that key before that time
<jnewbery> pinheadmz: I haven't seen any discussion of that recently. I think we'd want non-committed BIP157 filters to be used for a long time before proposing a soft fork to commit to them