Restrict timestamp when mining a diff-adjustment block to prev-600 (mining)

https://github.com/bitcoin/bitcoin/pull/15481

Host: jnewbery  -  PR author: TheBlueMatt

Notes

  • This PR was proposed in preparation for Matt Corallo’s Great Consensus Cleanup softfork proposal. (Mailing list post), (proposed BIP).
  • The proposal was discussed at the recent Bitcoin Core devtech meetup.
  • This PR is connected with the timewarp fix part of the BIP: By further restricting nTime fields on difficulty adjustment blocks, we propose fixing the long-standing “timewarp” inflation vulnerability in Bitcoin’s difficulty adjustment without risking existing mining hardware becoming unusable. This limits the worst-case difficulty adjustment target in case of attack from the current exponential growth, to once every roughly 600 seconds.
  • The expected time period of 2016 blocks (one retargetting period) isn’t exactly two weeks. See this thread from Pieter Wuille on why exactly that is.
  • The PR doesn’t enforce that consensus change. Rather, it changes the mining behaviour of a Bitcoin Core node to be compliant with the proposed changes.
  • There are two changes:
    • One to getblocktemplate (the RPC that a miner calls in order to get a template block to work on). This PR changes the returned mintime field returned by getblocktemplate, such that if the next block is a difficulty adjustment block, it is constrained by the prev-600 rule.
    • One to UpdateTime(), which is called when creating a new block (either for mining on the node for regtest, or for getblocktemplate).
  • At the time of writing, this PR is ‘Chasing Concept ACK’. That means that we want contributors to review the concept before reviewing the implementation.

Questions

  • What could a miner do by abusing the timestamp field in a difficulty adjustment block prior to the Great Consensus Cleanup change?
  • After the Great Consensus Cleanup change, what can a miner do by abusing the block timestamp?
  • Why not just enforce that difficulty adjustment blocks to have a timestamp after the previous block, rather than after prev-600?
  • Which function would need to be changed to enforce this timestamp restriction as a consensus rule?
  • The curtime return value of getblocktemplate is now constrained by the prev-600 rule. Why?

Meeting Log

  113:00 <jnewbery> hi
  213:00 <kanzure> hi
  313:00 <pinheadmz> hi
  413:00 <sosthene> hi
  513:00 <ccdle12> hi
  613:00 <jonatack> hi !
  713:00 <rockstardev> hi... @jnewbery I'll follow up with details after PR review
  813:01 <amiti> hi
  913:01 <ariard> hi
 1013:01 <jnewbery> Before we start, can you raise your hand o/ if you managed to clone/build the PR and read the notes/questions?
 1113:02 <nehan> hi
 1213:02 <pinheadmz> read everything, did not test locally
 1313:02 <ccdle12> same as above
 1413:03 <jnewbery> ok, let's give everyone a couple of minutes to catch up. Notes are at https://bitcoin-core-review-club.github.io/15481.html
 1513:03 <ariard> read notes/BIP and skim over commits
 1613:03 <jnewbery> I'd encourage everyone to spend at least a few minutes preparing before the meeting by reading the notes. It'll really increase the benefit you get from the meeting if you come with questions prepared.
 1713:03 <ariard> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-August/016316.html
 1813:03 <ariard> Getting around to fixing the timewarp attack
 1913:04 <jonatack> * yup built, tested, read
 2013:04 <PaulTroon> I looked over Jon's shoulder :-)
 2113:04 <jnewbery> ok, so let's hit some of those questions
 2213:05 <jnewbery> pinheadmz: What could a miner do by abusing the timestamp field in a difficulty adjustment block prior to the Great Consensus Cleanup change?
 2313:05 <pinheadmz> force a downward difficultly adjustment
 2413:06 <ariard> and so increase the block production rate?
 2513:06 <jnewbery> exactly right
 2613:06 <pinheadmz> making mining easier
 2713:06 <jnewbery> either push difficulty down or keep it low
 2813:06 <jnewbery> the idea is that the retarget algorithm only looks at 2015 out of 2016 blocks, so the retargeting periods don't overlap
 2913:07 <jnewbery> the timewarp exploits this by pushing the difficulty adjustment block way into the future, and then the next block back into the present
 3013:07 <jnewbery> that makes it appear to the retarget algorithm that it took a long time for the prior retarget period, so difficulty stays low or is adjusted down
 3113:08 <sosthene> Is it the same vulnerability that was exploited against Verge was attacked last year?
 3213:08 <pinheadmz> can we clarify which blocks by # ? Like, block 2015 (the ACTUAL last block for retarget algo) is the one thats post-dated 2 hours into the future?
 3313:09 <jnewbery> sosthene: I think similar. As far as I'm aware verge had some system that used multiple POW algorithms
 3413:09 <lightlike> jnewbery: "way in the future": is it possible, i read somewhere that you cannot go more than 2 hours into the future with the current rules?
 3513:09 <pinheadmz> lightlike: correct, 2 hours
 3613:09 <jnewbery> lightlike: correct, so you couldn't use this attack at the tip, but you could redo a blockchain from genesis that does this because you have 10 years to play with
 3713:09 <rockstardev> @lightlike: https://bitcoin.stackexchange.com/a/20490/13744
 3813:10 <jnewbery> (in fact there are stome checkpoints, so you could only do it from block 295,000 if you wanted your blockchain to be accepted by Bitcoin Core)
 3913:10 <pinheadmz> and that 2 hours is measured against netwoek-adjusted wall-clock time? not MTP
 4013:10 <nehan> jnewbery: in matt's bip, it's called "timewarp inflation vulnerability". How does inflation come into it?
 4113:10 <jnewbery> pinheadmz: yes, your local time
 4213:10 <ariard> you could redo a blockchain but with less work, so you can't override current chain right?
 4313:11 <kanzure> inflation because subsidy goes out quicker than scheduled
 4413:11 <jnewbery> nehan: because if you can increase the rate of block production, you can increase the rate of new coin emission
 4513:11 <nehan> ok, so it refers to schedule and not # of coins
 4613:11 <jnewbery> nehan: exactly
 4713:11 <jnewbery> it doesn't affect the 21m cap
 4813:12 <pinheadmz> since retarget algo is based on median, how much effect can post-dating block #2015 by 2 hours really have?
 4913:12 <jnewbery> ariard: yes, you'd still need to do more work than the current blockchain if you wanted others to reorg to your chain
 5013:12 <nehan> i'd appreciate more explanation about the bug, like what pinheadmz asked for (which blocks)
 5113:12 <pinheadmz> or does the attack have to involve more than just the last block
 5213:13 <jnewbery> so if I wanted to create a blockchain with many blocks, I'd do the following:
 5313:13 <jnewbery> create block 1 with a timestamp 1 second after the genesis block
 5413:13 <jnewbery> create block 2 with timestamp one second after that
 5513:13 <jnewbery> ...
 5613:14 <jnewbery> create block 2015 with timestamp genesis+2015
 5713:14 <jnewbery> create block 2016 with timestamp 2 weeks in the future
 5813:14 <jnewbery> sorry, timestamp genesis+2 weeks
 5913:14 <jnewbery> create block 2017 with timestmp genesis+2017
 6013:14 <jnewbery> ...
 6113:15 <nehan> jnewbery: is that within the 2 hour time bound?
 6213:15 <jnewbery> nehan: correction, not 2 weeks in the future from clock time, 2 weeks in the future from genesis
 6313:15 <nehan> it seems like two weeks + genesis - 2016s + geneses > 2 hours
 6413:15 <jnewbery> so the retarget algorithm compares the timestamp of block 2016 with block 1 and thinks "hmmm, it took 2 weeks to mine these blocks, keep difficulty at 1"
 6513:16 <jnewbery> but because 2017 is back to genesis+2017 seconds, the time hasn't advanced two weeks
 6613:16 <ariard> the hard limit of 2 hours is based on which clock ? Median-Time-Past of X blocks ?
 6713:16 <jnewbery> nehan: the 2 hours check is against clock time on the node
 6813:16 <nehan> hasn't it for block 2016 though? why would nodes accept 2016's with such a far off timestamp?
 6913:16 <jnewbery> the two hour rule is really weird
 7013:16 <nehan> or is it averaged over blocks?
 7113:17 <jnewbery> it's the only 'consensus' rule that isn't based on blockchain data but on local data
 7213:17 <ariard> okay if you're a malicious miner just tune your clock time like you want?
 7313:17 <jnewbery> it's almost better to not think of it as a consensus rule
 7413:17 <jnewbery> ariard: sure, but other people wouldn't accept your block
 7513:18 <jnewbery> because they're all comparing to their (presumably correct) clocks
 7613:18 <ccdle12> so what is the significance of updating previous block nTime to -600 seconds? (why -600 as an arbitrary number)
 7713:19 <ariard> the 600 seconds is to avoid messing with miner software using timestamp field for pow
 7813:19 <jnewbery> so in summary: it allows someone to create arbitrarily long blockchains without the difficulty increasing. The Median Time Passed of the blockchain creeps forward slowly, but one block out of every 2016 is sent forward in time to artificially keep the difficulty down
 7913:19 <pinheadmz> jnewbery: in your example, why not just create blocks 1-2016 at ten minute intervals? the diff will still stay unchanged?
 8013:20 <fl> ariard: using the node's local clock, it increases the attack surface from inaccurate time attestations in blocks to requiring the persistent manipulation of timeserver/nodes' local time
 8113:20 <jnewbery> ariard: exactly right. Some miners apparently use the timestamp field as extra nonce and roll it forward up to 10 minutes
 8213:20 <rockstardev> that's a good one... interesting to learn that miners do that with timestamp
 8313:20 <jnewbery> pinheadmz: because at blocks every 10 minutes the timestamp will advance to current time after ~580000 blocks
 8413:20 <fl> there's a bit of a dearth of nonce space IIRC
 8513:21 <jnewbery> the attack is to make a longer blockchain, but without increasing the difficulty
 8613:21 * pinheadmz thumbs up
 8713:21 <jnewbery> so more coinbase subsidy is realeased
 8813:21 <lightlike> jnewbery: why would other nodes accept the longer blockchain? I thought the one with the most cumulated work is accepted, not with the largest block count.
 8913:22 <fl> jnewbery wouldn't a timewarped fork chain require an eclipse attack to prevent nodes from learning about a non attacking chain? or is the desire just to make controlled supply more metered?
 9013:22 <jnewbery> lightlike: correct, so the full attack would be: I control 51% of the mining power, so I can create a chain with more work than anyone else. I do that, but also use timeward to make that most-work chain really long
 9113:23 <pinheadmz> does this attack have to start from genesis?
 9213:23 <fl> pinheadmz no
 9313:23 <lightlike> jnewbery: ok, thanks, I think I got it now :-)
 9413:23 <jnewbery> it means that a 51% attack can now increase the rate at which new coins are minted
 9513:23 <fl> timewarping can start at any block
 9613:23 <jnewbery> fl: yes, exactly correct
 9713:24 <fl> jnewbery to which comment?
 9813:24 <jnewbery> fl: correct that a timewarp can start at any block
 9913:24 <fl> ty
10013:24 <jnewbery> I used genesis as an example because I thought it might simplify the concept, but you could start anywhere
10113:24 <hugohn> q: why is the 600sec limit in miner.cpp? don't miners run their own sw?
10213:25 <jnewbery> obviously if you start a more recent block, it'll be more work
10313:25 <jnewbery> I created a 500,000 long mainnet blockchain by starting at genesis with an S7 a few years ago. At the time I believe it was the longest mainnet Bitcoin blockchain (but not most work, obviously)
10413:26 <pinheadmz> jnewbery: thats awesome! ha
10513:26 <nehan> jnewbery: i still don't understand why, in your example, other nodes would accept block 2016 (its timestamp is far in the future) but i am happy to take it offline
10613:26 <ariard> fl: yes on increasing the attack surface to require the persistent manipulation of timeserver but I wonder if eclipse attacks can destroy this assumption and instead relying on public data wouldn't be better...
10713:27 <fl> What would be the effect of sending a timewarped-spoofed chain to a node, would chainstate get erased?
10813:27 <jnewbery> block 2016 is in the future from the prior block, but it's in the past when your node is validating
10913:27 <wallet42> jnewbery: awesome! wen polo?
11013:27 <nehan> jnewbery: aha, because you're starting at genesis, which was a long time ago. got it, thanks!
11113:27 <jnewbery> yes, my use of the word future was probably confusing
11213:28 <jnewbery> hugohn: I believe most miners use the Bitcoin Core software to get block templates, which they then do work on
11313:28 <jnewbery> the `getblocktemplate` RPC will give a block template with the timestamp filled in
11413:29 <jnewbery> Here's the code: https://github.com/bitcoin/bitcoin/blob/0221420d1a0550cd849e0f3a5ada3738d5931bdd/src/rpc/mining.cpp#L292
11513:29 <jnewbery> You can see that it also returns `curtime` and `mintime`
11613:29 <jnewbery> this PR means the value returned for those fields may change. Why?
11713:30 <ariard> fl: headers-sync first would happen first and your spoffed chain not queried as new chainstate ?
11813:31 <jnewbery> ariard fl: assuming your peer is on the most work tip, and your spoofed chain is below block 295,000, it wouldn't accept the headers because of the checkpoint
11913:31 <fl> ah yeah
12013:32 <fl> forgot
12113:32 <jnewbery> you could create a timewarped chain after block 295,000 but even back at that height, it's quite a lot of work
12213:33 <ariard> jnewbery: yes and after 295,000 headers are the counter-measures against resource consumption to track spoofed-chain?
12313:33 <jnewbery> anyone want to take a shot at why `curtime` and `mintime` might be changed?
12413:33 <fl> i.e. unless timewarped chain had more chainwork after 295e3 nodes wouldn't switch tips, correct?
12513:33 <jnewbery> fl: right, we wouldn't reorg unless the new chain contained more work
12613:34 <jonatack> to cap the mintime at not less than nTime - 600 ?
12713:34 <hugohn> so this is not a consensus change right? as the 600-limit is only in miner.cpp
12813:34 <fl> jnewbery but a timewarp spoof chain target node would have some some resource consumption based on header-sync assesment I think
12913:35 <jnewbery> ok, so mintime might change because of the code change here: https://github.com/bitcoin/bitcoin/pull/15481/files#diff-9651347c8e00bed3ddc7631de569406dR656
13013:35 <pinheadmz> Is there any way timewarp can benefit one single attacker? Seems like it lowers difficulty for EVERYONE, doesn't give one party a specific advantage
13113:35 <jnewbery> if the last block of the retarget period was more than 10 minute in the future, then `mintime` would be that timestamp-600
13213:36 <fl> pinheadmz it would benefit miners with low block propogation latency
13313:36 <jnewbery> hugohn: correct, not a consensus change. This is just miner behaviour change
13413:36 <jnewbery> pinheadmz: potentially you could combine it with selfish mining if you're not sharing your chain
13513:37 <jnewbery> but really, the dangerous attack is from a miner (or group of miners) that control 51%
13613:37 <jnewbery> why might `curtime` change?
13713:37 <ariard> fl: starting from checkpoints that's only 2,3 GB of headers with current height of 581487
13813:38 <jonatack> to close the vuln without potentially breaking miner machines
13913:38 <wallet42> 23 MB
14013:38 <jnewbery> fl: I believe we won't download the full block if it's not the most work chain - just the header
14113:38 <wallet42> (581487−295000)×80
14213:38 <nehan> jnewbery: block->nTime is set on line 41 in UpdateTime(), and it might have been changed at line 37
14313:39 <ariard> wallet42: oh sorry right
14413:39 <jnewbery> so if you're trying to use this to DOS nodes, you need to do a ton of work just to waste 80 bytes
14513:39 <fl> jnewbery I agree just header data would get computed on, thought I would mention that there is some marginal increase in resource consumption
14613:39 <jnewbery> if you want to use timewarp to ratchet down the difficulty, you still need to get to the next difficulty retarget block after 295000, and then you can only lower it by 75%
14713:40 <jnewbery> fl: yes, but it's asymetric. The attacker needs to do a *lot* of work for a minimal amount of bandwidth/CPU/memory wastage on the civtim
14813:40 <jnewbery> *victim
14913:40 <fl> issuance rate and preserving DMMS are the more important network properties anti-timewarping attack measures benefit
15013:41 <fl> IMO
15113:41 <wallet42> hm but once you lowered it enough, you can generate 80 bytes very easily no?
15213:41 <jnewbery> nehan: yes exactly right. The update time change here: https://github.com/bitcoin/bitcoin/pull/15481/files#diff-4a59b408ad3778278c3aeffa7da33c3cR36 is used when constructing the block template
15313:41 <jnewbery> and `curtime` is taken from that block: https://github.com/bitcoin/bitcoin/blob/0221420d1a0550cd849e0f3a5ada3738d5931bdd/src/rpc/mining.cpp#L667
15413:42 <jnewbery> ok, last question: Which function would need to be changed to enforce this timestamp restriction as a consensus rule?
15513:42 <fl> jnewbery the resource consumption on a spoof timewarp chain is negligible but in your case where 51% miner stretches out the chain, they could start stuffing more and more data into blocks increasing node operation costs more
15613:42 <nehan> jnewbery: but *why* was that change made?
15713:43 <jnewbery> nehan: because the aim of this PR is to make miners not mine a difficulty retarget block more than 600 seconds before the prior block. When the miner callers GBT we want the block template returned to them to have a timestamp that is constrained by that rule
15813:43 <fl> well not more data into blocks but more data per time
15913:44 <jnewbery> and `curtime` is just taken from the block
16013:44 <jnewbery> the help text would be technically wrong after this PR: https://github.com/bitcoin/bitcoin/blob/0221420d1a0550cd849e0f3a5ada3738d5931bdd/src/rpc/mining.cpp#L361
16113:45 <jnewbery> by the way, GBT is defined in a BIP here: https://github.com/bitcoin/bips/blob/master/bip-0022.mediawiki
16213:45 <jnewbery> Anyone brave enough to try to answer "Which function would need to be changed to enforce this timestamp restriction as a consensus rule?" :)
16313:46 <jnewbery> hint: it's a consensus rule, so it's probably in validation.cpp
16413:46 <lightlike> jnewbery: Change CheckBlock() in validation to not accepts new block that break it?
16513:47 <jnewbery> lightlike: almost!
16613:47 <jnewbery> the change would actually be in ContextualCheckBlockHeader()
16713:47 <jnewbery> Header, because this is a field in the block header
16813:48 <jnewbery> Contextual, because it's a context-dependant check (ie it depends on where the block is in the blockchain)
16913:48 <jnewbery> Matt has a reference implementation for the Consensus Cleanup softfork: https://github.com/bitcoin/bitcoin/pull/15482
17013:49 <jnewbery> the relevant change is here: https://github.com/bitcoin/bitcoin/pull/15482/files#diff-24efdb00bfbe56b140fb006b562cc70bR3251
17113:49 <hugohn> q: what happens if there is a large hashrate drop between the second-to-last block and the last block in a period (so the gap between those 2 blocks is truly more than 2 minutes)? Won't this change make it so that we won't account for this hash rate drop in retargeting? then that means the next period's difficulty will be a lot longer than it has to be?
17213:49 <hugohn> *more than 10 minutes
17313:49 <jnewbery> so the code changes this week were very minor, but I hope you agree that there's a lot of interesting discussion behind them
17413:50 <hugohn> *harder than it has to be
17513:50 <jnewbery> hugohn: the change imposes a new minimum on the block timestamp, not a new maximum
17613:51 <jnewbery> if the hashrate is dropping, then we'd expect the timestamps to become more spread out
17713:51 <jnewbery> last 10 minutes. Any questions?
17813:51 <pinheadmz> <timewarp attack> actually we have 2 hours still :-)
17913:52 <fl> I dunno my local clock says...
18013:53 <jnewbery> ok, thanks everyone!
18113:53 <wallet42> thanks @jnewbery!
18213:53 <lightlike> thanks!
18313:53 <jonatack> V interesting discussion. No Concept ACK from you on the PR?
18413:53 <pinheadmz> thanks
18513:53 <fl> jnewbery thanks great pr review
18613:53 <ariard> thanks jnewbery, was really interesting!
18713:53 <rockstardev> pinheadmz: good one
18813:54 <jnewbery> I'm out of office and won't make the next couple of review clubs. I'm going to try to find someone else to host, so keep an eye out on the website for details.
18913:56 <jnewbery> jonatack: I believe Matt has some updates to make to his consensus cleanup proposal. I'm going to hold off on concept reviewing until after that
19013:57 <jonatack> jnewbery: thanks