The purpose of the video meeting is that Larry will share his display and show how
one might use the gdb debugger as an aid to reviewing this or any PR. The session
will be recorded, and the link to the recording sent out with the IRC log after
Bitcoin Core developer Fabian Jahr has provided an excellent
video on using the system debugger and much
more. Larry will more narrowly focus on using gdb effectively.
Please review to the Notes for the PR 21603 review
for background information about logging.
Many operating systems solve the logging growth problem by automatically rotating their
system logs; run ls /var/log on any Linux system and you’ll likely see several different
kinds of log files with numbers as part of their filenames;
these are “rotated” log files – older versions of the main log file. Some of these
are also compressed.
Linux and other Unix-like systems include a tool specialized to this exact task,
logrotate. It’s usually configured to run periodically (daily, for example) using
the built-in cron facility.
The logrotate command is difficult to configure (run man logrotate),
knowledge of cron is also required, and is not available on Windows or on some
stripped-down Linux distributions.
For that reason, some programs do their own internal log rotation.
The concept is, when a log file reaches a certain size, the system that “owns” the log
file renames it to an alternate name (usually by including a sequence number), and resets
the main log file to empty.
Usually this occurs as a side-effect of writing a log message.
Different things can be done with the rotated log files. For example:
Retain only a configurable number of the most recent log files, deleting older files.
Compress and save the rotated log files for a configurable time period.
Upload the log files to a remote system that can archive them (store them cheaply).
Many commercial data storage systems do internal
log file rotation (examples here
While Bitcoin Core’s debug.log file grows without limit for as long as bitcoind
runs, it does do one kind of trimming: When bitcoind starts up, it checks the
length of debug.log; if it’s greater than 11 MB, it
down to the most recent 10 MB of logging data.
Why does bitcoind include the concept of a log file? It uses disk space; what’s
the advantage? Have you needed to look at debug.log; has it been useful?
What’s the connection with denial-of-service attacks?
If we receive an invalid transaction, why don’t we log that this happened?
PR 21603 uses a different approach,
log rate limiting. (Quick summary: Parts of the code that are generating too
much logging too quickly are throttled; not all their log messages are saved,
although this limiting itself is logged so readers of the log are aware that
it’s happened.) What are the advantages and disadvantages of these two
approaches? Which do you think would result in more useful log output?
Other log rotation systems append a sequence number to rotated logs (for example,
on Linux systems, in /var/log, you see syslog.1, syslog.2, … This PR
names the rotated log files by date, such as debug-2021-06-27T21:19:42Z.log.
What are the trade-offs? Which naming style do you prefer?
Should this PR also compress rotated log files?
Is this PR even necessary, given that the logrotate facility exists? What
are the trade-offs?
On Linux systems, logrotate is located in /usr/sbin. What kinds of programs
are stored there, as opposed to the more standard /usr/bin?
<LarryRuane> Okay, so it's used to print messages that indicate what it's doing, what problems the node may be having ... stuff that's useful for debugging (hence the name), but also just for monitoring
<LarryRuane> yes! And notice that it might not be an intentional DDOS, there could be a bug that causes repeated maybe infinite logging, which would quickly fill up disk space, causing the node to crash
<jnewbery> glozow: very much depends how much logging you have enabled. The default (everything that's logPrintf() in the code) is not very verbose. If you start with --debug=all you'll get some very verbose categories like net, libevent and leveldb.
<LarryRuane> Without any extra logging enabled, I've seen tons of logging during syncing (especially IBD of course), but during normal operation, only some P2P messages and the UpdateTip messages, so not really very much
<LarryRuane> Yes I think that's one of the downsides to using `logrotate` plus the fact that it's complicated to configure (I did that recently for our `debug.log` file and it took me a couple of hours to figure it out, but maybe that's just me :) )