Use fast key erasure in the ChaCha20 random number generator

[mtrossbach]

Hi Ulrich,

this is the last of the random number follow-ups I mentioned in #262. It changes only chacha20_rng() in src/chacha20poly1305.c and keeps your structure: the fork detection, the locking and the failure handling stay as they are.

Type: Enhancement (security hardening)

Background

Same background as #267. For random number generators, the BSI technical guideline BSI TR-02102-1 (chapter 8) recommends deterministic generators of class DRG.3 or DRG.4 in the sense of AIS 20. DRG.3 requires that earlier output can't be computed from the generator's current state.

The problem

chacha20_rng() draws a key and a nonce from entropy() once and derives all output from them. It reseeds only on the first call, after a fork, or when the 32-bit block counter wraps, which takes 256 GiB of output. Anyone who learns the generator state at some point, for example from a memory dump, can therefore compute all output since the last reseed: the salts, IVs and nonces that were already used.

The change (only chacha20_rng())

• Fast key erasure: every refill produces 256 bytes of keystream. The first 32 bytes immediately replace the key, and only the remaining 224 bytes are handed out. Output that has already been returned can no longer be computed from the current state.
• Regular reseeding: the generator reseeds from entropy() every 16384 refills (3.5 MiB of output), in addition to the first call and the fork detection.
• Wiping: handed-out bytes are cleared from the buffer.

This is the construction D. J. Bernstein describes in "Fast-key-erasure random-number generators" (2017). The random number generator of the Linux kernel (crng_fast_key_erasure() in drivers/char/random.c) and OpenBSD's arc4random work the same way.

Cost (Apple M4):

Request	Before	After
16 bytes (page nonce of the chacha20 scheme)	19 ns	25 ns
128 bytes (reserved area of the sqlcipher scheme)	140 ns	170 ns

Both are small compared to encrypting the page itself.

How serious is it? Low in practice: salts, IVs and nonces end up in the database file anyway, and whoever can read the process memory usually has the database key as well. But backtracking resistance is one of the properties an audit of the random number generation checks, and the change is small.

How I tested it

• Known-answer test: a test program injects a fixed seed and compares the first 480 bytes of output (three refills, mixed request sizes) with an independent Python implementation of the construction, which is itself checked against the RFC 8439 test vector. The new code matches; the old code doesn't, as expected.
• Reseeding: getrandom() is called twice for the first 3.5 MiB of output and twice more after one more byte.
• Both checks pass on Linux aarch64 (GCC and Clang) and x86_64 (glibc), Alpine Linux (musl) and Android (NDK r28).
• Fork and distinct draws: on macOS, parent and child produce different output after fork(), and two draws differ.
• The CI recipe (autoreconf, ./configure, make, the SQL tests and cryptotest) passes on Linux aarch64 and x86_64 with GCC and Clang. cryptotest also passes on macOS and WebAssembly.
• No new compiler warnings. A MinGW-w64 cross-compile of the amalgamation gives the same result as before.

Best regards,
Markus

[utelle]

Hi Markus,

thanks for this PR, and for the detailed explanation of the changes, and the references to the literature..