Keep temporary data in memory unless SQLITE_TEMP_STORE is set

[mtrossbach]

Hi Ulrich,

a small follow-up to #148.

Type: Enhancement (security hardening)

The problem

Temporary files (temporary tables, sorter files, transient databases) are not encrypted, as discussed in #148. The documentation therefore strongly recommends SQLITE_TEMP_STORE=2 or 3.

The build files of SQLite3 Multiple Ciphers set SQLITE_TEMP_STORE=2, but the source itself doesn't define it. So the setting doesn't apply when the amalgamation (or src/sqlite3mc.c) is compiled with other build scripts, as many packages do. Without the flag, SQLite's default 1 applies, and temporary data is written to disk in plaintext. Of ten projects I looked at that compile SQLite3 Multiple Ciphers themselves, four don't set it; one of them sets PRAGMA temp_store=MEMORY at runtime instead.

The change

If SQLITE_TEMP_STORE is not defined, sqlite3mc.c now defines it as 2 before including SQLite. The amalgamation then behaves the same as the builds from the build files, whatever it is built with. An explicit value, including 1, is kept, and PRAGMA temp_store works as before.

What do you think?

Would this be a good idea for you? SQLCipher goes further: since March 2025 its build stops unless SQLITE_TEMP_STORE is 2 or 3. I chose the softer default so that builds that set 1 on purpose keep working. If you'd rather have an error like that, or prefer to leave this to the build files as before, that's fine with me too.

How I tested it

I built the shell from src/sqlite3mc.c without setting SQLITE_TEMP_STORE, filled a temporary table with a marker and searched the open temporary files of the process for it (Linux, /proc/<pid>/fd):

Build	TEMP_STORE	Marker in plaintext in a temporary file
main (8392d70), no flag	1	yes
with this change, no flag	2	no
with this change, -DSQLITE_TEMP_STORE=1	1	yes (the explicit value is kept)
with this change, -DSQLITE_TEMP_STORE=3	3	no
amalgamation from scripts/amalgamate.py with this change, no flag	2	no

The build files already set SQLITE_TEMP_STORE=2, so builds made with them don't change.

Best regards,
Markus

[utelle]

Hi Markus,

The problem

Temporary files (temporary tables, sorter files, transient databases) are not encrypted, as discussed in #148. The documentation therefore strongly recommends SQLITE_TEMP_STORE=2 or 3.

Correct. Unfortunately, I haven't found a (reasonable) way to force encryption for temporary SQLite files in case of an encrypted main database.

The build files of SQLite3 Multiple Ciphers set SQLITE_TEMP_STORE=2, but the source itself doesn't define it. So the setting doesn't apply when the amalgamation (or src/sqlite3mc.c) is compiled with other build scripts, as many packages do. Without the flag, SQLite's default 1 applies, and temporary data is written to disk in plaintext. Of ten projects I looked at that compile SQLite3 Multiple Ciphers themselves, four don't set it; one of them sets PRAGMA temp_store=MEMORY at runtime instead.

I fully agree that this is anything but ideal. And unfortunately, few people read the documentation. So, it seems one has to force people to be happy sometimes.

The change

If SQLITE_TEMP_STORE is not defined, sqlite3mc.c now defines it as 2 before including SQLite. The amalgamation then behaves the same as the builds from the build files, whatever it is built with. An explicit value, including 1, is kept, and PRAGMA temp_store works as before.

Good idea. And it should have been applied much earlier. But better late than never.

Would this be a good idea for you?

Yes, absolutely.

SQLCipher goes further: since March 2025 its build stops unless SQLITE_TEMP_STORE is 2 or 3. I chose the softer default so that builds that set 1 on purpose keep working.

If the option was set intentionally, it should be respected.

If you'd rather have an error like that, or prefer to leave this to the build files as before, that's fine with me too.

No, I think it is a good thing to set SQLITE_TEMP_STORE to a reasonable default, if it wasn't set explicitly.

However, I would prefer to have these preprocessor statements further up in the source file sqlite3mc.c. For example, directly at the beginning of the file (after line 9). There might be other options now or in the future that should deviate from the usual SQLite defaults. One should not have to search their settings in the depths of the source file.

[mtrossbach]

Hi Ulrich,

thanks! I moved the block to the beginning of sqlite3mc.c, right after the file header, and added the meaning of the values to the comment. The tests from the description give the same results.

Best regards,
Markus

[utelle]

Hi Markus,

I moved the block to the beginning of sqlite3mc.c, right after the file header, and added the meaning of the values to the comment.

Perfect! Thanks!