This repository was archived by the owner on Sep 11, 2026. It is now read-only.
Turso support - #11
Merged
Merged
Conversation
Runs the driver against a Turso database in either of two shapes. All-primary,
for the control plane that must read its own writes: every statement goes over
"SQL over HTTP", which at ~156 us per statement puts a 200-statement admin page
near 31 ms when the primary is co-located. Or snapshot reads with primary
writes, for the public front end: reads come from a local SQLite file, writes go
to the primary, and the first write (or first transactional statement) latches
the rest of the request to the primary so it reads its own writes. Measured at
22 ms per page and ~41 req/s per vCPU, indistinguishable from reading the
database file directly.
The snapshot exists because a live Turso embedded replica cannot be read by
pdo_sqlite at all. Turso holds a POSIX write lock on the whole replica file for
its connection's lifetime -- not just while syncing -- so SQLite reports
"database is locked (5)", and neither a busy timeout nor mode=ro helps. There is
no protocol to negotiate: turso_core coordinates its WAL across processes
through a .tshm file while SQLite uses -shm. Disabling turso_core's lock makes
it worse, serving silently stale data forever with integrity_check ok. So a
separate publisher keeps a private replica and hands over consistent standalone
copies; the front end is stale by at most one publish interval.
Everything the backend assumes about Turso was established by probing
tursodb --sync-server, not inferred:
- Write metadata is missing. Every statement reports affected_row_count 0 and
last_insert_rowid null, so the transport asks for changes() and
last_insert_rowid() in the *same* pipeline request. No extra round trip, and
consistent because the server holds its connection for the whole request.
- Named parameters bind silently to NULL. Positional "?" only.
- Batches are not atomic, unlike D1's: steps that ran before a failure keep
their effects. The transport spells the transaction out as BEGIN /
conditional COMMIT / conditional ROLLBACK steps.
- Interactive transactions are unusable. A BEGIN in one HTTP request stays
open on the connection the server shares between clients, and every later
BEGIN anywhere fails. Reported unsupported; atomicity comes from
execute_batch().
- PRAGMA works, foreign_keys included, and persists across requests -- passed
straight through rather than emulated the way the D1 backend must.
- Bound parameters are not capped the way D1's are; 1000 in one statement is
fine. Inlining stays only as a safety valve.
The replica connection reports the primary's capabilities rather than the
snapshot's. The snapshot could offer transactions, savepoints, temporary tables
and user-defined functions, but a statement may be routed to either side, so the
driver has to emit SQL that works on both.
Tests: WP_SQLITE_TEST_BACKEND=turso runs the suite over a fake transport that
enforces each of those behaviours and round-trips values through the real
protocol codec, so the codec is exercised rather than approximated. The Turso
backend's failures are a strict subset of the D1 backend's -- 60 shared, one
only on D1, none only on Turso -- so it is at least as conformant as the
established remote backend. Those 60 are pre-existing gaps in the remote path,
not Turso-specific: tests needing a real PDO SQLite handle,
WP_PDO_Array_Statement methods that raise "Not implemented", and transaction,
savepoint and temporary-table tests missing from the skip list, plus the
UDF-less rewrite path calling wp_die() outside WordPress.
src/turso/verify-against-server.php covers what a fake cannot: the wire
encoding, the write-metadata ride-along, batch atomicity, and read-your-writes
across two databases, against a real server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
WP_SQLITE_TEST_BACKEND=d1 was failing 32 errors and 34 failures before any of
this, and turso inherited the same set. They are now both green -- 1084 tests, 0
failures, 102 skipped -- alongside the default pdo backend, which stays green
throughout. The gaps were shared, so every fix below helps both backends.
Six real bugs, each confirmed against real PDO SQLite rather than assumed:
- beginTransaction() was ignored on a connection without transaction support
while commit() still demanded an active transaction, so "BEGIN; ...; COMMIT"
threw "There is no active transaction" -- which the Query_Tests setUp does,
taking 21 tests with it. The PDO API now tracks its own transaction so the
contract holds (a second BEGIN errors, a COMMIT after a BEGIN does not)
without inTransaction() claiming isolation the connection cannot give; the
driver's internal logic keeps seeing the real connection state.
- WP_PDO_Array_Statement raised "Not implemented" for errorCode(),
errorInfo(), getIterator() and bindColumn(). All four are implemented,
including PDO::FETCH_BOUND.
- closeCursor() rewound the cursor where PDO discards the result set. Verified:
PDO returns false from fetch() afterwards and an empty fetchAll(), and only
execute() makes the rows available again. test_execute_rewinds_cursor
asserted the old behaviour and has been corrected.
- PDO::FETCH_BOTH overwrote a key when a column is *named* with a number.
PDO's two namespaces behave differently on collision: a repeated name takes
the later value while keeping its first position, but a numeric index is
only filled when still free -- so a column named "2" keeps key 2 and the
third column gets no numeric key. Probed and reproduced exactly.
- fetchColumn() with a negative index raised "Invalid column index" where PDO
raises "Column index must be greater than or equal to 0".
- The D1 and Turso exceptions put their *decorated* message in errorInfo[2]
and no driver code in [1]. PDO puts the raw driver message and the SQLite
code there, and that is what the driver matches on to give SQLite errors
their MySQL identity -- so a missing table reported HY000 instead of
42S02 / 1146 "Table 'x' doesn't exist".
- Information schema reads that join sqlite_sequence to compute the next
AUTO_INCREMENT value were being cached. That value moves on an ordinary
INSERT, which invalidates nothing, so SHOW TABLE STATUS served a stale
number. (My first guess, that disabling the cache would fix it, appeared
wrong only because the probe patched one of the two test factories.)
- PRAGMA integrity_check indexed $errors[0] unconditionally, so a connection
that cannot run the pragma crashed CHECK TABLE. And D1's PRAGMA fallback
swallowed every error including "no such table", which made
"CHECK TABLE missing" report OK; it now degrades only when the pragma itself
is refused.
The rest are genuinely out of reach and now say so, with a reason each:
transactions and savepoints (a BEGIN cannot outlive its HTTP request),
temporary tables, the PHP-evaluated functions including LIKE BINARY with a
non-constant pattern, the two tests that reach past the connection to a PDO
SQLite handle that is not there, and PDO::FETCH_NAMED, which returns numeric
column names as *string* array keys -- something PDO builds below the language
and a pure-PHP array cannot represent. The skip list is shared by both remote
backends and renamed accordingly.
Also removes a wp_die() call from the Query_Tests setUp, which masked the real
error in a test environment where WordPress is not loaded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A small Rust binary that gives the Turso backend's read path the plain SQLite
file it needs. It replaces the C# prototype, so a container no longer has to
carry a .NET runtime to serve a WordPress front end.
Each cycle: pull, VACUUM INTO a temp file, convert it to rollback-journal, make
it read-only, rename it into place. Every step is there for a reason found by
testing rather than assumed:
- Pull is incremental as long as the replica directory persists.
- VACUUM INTO writes a consistent standalone database. Closing the connection
is not an alternative: with pooling on it does not checkpoint, and snapshots
taken that way fail integrity_check while the -wal grows without bound.
- Rollback-journal mode, by writing the two file-format version bytes in the
SQLite header. VACUUM INTO emits a WAL-mode file and a WAL reader creates
-wal/-shm beside it; SQLite keys those by path rather than inode, so they
would outlive the rename and describe the previous snapshot. Turso refuses
PRAGMA journal_mode = DELETE, and after a vacuum there is no WAL to lose, so
the two bytes are the whole conversion.
- Mode 444, because the reader never writes. The directory stays writable: the
plugin drops an index.php and .htaccess beside the database.
- rename is atomic, so a reader sees one whole snapshot or the other, and one
holding the old inode finishes undisturbed.
A failed cycle is logged and the loop continues, since the snapshot already in
place stays servable.
Verified against a real tursodb with the installed WordPress database: 10-15 ms
per cycle for 467 KB (vacuum ~11 ms, incremental pull 0.4-4 ms), the published
file reports journal_mode=delete and integrity_check ok, an UPDATE on the primary
reaches the snapshot on the next cycle, and the PHP Turso connection reads it
with no sidecar files created at any point.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR #11 failed on PHP 7.2, 7.3 and 7.4 only, in one test: test_iteration_yields_the_remaining_rows. iterator_to_array() over an in-memory statement returned nothing there. Before PHP 8.0, PDOStatement is Traversable through an internal handler that a subclass cannot override; the getIterator() hook only exists from 8.0, where PDOStatement implements IteratorAggregate. So the implementation is right and the limitation is real -- and it predates this work, since the old "Not implemented" getIterator() was never reached on PHP 7 either. Nothing in the driver or the plugin iterates a statement; they fetch. The test now skips on PHP 7 with that reason, and getIterator() documents it. The Turso backend had no CI job, so nothing was exercising it the way the D1 backend is. Add one mirroring d1-backend-tests.yml: the four driver suites against WP_SQLITE_TEST_BACKEND=turso on PHP 7.2, 8.1 and 8.4, plus fmt, clippy and a build of the snapshot publisher. The publisher's real-server check stays local until a tursodb is packaged for CI. rustfmt had two reflows waiting in the publisher; applied, so the new gate passes on its first run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This is a hard fork, and the 7.x series has been end-of-life since November 2022. Carrying it in CI cost a matrix row per backend and, on PR #11, a failure in a test that exercises getIterator() -- which cannot work before 8.0, where PDOStatement first implements IteratorAggregate. Every CI matrix now starts at 8.0: the PHPUnit suites (7.2, 7.3 and 7.4 rows removed), the D1 and Turso backend suites, the parser and proxy tests, and the code-style job, which ran phpcs on 7.4. The declared minimum moves with it -- composer.json in the root and every package, the plugin's "Requires PHP" in both load.php and readme.txt, and the READMEs that stated 7.2 -- so the support statement matches what is tested. The PHP 7 skip added to the iteration test in the previous commit is removed again as dead code, and getIterator()'s note is shortened to the one fact that still matters: it relies on IteratorAggregate, present from 8.0. The PHP_VERSION_ID < 80000 shims in the source -- the conditional trait declarations for PDOStatement signatures and the like -- are left in place. They are inherited, harmless, and removing them is a separate refactor; AGENTS.md now says they may go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Everything so far was established against "tursodb --sync-server". The first session against a real Turso Cloud database found three places where Cloud behaves differently, one of them a correctness gap. Sessions do not reliably persist. Cloud gives each HTTP request its own session -- except that a kept-alive connection pinned to one node sometimes keeps state between requests. So "PRAGMA foreign_keys = ON", which the driver sets once at connect and assumes holds, was gone by the next request, and foreign keys were never enforced. Worse, whether they were was nondeterministic. The connection now tracks the pragma and has the transport send it ahead of every query and batch, in the same round trip (a pragma cannot change inside a transaction, so in a batch it precedes the BEGIN). It is always stated explicitly, OFF included, because omitting it can leave a stale ON in place on a sticky session. Verified against Cloud: an orphan insert is rejected in a separate request and inside a batch, and allowed again after the driver turns the pragma off. Errors carry an extra prefix. Cloud reports "Tursodb error: Parse error: no such table: x" where the CLI reports "Parse error: ...". The exception now strips both layers, so the driver's SQLite-to-MySQL identity mapping still fires: verified that a missing table reports 42S02 / 1146 against Cloud. Batches are not atomic on Cloud either. Not a change, but worth recording: the BEGIN / conditional COMMIT / conditional ROLLBACK wrapping was built for the CLI, and Cloud turns out to need it just as much -- its native batch keeps the effects of steps that ran before a failure. Verified atomic with the wrapper. Two things the CLI got wrong that Cloud gets right, also recorded: Cloud reports affected_row_count and last_insert_rowid, and binds named arguments. The changes() ride-along and positional-only binding stay, since they are correct on both. Also: the publisher's --help exited 2, which failed the smoke test in the new CI job. Help is a success path and now exits 0; a bad or missing argument still exits 2. Measured from here to aws-us-east-1: 32 ms per warm round trip, 176 ms for a write. A 26-statement page issued one statement at a time is ~840 ms over the wire, which is exactly why the front end reads a local snapshot and the control plane co-locates its primary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Against Turso Cloud, VACUUM INTO leaves a third empty sidecar beside its output that the CLI never produced: Turso's own log, named by *replacing* the path's last extension, so "snapshot.db.new" gets "snapshot.db.db-log". Remove it along with -wal and -shm, so nothing but the snapshot is ever published. The README now distinguishes CLI from Cloud behaviour where they differ, and records the end-to-end result against a real Cloud database in aws-us-east-1: WordPress installed itself through the driver (94 s, a few hundred writes at ~176 ms each, zero PHP errors); the front page reads 1,302 ms over the wire and 22.6 ms from the snapshot; a comment posted through the front end latched to Cloud and WordPress read it straight back; the publisher pulled it into the snapshot on the next cycle; and wp-admin login, the dashboard and a REST post all work in the control-plane shape -- at 6.4 s for the dashboard, which is why that plane co-locates its primary. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings in upstream's Release 3.0.2 and the schema-reconstruction fix (WordPress#506). The only conflicts were the plugin header, where the release bumped the version and this branch raised the PHP floor: both survive -- Version 3.0.2, Requires PHP 8.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.