Skip to content

Repository files navigation

AQL — AST Query Language

CI License: MIT Node Container

AQL is a high-level, strongly-typed query language for relational databases that replaces hand-written SQL strings with a structured, typed representation of queries.

You describe relational operations as compositions of typed language constructs. Those constructs form an immutable Abstract Syntax Tree (AST), which a compiler validates, type-checks, optimizes, and lowers into parameterized SQL for your target database. SQL becomes a compilation target — an implementation detail — rather than the interface your application code talks to. Because untrusted input can only ever enter a query as data, SQL injection is impossible in the safe subset of the language (security model).

import { table, col, from, postgres } from "@fmlgamer/aql";

const users = table("users", {
  id:   col.uuid().primaryKey(),
  name: col.text(),
  age:  col.integer(),
});

const query = from(users)
  .where((c) => c.users.age.gte(18))
  .select((c) => ({ id: c.users.id, name: c.users.name }));

query.compile(postgres());
// {
//   sql:    'SELECT "users"."id" AS "id", "users"."name" AS "name"
//            FROM "users" WHERE ("users"."age" >= $1)',
//   params: [18],
// }

The one core principle

Query structure and query data are fundamentally different kinds of information and must never be represented the same way.

  • Structure — tables, columns, operators, joins, functions, clauses — is represented as typed AST nodes known to the compiler.
  • Data — runtime values — lives only inside literal and parameter nodes and is always emitted as a bound parameter.

Because untrusted input can only ever enter a query as data, it can never become executable query syntax. SQL injection is therefore impossible in the safe subset of the language — not by sanitizing strings, but by construction.


Why AQL

Property How AQL delivers it
Security by construction Values are never concatenated into SQL; they become bound parameters. There is nothing to escape.
Type safety Tables, columns, expressions, and functions are statically typed. Invalid queries fail to compile.
Compile-time validation A validator rejects type errors, malformed aggregation, and dialect-unsupported features before any SQL runs.
Backend independence The same query compiles to PostgreSQL, MySQL, SQLite, or SQL Server.
Optimization Semantics-preserving passes simplify the AST before code generation.
Extensibility New functions, operators, and dialects plug into the existing pipeline.
A clear escape hatch When you must drop to raw SQL, raw is explicit, visible, and still parameterizes interpolated values.

Installation

npm install @fmlgamer/aql

The package is published to GitHub Packages as @fmlgamer/aql. Add a line to .npmrc so the @fmlgamer scope resolves there:

@fmlgamer:registry=https://npm.pkg.github.com

Requires Node.js 18+. TypeScript 5+ is recommended to get the full type-checking benefits, but AQL works from plain JavaScript too.

A ready-to-run container image with the aql CLI is published to ghcr.io/fmlgamer/aql (see Command line & Docker).

Building from source

git clone https://github.com/fmlgamer/aql.git
cd aql && npm install
npm run build      # emit dist/
npm test           # run the suite (unit + real SQLite integration)
npm run test:types # type-check source, tests, and examples

The compilation pipeline

AQL is a compiler, not an interpreter. Applications never construct SQL directly.

flowchart LR
  A["Typed builders<br/>(from/select/insert/…)"] --> B["Immutable AST<br/>(structure + literals)"]
  B --> C["Validate<br/>(types, schema, dialect)"]
  C --> D["Optimize<br/>(fold, simplify)"]
  D --> E["Generate SQL<br/>(dialect backend)"]
  E --> F["CompiledQuery<br/>{ sql, params }"]
  F --> G["Driver<br/>(pg / mysql2 / sqlite)"]
Loading

Each stage is a pure function over immutable data, so queries are inspectable and reproducible.


Defining a schema

import { table, col } from "@fmlgamer/aql";

const users = table("users", {
  id:        col.uuid().primaryKey(),
  email:     col.text().unique(),
  name:      col.text(),
  age:       col.integer().nullable(),   // → number | null
  active:    col.boolean().default(),    // optional on INSERT
  createdAt: col.timestamptz().default(),
});

Column builders: boolean, smallint, integer, bigint, real, double, numeric, text, date, time, timestamp, timestamptz, uuid, json, jsonb, bytea. Modifiers: .nullable(), .primaryKey(), .unique(), .default(), .references(table, column).

The schema drives type inference and validation: nullable columns widen to T | null, and columns that are nullable or defaulted may be omitted from an INSERT — enforced at compile time.


Building queries

SELECT, filtering, ordering, paging

from(users)
  .where((c) => c.users.age.gte(18))
  .where((c) => c.users.active.eq(true))   // repeated WHERE → AND
  .orderBy((c) => [c.users.name.asc(), c.users.age.desc()])
  .limit(20)
  .offset(40)
  .distinct()
  .select((c) => ({ id: c.users.id, name: c.users.name }));

The callback argument c only contains the tables currently in scope, each fully typed — referencing a non-existent column, or a table you have not joined, is a compile error.

Joins

from(users)
  .innerJoin(orders, (c) => c.orders.userId.eq(c.users.id))
  .where((c) => c.orders.total.gt(100))
  .select((c) => ({ name: c.users.name, total: c.orders.total }));

innerJoin, leftJoin, rightJoin, fullJoin, and crossJoin are available. Each join adds its table to the scope threaded through subsequent callbacks.

Aggregation

from(orders)
  .groupBy((c) => c.orders.userId)
  .having((c) => count(c.orders.id).gt(5))
  .select((c) => ({
    userId:  c.orders.userId,
    orders:  count(c.orders.id),
    revenue: coalesce(sum(c.orders.total), 0),
  }));

The validator ensures every bare column in the projection is either grouped or wrapped in an aggregate.

Set operations

const a = from(users).where((c) => c.users.active.eq(true)).select((c) => ({ id: c.users.id }));
const b = from(users).where((c) => c.users.age.gte(65)).select((c) => ({ id: c.users.id }));
a.union(b);          // also: unionAll, intersect, except

Subqueries

// IN (subquery)
from(users).where((c) => c.users.id.in(activeUserIds)).selectAll();

// EXISTS
from(users).where((c) => exists(ordersForUser)).selectAll();

// Derived table (subquery in FROM)
const recent = from(orders).where(/* … */).select((c) => ({ userId: c.orders.userId }));
from(recent.as("r")).selectAll();

Expressions

Every column and value is an Expression<T> with a fluent, typed combinator surface:

  • Comparisoneq, ne, lt, lte, gt, gte, isDistinctFrom, isNotDistinctFrom
  • Null testsisNull, isNotNull
  • Logical (boolean expressions only) — and, or, not, plus free functions and(...), or(...), not(x)
  • Arithmeticadd, sub, mul, div, mod, neg
  • Textlike, notLike, ilike, concat
  • Sets / rangesin, notIn, between, notBetween
  • Conversioncast(type)
  • Orderingasc(), desc()

Raw JavaScript values passed to any combinator are automatically lifted to parameters:

c.users.name.eq("Ada");           // ("users"."name" = $1), params: ["Ada"]
c.users.age.between(18, 65);      // ("users"."age" BETWEEN $1 AND $2), params: [18, 65]
c.users.email.in(["a", "b"]);     // ("users"."email" IN ($1, $2)), params: ["a", "b"]

The type system uses TypeScript's this-parameter constraints so that, e.g., .and() is only callable on boolean expressions and .ilike() only on text expressions. Cases the static system cannot express (comparing a text column against a numeric column) are caught by the runtime validator.

CASE

caseWhen<string>()
  .when(c.users.age.gte(65), "senior")
  .when(c.users.age.gte(18), "adult")
  .else("minor");

Functions

Aggregates: count, sum, avg, min, max, stringAgg. Scalar: lower, upper, length, trim, concat, coalesce, abs, round, floor, ceil, now, currentDate. Application-defined: customFn<T>(name, args, resultType).


Mutations

// INSERT — required columns enforced at compile time
insertInto(users)
  .values({ id, name: "Ada", email: "ada@x.io" })
  .onConflictDoUpdate(["email"], { name: "Ada Lovelace" })
  .returning((c) => ({ id: c.id }));

// UPDATE — RHS can reference existing columns
update(orders)
  .set((c) => ({ total: c.total.add(5) }))
  .where((c) => c.id.eq(orderId));

// DELETE — an unqualified delete must opt in with .all()
deleteFrom(users).where((c) => c.active.eq(false));

RETURNING compiles on PostgreSQL and SQLite; the validator rejects it for MySQL and SQL Server rather than producing SQL that would fail at runtime.


Type safety in action

from(users).select((c) => ({ x: c.users.nope }));
//                                       ^^^^ compile error: no such column

from(users).where((c) => c.orders.total.gt(1));
//                          ^^^^^^ compile error: 'orders' is not in scope

insertInto(users).values({ id, name: "Ada" });
//                        ^ compile error: 'email' is required

const rows = await db.all(query); // rows: { id: string; name: string }[]  (inferred)

Result row types are inferred from the projection, so consuming query results is fully typed end to end.


Security by construction

const attacker = "'; DROP TABLE users; --";
from(users).where((c) => c.users.name.eq(attacker)).selectAll().compile(postgres());
// sql:    SELECT * FROM "users" WHERE ("users"."name" = $1)
// params: ["'; DROP TABLE users; --"]     ← inert data, never syntax

There is no string escaping anywhere in the code path because untrusted values are never turned into SQL text in the first place. This holds all the way down to a real database — see the end-to-end injection tests in tests/integration.sqlite.test.ts.

Compiling untrusted ASTs

The AST is plain data, so it can also arrive from JSON or a generator. Those paths bypass TypeScript, so the compiler independently validates every field it emits verbatim — identifiers, function names, operators, join/set/sort keywords — and rejects anything else. To additionally forbid the raw() escape hatch when you do not fully trust an AST, compile in safe mode:

compile(untrustedAst, postgres(), { allowRaw: false }); // throws on any raw SQL

The full threat model and guarantees are documented in SECURITY.md.


Backends

import { postgres, mysql, sqlite, sqlserver } from "@fmlgamer/aql";

query.compile(postgres());   // $1, $2  · "ident"  · native ILIKE · RETURNING
query.compile(mysql());      // ?       · `ident`  · CONCAT()     · LIMIT ? OFFSET ?
query.compile(sqlite());     // ?       · "ident"  · emulated ILIKE
query.compile(sqlserver());  // @p1     · [ident]  · OFFSET…FETCH paging

A dialect encapsulates placeholder style, identifier quoting, paging syntax, type mapping, and capability flags. The compiler is written once against the Dialect interface, so a new backend is a subclass — the core language model is untouched.

Installation and a ready-made driver adapter for every destination (PostgreSQL, MySQL/MariaDB, SQLite, SQL Server) are in docs/backends.md.


Optimization

Optional, semantics-preserving passes run before code generation:

  • boolean identity/annihilator laws — x AND TRUE → x, x OR TRUE → TRUE
  • double-negation elimination — NOT (NOT x) → x
  • constant folding of arithmetic over literals — (1 + 2) * 10 → 30

Folding only ever touches constant literal nodes, so the security guarantees are preserved. Disable per compile with .compile(dialect, { optimize: false }).


The unsafe escape hatch

When a feature has no safe combinator, drop down explicitly with raw — interpolated values are still bound as parameters:

import { raw } from "@fmlgamer/aql";

const q = "ada";
from(users)
  .where(() => raw<boolean>`similarity(name, ${q}) > ${0.3}`)
  .selectAll()
  .compile(postgres());
// sql:    SELECT * FROM "users" WHERE similarity(name, $1) > $2
// params: ["ada", 0.3]

raw trusts the SQL you write, never the data flowing through it. Only the static fragments are emitted verbatim. rawSource(alias)\…`` provides a raw FROM/JOIN source for table-valued functions and vendor constructs. Every use is greppable and obvious in review.


Executing queries

AQL compiles queries; a tiny Driver runs them. Implementing a driver is the only integration work required.

import { Database } from "@fmlgamer/aql";

const db = new Database(new PostgresDriver(pool));

const rows  = await db.all(query);   // typed rows
const one   = await db.first(query); // first row or null
const { rowCount } = await db.run(insertStatement);

A MockDriver records executed statements and returns canned results for tests and demos. Adapter sketches for pg, mysql2, and better-sqlite3 are in examples/adapters.ts.


Command line & Docker

Because the AST is plain JSON, AQL ships an aql CLI that compiles a serialized query to parameterized SQL — handy for pipelines or other languages.

# Compile a serialized AST for a chosen dialect
aql compile --dialect postgres query.json
cat query.json | aql compile -d sqlite --safe --pretty

aql dialects      # list supported dialects
aql --help

--safe compiles in safe mode (allowRaw: false), so a hostile AST cannot smuggle raw SQL. A prebuilt image is on GHCR:

docker run --rm -i ghcr.io/fmlgamer/aql compile -d postgres < query.json

See examples/query.sample.json for the AST format.


Extensibility

  • FunctionscustomFn<T>("earth_distance", [a, b], "double") builds a typed call with no compiler changes.
  • Dialects — subclass Dialect and override the handful of hooks your engine needs.
  • AST — the tree is modular data; new expression kinds integrate with the existing validate → optimize → generate pipeline.

Project layout

src/
  types/sql-types.ts        Canonical SQL type system + TS type mapping
  ast/nodes.ts              Immutable AST node definitions (structure vs data)
  schema/                   Typed tables and columns
  builder/                  Fluent builders: expression, functions, select, insert, update, delete
  compiler/
    dialect.ts              Backend abstraction
    dialects/               postgres, mysql, sqlite, sqlserver
    safety.ts               Identifier/function/operator allowlists (injection guards)
    validator.ts            Type & structural checking
    compiler.ts             validate → optimize → generate SQL
  optimizer/optimizer.ts    Semantics-preserving rewrites
  unsafe/raw.ts             Explicit raw-SQL escape hatch
  executor/executor.ts      Driver interface + Database facade + MockDriver
  bin/aql.ts                Command-line compiler
  errors.ts                 Error hierarchy
tests/                      Unit, security-hardening, and real SQLite integration tests
docs/                       Backend setup guide
examples/                   Readable, type-checked tour + driver adapters

Further reading: SECURITY.md (threat model & guarantees), docs/backends.md (per-database setup), CONTRIBUTING.md, and CHANGELOG.md.


Design philosophy

SQL is treated as a compilation target, not a programming language for application developers. Developers describe relational operations with structured, typed constructs; the compiler is solely responsible for turning them into efficient, parameterized SQL. This separation is what enables stronger type checking, backend independence, compile-time validation, better tooling, and security guarantees that are simply unavailable when queries are strings.

The ultimate goal: make unsafe query construction impossible by design.


License

MIT

About

No description, website, or topics provided.

Resources

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages