Skip to content

[GP-02] Support the power operator ^ in the expression language - #288

Open
aoustry wants to merge 8 commits into
mainfrom
claude/exponent-operator-plan-kw8b03
Open

[GP-02] Support the power operator ^ in the expression language#288
aoustry wants to merge 8 commits into
mainfrom
claude/exponent-operator-plan-kw8b03

Conversation

@aoustry

@aoustry aoustry commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Process ID

Process: GP-02

Closes #287.

Description

Adds the binary power operator ^ to the GEMS expression language. It was previously not even a lexer token, so a library YAML using ^ failed to parse.

Grammar (grammar/Expr.g4) — ^ is right-associative and binds tighter than unary minus and than * /, following standard mathematical notation, so -2^2 parses as -(2^2) and 2^3^2 as 2^(3^2).

Time shifts^ means inside x[...] exactly what it means in a general expression; what differs is the machinery needed to get there. Time shifts are parsed by a separate sub-grammar (shift / shift_expr / right_expr), which exists because a shift must begin with an explicit + or - and that sign has to bind loosely (t - d + 1 is (t - d) + 1, not t - (d + 1)). Unlike expr, that sub-grammar is a hand-written cascade of rules, so precedence is not implicit in the order of the alternatives — it is encoded in which rule each operand recurses into. Adding ^ to expr therefore gives it nothing inside a shift: the same precedence has to be restated there, as a new tier shift_operand / shift_primary sitting above right_expr. Two operands go through it:

  • the leading sign's operand is widened from a bare atom to shift_operand, so the exponent binds tighter than the sign and x[t-2^2] shifts by -4 rather than +4 — matching -2^2 = -(2^2) in expr. It cannot simply be right_expr, which would also swallow * / / and make t - 2*3 ambiguous;
  • the right-hand side of ^ is shift_operand rather than right_expr, which is the muldiv tier and would be entered at precedence 0 and greedily swallow * / / — so x[t-2^2*3] shifts by -12, not -64, again matching expr.

As a consequence signedAtom and signedExpression merge into a single signedOperand alternative. The one deliberate divergence from expr is that the exponent is unsigned here, so a signed exponent is rejected inside a time shift (x[t+2^-1] is a parse error), a fractional shift being meaningless, while 2^-1 remains available in a general expression. ** is not accepted as an alias anywhere.

Degree^ adds no rule of its own; where it may be used follows from the existing linearity requirement, as for floor / ceil / abs / round / min / max. ExpressionDegreeVisitor gains the corresponding case, and the linear builder handles the literal exponents 0 and 1, which the degree check lets through.

AST — new PowerNode(BinaryOperatorNode), plus __pow__ / __rpow__ on ExpressionNode and __pow__ on the SupportsOperations protocol, so ExpressionVisitorOperations supplies a default power and only the visitors that do not derive from it need an explicit implementation.

Also fixes the stale output path in grammar/generate-parser.sh and grammar/README.md, which still pointed at the pre-rename src/gems/....

Note on the diff size

Most of the diff is regenerated ANTLR output, not hand-written code:

file changed lines of which bare integers
ExprLexer.py 525 512 (97%)
ExprParser.py 1584 952 (60%)

Those integers are the serialized ATN, ANTLR's state machine, emitted as a flat list that black puts one entry per line. Two things inflate it further: ANTLR numbers implicit literal tokens in the order they appear in the grammar, and the power alternative has to sit above negation for the precedence to be right — so '^' lands at index 2 and shifts every later token by one, renumbering the whole automaton and both .tokens files.

None of it is formatting churn: the previously committed files already pass black --check, so the entire diff comes from regeneration. The reviewable unit is grammar/Expr.g4; the rest is reproducible by running grammar/generate-parser.sh.

Impact Analysis

Affected modules: expression/ (grammar, AST, parsing, all visitors), model/ (resolve_library, port), simulation/ (linearize, vectorized_builder).

No change to solver output for existing studies. This is purely additive: ^ was previously a parse error, so no GEMS file in circulation can contain it, and the precedence of the existing operators is unchanged — power is inserted above muldiv and negation, which does not reorder any + - * / parse. The full suite passes unchanged, including all e2e reference studies.

The only behavioural change is that expressions previously rejected at parse time are now accepted, and ^ applied to a variable outside extra-outputs is rejected with the existing non-linearity error rather than a parse error.

Tests

51 new tests (585 → 636 passing): parsing and precedence, print/re-parse round-trip, degree, printer / equality / copy, library-level accept and reject, numeric evaluation, the linear builder (parameter powers match the manually expanded expression), extra-outputs, and a new YAML-driven e2e test asserting hand-computed values.

Checklist

  • Unit tests pass (pytest) — 636 passed, 6 skipped, 2 xfailed
  • Type checking passes (mypy) — no issues in 66 source files
  • Formatting passes (black, isort)
  • pyproject.toml version bumped if applicable — not bumped: Unreleased already carries two features with version = "0.1.3" untouched, so versioning looks like a release-time step here. Say if it should be bumped in this PR.
  • AGENTS.md reviewed for impact and updated if needed — reviewed, no change needed

claude added 8 commits August 22, 2026 12:44
Adds the binary power operator `^` to the GEMS expression language, as in
Antares Simulator. It was previously not even a lexer token, so a library
YAML valid for the C++ interpreter failed to load in GemsPy.

Grammar (`grammar/Expr.g4`):
- `^` is right-associative and binds tighter than unary minus and than
  `*` `/`, following standard mathematical notation. This is a deliberate
  deviation from Antares Simulator, where `-2^2` currently parses as
  `(-2)^2 = 4`; here it parses as `-(2^2) = -4`.
- Inside a time shift, the sign's operand is widened to a power operand so
  that `x[t-2^2]` shifts by -4, and the right-hand side of `^` uses a
  dedicated `pow_expr` rule rather than `right_expr`, so `^` does not
  swallow a trailing `*` (`x[t-2^2*3]` shifts by -12). `signedAtom` and
  `signedExpression` merge into a single `signedOperand` alternative.
- A signed exponent is rejected inside a time shift: `x[t+2^-1]` is a parse
  error, a fractional shift being meaningless.
- `**` is not an alias: GEMS accepts `^` only, matching Antares.

AST and visitors:
- New `PowerNode(BinaryOperatorNode)` plus `__pow__` / `__rpow__` on
  `ExpressionNode`, and `__pow__` on the `SupportsOperations` protocol so
  `ExpressionVisitorOperations` supplies a default `power`.
- Explicit `power` implementations for the visitors that do not derive from
  it.

Degree and linearity:
- The exponent must not depend on a variable, in any context. A degree-0
  base stays degree 0; otherwise the degree is `base_degree * exponent` for
  a non-negative integer literal exponent, and infinite otherwise. So `x^2`
  in a constraint raises the existing non-linearity error, while
  `extra-outputs` accept it. The linear builder handles the literal
  exponents 0 and 1, which the degree check lets through.

Also fixes the stale output path in `grammar/generate-parser.sh` (it still
pointed at the pre-rename `src/gems/...`) and in `grammar/README.md`.

Documentation: new "Expression syntax" user-guide page covering the operator
set, precedence, the `-2^2` convention and the Antares divergence.

No change to solver results for existing studies: `^` was previously a parse
error, so no GEMS file in circulation can contain it, and the precedence of
the existing operators is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DqK4sz5N7o9FW3uyRjq6U
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DqK4sz5N7o9FW3uyRjq6U
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DqK4sz5N7o9FW3uyRjq6U
`ExpressionDegreeVisitor.power` raised a ValueError when the exponent
depended on a variable, which made `is_linear` / `is_constant` partial:
every caller treats them as plain booleans, so a port-field definition
such as `2^g` aborted `resolve_system` instead of taking the non-linear
branch, and `optimization.py` called them from inside an
`except NotImplementedError` handler, chaining the two exceptions.

Returning `math.inf` keeps the predicates total and aligns `^` with the
other non-polynomial operators (`floor`, `ceil`, `abs`, `round`, `min`,
`max`). A variable exponent in a constraint is still rejected, now via
the usual "Non-linear expression is not allowed in ..." error, which
names the offending context.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DqK4sz5N7o9FW3uyRjq6U
A variable-dependent exponent is no longer a rule of its own: it yields
an infinite degree, so it is covered by the existing constraint that
constraints, bounds and objective contributions be linear.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DqK4sz5N7o9FW3uyRjq6U
`pow_expr` / `pow_atom` are the two highest precedence tiers of the shift
sub-grammar, not power-specific rules: every operand of a time shift is
dispatched through them whether or not `^` is involved, so a bare `3` in
`x[t-3]` went through `pow_expr -> pow_atom -> atom`.

Renames them to `shift_operand` / `shift_primary`, and the two
pass-through labels `rightPow` / `rightPowAtom` to `rightOperand` /
`rightPrimary`. `rightExpression`, `rightAtom` and `rightPower` keep their
labels, so those visitor methods are untouched.

Behaviour is identical: the regenerated automaton is unchanged (only rule
and context names move), and the comment now records why the tiers cannot
be folded into `right_expr` — the leading sign of a shift needs an operand
that accepts `^` but not `*` and `/`, otherwise `t - 2*3` becomes
ambiguous and reassociates to `-(2*3)`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DqK4sz5N7o9FW3uyRjq6U
Conflicts were limited to two test files, where main and this branch each
appended a new test at the same place; both sides are kept:
  - tests/unittests/gems_craft/expressions/visitor/test_printer.py
  - tests/unittests/gems_runner/expression/test_evaluation.py
The import block of test_evaluation.py takes the union as well: main's
LowerBoundNode/UpperBoundNode alongside this branch's parse_expression.

Also corrects a silent auto-merge artifact in docs/CHANGELOG.md. This branch
listed the power operator under [Unreleased]; main renamed that section to
[0.2.0] and opened a new [Unreleased], so the entry was carried into the
released 0.2.0 block. Moved back under [Unreleased], since 0.2.0 shipped
without '^'.

main did not touch grammar/ or the generated ANTLR files, so no regeneration
is needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KouPcoHJZyxHurBzHqYSLk
State up front that '^' means inside a time shift exactly what it means in
a general expression, and that the extra rules exist only because the shift
sub-grammar is a hand-written cascade: precedence lives in which rule each
operand recurses into, not in the order of the alternatives, so an operator
added to "expr" gets nothing inside a shift until it is restated there.

Each reason for keeping shift_operand/shift_primary separate from right_expr
now names the "expr" behaviour it reproduces, and the unsigned exponent is
called out as the one deliberate divergence rather than one more item in the
same list.

Comments only; the generated parser is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KouPcoHJZyxHurBzHqYSLk
@aoustry
aoustry marked this pull request as ready for review August 25, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[GP-02] Support the power operator ^ in the expression language, as in Antares Simulator

2 participants