Skip to content

Port to PowerIO 0.11 - #76

Merged
samtalki merged 4 commits into
mainfrom
claude/powerdiff-powerio-migration-sulux7
Sep 5, 2026
Merged

Port to PowerIO 0.11#76
samtalki merged 4 commits into
mainfrom
claude/powerdiff-powerio-migration-sulux7

Conversation

@samtalki

@samtalki samtalki commented Aug 22, 2026

Copy link
Copy Markdown
Member

Summary

Port to PowerIO 0.11 (powerio C ABI 7). This PR previously targeted 0.9; 0.11
replaces the accessor-and-JSON-payload layer with typed element tables, so the seam
is rewritten rather than bumped, and the four upstream issues the 0.9 version filed
(eigenergy/PowerIO.jl#111-#114) are all closed, resolved by 0.10 and 0.11.

The governing rule is that PowerDiff is a consumer of PowerIO. Every electrical
quantity PowerIO states is read from it, and nothing PowerIO computes is computed
again here. Where PowerIO is missing something, an issue is filed upstream rather
than worked around locally.

What PowerDiff stopped computing

Was derived here Now read from PowerIO
b = -x/(r² + x²) in DCNetwork c8 - b_to off to_powerdata
g, b = r/z², -x/z² in ACNetwork c7 - g_to, c8 - b_to
two separate incidence assemblies one, shared by both network types
br_b summed then split evenly, g_fr/g_to forced to zero each terminal's charging admittance, carried per side
PWL cost inferred from the shape of a converted row Generator.cost.model, verbatim
source row numbers recovered by a second to_powerdata(net; filtered=false) pass row field i, which is that number

Nothing in src/ inverts a branch impedance any more. The charging change is a
fidelity gain: the old path averaged the two terminals and discarded the charging
conductance, which loses information on every source that states them separately.

What PowerDiff still computes, and why it is not a duplicate

admittance_matrix(net, sw) assembles Y from g, b, tap, shift and the four
terminal terms on every call. PowerIO.calc_admittance_matrix tabulates Y for one
network at its parsed values with a boolean in-service flag; PowerDiff needs Y as a
function of parameters a caller perturbs, because sw is continuous in [0, 1] and
(:vm, :g), (:va, :b) and eight more operand/parameter pairs differentiate through
g and b (src/sens/interface.jl:179-180, src/sens/topology_ac.jl:64-71 computes
∂Y/∂g and ∂Y/∂b in closed form). Freezing PowerIO's coefficients into the network
would leave those analytical derivatives comparing against a constant Y.

The same argument disposes of calc_bus_susceptance_matrix, calc_branch_flow_matrix,
calc_branch_flow_dc and calc_bus_injection_dc: each is that object evaluated at
boolean in_service. The incidence matrix is stated locally because
calc_incidence_matrix returns a bare sparse matrix with no branch or bus index map,
so relabeling it into PowerDiff's sorted-source-id space costs more than stating it.

The bug this upgrade would otherwise have shipped

PowerIO 0.11 reads rate_a through the bound check that permits infinities, so
MATPOWER's rate_a == 0 now arrives as Inf — and rate_a > 0 is true for Inf.
Left alone, every unrated branch would have reached the solver with an unbounded
thermal limit.
Stock IEEE 300 leaves all 411 branches unrated, so the restoration
example added in #77 would have stopped shedding load and failed its own assertion.
Both spellings now take the synthesized limit, with a test pinning that the 0 and
1e6 cases dispatch identically.

Also

  • parse_file/parse_matpower return a PioModule{BalancedNetwork}. It carries
    m.diagnostics (the reader's findings, as records with a code and a severity),
    m.sources[1].format, and m.history, and it is what PowerIO.emit writes back
    out. Every constructor accepts the module or the network inside it, so call sites
    are unchanged.
  • A PowerIO failure propagates as PowerIOError instead of being flattened into
    ArgumentError, which discarded the diagnostic code and the records behind it.
  • Isolated (type == 4) buses are selected out here, since to_powerdata no longer
    drops them and calc_incidence_matrix filters on branch status only. Keeping them
    would energize a bus MATPOWER declares out of service.
  • The format allowlist is replaced by pass-through, and the distribution-token
    blocklist by a check on what the source actually parsed to — one rule that also
    covers time series, scenario sets and calculation instances, with nothing here to
    trail what PowerIO reads.
  • network_findings is dropped before it ever shipped. It existed because 0.9
    reached findings only through a normalize pass PowerDiff had to own; in 0.11 they
    are a property of the module parse_file already returns.
  • parse_matpower_struct (an alias for an alias, no callers) is removed.
  • The integration contract moves into docs/src/ and is published. It was linked
    from the README and never built.
  • The restoration example from example use case of PowerDiff -  #77 switches branches through update_switching!
    rather than writing network.sw directly. Its output is byte-identical.

Verification

Run locally against PowerIO 0.11.0 / ABI 7 on Julia 1.12.5.

  • Full suite passes, zero failures and zero broken.
  • Numerically identical to the pre-port tree. 2627 quantities captured on
    pglib_opf_case118_ieee.m — dispatch, angles, flows, shedding, limits,
    susceptances, ID maps, and the Frobenius/infinity norms and per-column norms of
    eight sensitivity matrices — compared before and after. Worst absolute difference
    2.3e-12; nothing outside atol=1e-9, rtol=1e-8. The objective, sum(pg) and
    the norms of injection, flow, angle and shedding agree to all 17 digits, and the
    DCNetwork and ingest-table dumps are byte-identical.
  • PowerModels oracle, now a permanent test: ptdf_matrix against
    calc_basic_ptdf_matrix on case14, case30 and case118, agreeing to 1e-8. It shares
    no code with the ingest and pins topology, branch susceptances and reference bus.
  • The example use case of PowerDiff -  #77 example produces byte-identical output to the pre-port baseline.
  • Docs build clean, including checkdocs and cross-references.

Filed upstream rather than worked around

  • eigenergy/PowerIO.jl — to_powerdata returns bus va in degrees while its own
    docstring says angles are radians and the branch rows convert.
  • The eight DC calc_* return bare arrays with no bus or branch index map, while the
    AC family returns BusMappedMatrix; the in-service branch row selection is not
    stated in any of the eight docstrings.
  • Zero-impedance branches behave three different ways across to_powerdata,
    calc_admittance_matrix and the DC family.
  • No per-branch primitive admittance accessor, only the assembled Y and the
    ExaModels-shaped c1..c8.
  • to_powerdata drops cost.model, so a piecewise linear cost is not detectable
    from the row.
  • to_powerdata validates rows the caller will discard, so an out-of-service
    generator with a cubic cost refuses the whole conversion.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (Julia v1)

Time benchmarks
main e8ae30f... main / e8ae30f...
ac_opf/kkt_jacobian/pglib_opf_case300_ieee.m 30.3 ± 0.99 ms 30.8 ± 1.6 ms 0.984 ± 0.06
ac_opf/kkt_param/pglib_opf_case300_ieee.m/switching 2.01 ± 0.77 ms 2.05 ± 0.81 ms 0.978 ± 0.54
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_linear 1.01 ± 0.17 μs 0.992 ± 0.18 μs 1.02 ± 0.25
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_quadratic 0.43 ± 0.07 μs 0.401 ± 0.11 μs 1.07 ± 0.34
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/demand 3.14 ± 0.27 μs 3.08 ± 0.26 μs 1.02 ± 0.12
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/flowlimit 10.1 ± 6.2 μs 9.92 ± 6.2 μs 1.02 ± 0.9
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/full 0.0996 ± 0.011 ms 0.103 ± 0.0083 ms 0.966 ± 0.13
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/susceptance 6.97 ± 1.1 ms 7.21 ± 1.2 ms 0.966 ± 0.23
parser/pglib_opf_case300_ieee.m 0.745 ± 0.011 ms 1.64 ± 0.028 ms 0.453 ± 0.01
time_to_load 1.99 ± 0.021 s 2.01 ± 0.045 s 0.992 ± 0.025
Memory benchmarks
main e8ae30f... main / e8ae30f...
ac_opf/kkt_jacobian/pglib_opf_case300_ieee.m 0.373 M allocs: 12.3 MB 0.373 M allocs: 12.3 MB 1
ac_opf/kkt_param/pglib_opf_case300_ieee.m/switching 14.8 k allocs: 26.2 MB 14.8 k allocs: 26.2 MB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_linear 6 allocs: 1.83 kB 6 allocs: 1.83 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/cost_quadratic 6 allocs: 1.83 kB 6 allocs: 1.83 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/demand 9 allocs: 12 kB 9 allocs: 12 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/flowlimit 9 allocs: 16.3 kB 9 allocs: 16.3 kB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/full 0.111 k allocs: 0.754 MB 0.111 k allocs: 0.754 MB 1
dc_opf/kkt_jacobian/pglib_opf_case300_ieee.m/susceptance 27.4 k allocs: 21.5 MB 27.4 k allocs: 21.5 MB 1
parser/pglib_opf_case300_ieee.m 0.082 k allocs: 4.7 kB 0.033 k allocs: 1.22 kB 3.86
time_to_load 0.149 k allocs: 11.1 kB 0.149 k allocs: 11.1 kB 1

samtalki and others added 4 commits September 5, 2026 15:13
PowerIO carries a bound a case leaves unstated as an infinite one rather than
refusing the case, which is how MATPOWER, PowerModels, pandapower and PyPSA all
spell "no limit"; stock case9241pegase leaves the reactive limits off seven
generators.

The KKT layout is fixed, one complementarity row per bound, so an infinite bound
can be neither dropped nor carried: `rho * (qg - qmin)` with an infinite `qmin` is
`0 * Inf`, a NaN in the residual and an Inf in the Jacobian. The bound is left off
the solver model instead, and its row reads `rho = 0` -- the multiplier of a
constraint that is not there, and what a solver reports for a bound it was never
given, so the solved duals and the residual agree by construction. The derivative
of an absent bound is constant, so its fixed-regime sensitivity is zero and the
sparsity pattern is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PowerIO 0.11 replaces the accessor-and-JSON-payload layer with typed element
tables over C ABI 7, so the seam is rewritten rather than bumped. `parse_file`
returns a `PioModule{BalancedNetwork}`, which carries the reader's diagnostics,
the source record and the history alongside the case; every constructor accepts
the module or the network inside it.

PowerDiff now derives no electrical quantity PowerIO states. The series
conductance and susceptance are read back from `to_powerdata`'s terminal
admittance coefficients, and each branch terminal's charging admittance is carried
on its own side rather than summed and split evenly with the charging conductance
discarded. The incidence matrix is stated once and shared by both network types.
A generator's cost model is read off the element instead of inferred from the
shape of a converted row.

`to_powerdata` is unfiltered in 0.11: rows carry a status and their source row
number, which is the IDMapping index, so the two-pass reconciliation that used to
recover those numbers is gone. Isolated buses and everything standing on them are
selected out here, as PowerIO used to do.

A rating the source does not state now reaches PowerDiff as an infinite one where
it used to be zero, and both take the synthesized limit. Without that, every
unrated branch would reach the solver with an unbounded flow; stock IEEE 300
leaves all 411 branches unrated.

A PowerIO failure propagates as a PowerIOError rather than being flattened into an
ArgumentError, which discarded the diagnostic code and the records behind it. The
format allowlist is replaced by pass-through, and the distribution-token blocklist
by a check on what the source parsed to, which also covers series carriers and
calculation instances.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round trips go through `emit` and read the reader back off `m.sources`, and a
diagnostic is asserted as a record rather than as a line of text. New coverage for
what the port can get wrong: a source stating no thermal limit dispatches
identically whether it spells that zero or infinity, an isolated bus in the middle
of the bus table takes its branch with it without shifting the survivors, terminal
charging is carried per side, and only a balanced transmission network is modeled.

The ingest is held to an oracle that shares no code with it: `ptdf_matrix` against
`PowerModels.calc_basic_ptdf_matrix` on three PGLib cases, which pins the
topology, the branch susceptances and the reference bus together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The integration contract stated the ingest but sat outside `docs/src/`, so it was
linked from the README and never built. It moves into the docs and is rewritten
for 0.11: what PowerIO states, what PowerDiff adds on top, and why the parametric
admittance kernel is not a duplicate of `calc_admittance_matrix`.

The restoration example switches branches through `update_switching!` rather than
writing `network.sw` directly, which the DCNetwork docstring does not sanction,
and its write-up no longer pins a PowerIO version in prose. Its output is
unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@samtalki

samtalki commented Sep 5, 2026

Copy link
Copy Markdown
Member Author

Six gaps found while doing this are filed upstream rather than worked around here:

#139 is why this PR states its own incidence matrix rather than consuming calc_incidence_matrix, #141 is why the series admittance is read back by subtraction, and #142 is why the ingest collects the generator table for one field.

@samtalki
samtalki merged commit 9f1b0e2 into main Sep 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant