Roadmap

What we build toward and what we have decided never to build. Ordered work is in GitHub issues; this is the index and the rationale. Everything is judged against the ceiling in docs/ARCHITECTURE.md: affine ∩ relational ∩ local.

How we measure

Not "can we say what other tools say?" — that has no finite answer — but "what is sayable in a language that must stay streamable?" A primitive is done when lowering.py accepts it and the differential test passes.

What we publish is coverage against two corpora, generated by a test rather than by hand (#27): Calliope's base math plus its published custom-math library, and PyPSA's optimization/constraints.py. Every item scores expressible, out of scope by design (data prep — SPEC §11), or genuinely arbitrary Python (an escape island, or stays a callback). No numbers are published until #27 produces them.

Expressiveness is not the axis we lead on. Cost is, on the one claim a reader can check rather than take: end-to-end cost to a loaded solver, wall and peak, against the eager lane's own best path to the same place (benchmarks). Build memory alone is not that claim — it is a small fraction of what solving costs, so shrinking it further changes nothing a caller feels. Two claims we would rather lead on are not measured yet and are therefore not made: the floor under the LP-file route as a cold cost, and marginal cost per model in a loop. Both are listed in Not measured yet; neither is quoted until a table backs it.

The degree axis

Of the three admissibility rules, degree is the only scope choice; relational and local fall out of the architecture. The plan is affine-by-design, quadratic out of scope — a decision with a revisit condition, not an axiom. It holds because demand is almost entirely quadratic objectives, and for convex 1-D curves we ship the substitutes practitioners prefer (piecewise: with convex: true, and the epigraph pattern in plain YAML) which keep LP duals, warm starts and MILP compatibility that a quadratic objective gives up. Quadratic constraints are a class HiGHS cannot solve at all, and every sink we ship assumes affine rows.

What would have to hold to move it. The old list said "sinks carry a quadratic stream", and that condition is already met — HiGHS passHessian and Gurobi setMObjective both exist, and linopy's QuadraticExpression means the oracle covers it. What remains is one blocker and one cost:

1. MIQP is excluded on the default solver — the blocker. HiGHS returns kError for Hessian + integrality, and binary:/integer: and nonconvex piecewise: are already in the language, so quadratic would conflict with shipped features. Gurobi has no such exclusion, which makes this a capability question: quadratic is the motivating case for Track 4's conjunction-exclusion axis, not an exception to it.
2. The Hessian is passed whole — a cost, not a rule-4 violation, since under the aligned-only scope Q is diagonal and small beside the solver's own model. The arithmetic and its two caveats are in docs/benchmarks.md.

Plus a data-time convexity guard, a new plan node, a lowering case and a label-ordering change — a bigger diff than SOS, which is a declaration and needs none of it. The question that actually decides it is whether the demand is LP-only: if the fuel-curve users also want unit-commitment binaries, (1) kills it on the default path however cheap the plumbing is, and piecewise: {convex: true} stays the answer.

An escape cannot lift degree either way: islands return affine COO rows. Scope if it lands, fixed in advance: degree ≤ 2, factors coordinate-aligned, objective only, convex-guarded — aligned products are a pointwise self-join; general bilinear coupling is a cross join with |terms|² rows and is excluded permanently.

Track 1 — language primitives (taxed)

#	Primitive	Proposed surface	Unlocks	Locality	Status
1	Indexed access, constant	at(x, over=dim, index=value)	boundary conditions, multi-period linking, carrier ratios	pointwise	planned
2	Indexed access, lookup	at(x, over=dim, index=map)	DC power flow, hydro cascades — topology math beyond nodal balance	pointwise	planned
3	Coordinate helpers in where	index(dim, i), negative i from the end	"first/last snapshot" without hardcoding a label	coordinate-space	blocks 4
4	Window sum	sum_next_n(x, over=dim, n=N)	rolling horizons, DSR, reserves, min up/down time	bounded-halo	needs 3
5	Component-aware where	defined(v) over variables	masks compose across components instead of being re-derived and drifting	pointwise	needs design
6	all-reduction in where	all(x, over=dim)	reducing a mask over an out-of-foreach dim at all — today that is a DimensionError, since an implicit reduction would silently widen the mask	pointwise	small
7	Bounds as expressions	—	signs, bidirectional flows, component libraries	pointwise	#31
8	Namespacing / qualified names	—	component libraries	n/a	#29
9	Compose-then-build (schema merge)	—	one lower/stream pass over merged libraries	n/a	#30

Items 1–4 are one piece of work — indexed access plus the masks that make it usable — and are the difference between "dispatch and balances" and "network and multi-period math". Item 2 is the adjoint of group_sum: same mapping table, join without the aggregate. Calliope spells 1 and 2 as x[dim=value]; we do not, because a dimension in a kwarg key cannot be parameterised by a macro.

Tracks 2–3 — untaxed

Nothing here touches the build path or either lane's semantics, with one flagged exception noted under 2b.

• 2 — post-solve stage. Declared derived results (LCOE, curtailment, emissions by group) are SQL over solution tables we already materialise. Sequenced after Track 1 items 1–4.
• 2b — the operational surface. Mostly queries, and the work that decides whether the engine is usable at 3am: algebraic rendering of a row, IIS read-back where a solver provides one, model statistics and coefficient-range diagnostics. Dual read-back has shipped — sol.dual(constraint), the same label join as primal against the row table, refusing rather than zero-filling where a model has no dual solution; reduced costs and slacks ride the same join and have not (#78). The headline item is elastic relaxation, the infeasibility answer — HiGHS has no IIS, so solver_direct returns a status code for a model too large to open in an editor; slack variables with penalty costs, then a query of nonzero slacks grouped by constraint block, needs no solver feature, works on every sink, and points at which constraints conflict rather than at a minimal set. It is the exception on this track: adding slacks means new variables, so it needs a schema-level expansion pass the way piecewise: does, and is taxed like a primitive rather than free (#80).
• 2c — value-only re-solve. In scope, because var_label is the solver column index with no remapping: changing a bound or an RHS is a label query plus changeColsBounds/changeRowsBounds, which is what rolling horizon, sweeps and Benders need. With one precondition: "value-only" has to be a property of the model, not of the user's intent. var_label is a ROW_NUMBER() over the rows that survive the where mask, so a parameter read by a where changes which rows exist and every later label shifts — an in-place bound update then writes to the wrong columns. A changed group_sum mapping leaves the labels alone but rewrites A, which a bounds-only update cannot express at all. Both are decidable off the resolved AST, and that check ships with the feature (#82). Structural editing stays out — it invalidates the label contract. Related gap: warm starts, which solver_direct would need to set through highspy.
• 3 — AST consumers (#21): math → LaTeX (a tree walk, no data — the product is the declarative math, so it should render as typeset documentation), CLI check/solve/write (#35), observability (#34).

Track 4 — sink capabilities

Decision: the ceiling and sink capability are two axes, not one. The closure (affine ∩ relational ∩ local) is about streamability and is solver-independent; what a sink can ingest is separate. Treating them as one let HiGHS's limits read as architectural law — the measured table is in docs/benchmarks.md.

Adopted: a declared capability per sink (not per solver — lp_file is not a solver but has capabilities), modelled on linopy's SolverFeature / Solver.features, with two divergences. Entries are three-valued (native / reformulated / absent) so satisfying a capability by reformulation later is additive; and the model expresses conjunction exclusions, because linopy declares HiGHS with INTEGER_VARIABLES and QUADRATIC_OBJECTIVE in one flat frozenset while HiGHS refuses the pair. check(model, sink=...) takes the sink optionally: bare check asks whether the model is sayable and lowerable, the argument asks whether that sink will take it. Quadratic is in scope as the case that forces the exclusion axis; SOS goes first on effort, not principle.

Hard rule 3 is untouched — it governs lanes accepting the same language, and naming an alternative sink is not the lane redirection it forbids.

Full design, work breakdown and open questions: #89. First spike is SOS2 on lp_file: no new dependency, no license question, no memory risk.

Track 5 — the memory axis

The engine holds the model it builds, so peak tracks the model rather than a number the caller sets. That is the right default — it is also what makes the lifetime disappear from the API — but it leaves two things open, and they are worth stating rather than discovering:

• A declared ceiling. There is no way today to say "build this within N gigabytes or fail". The honest version is not a knob bolted onto the engine but partition-wise execution: build the model in slices of the leading foreach dim and sink each slice, which the locality closure (pointwise ∩ bounded-halo) already guarantees is safe. Wanted when a model is written once and run on a machine chosen by someone else.
• The solver is the larger term anyway. Measured at roughly an order of magnitude above the build at 10⁷ variables (benchmarks), so a ceiling on the build is worth having for the write path — fk.write to LP or MPS, handed to something else — and worth much less when the same process goes on to solve.

Deliberate non-primitives

Billed vs banned: an escape: island (#38) returns affine COO rows, so it buys back what the relational and local rules exclude, priced against a label budget enforced before any Python runs. It cannot buy back degree. Reach for one only after #38's decision tree: different values → a parameter; different in-language shape → a second constraint with where; only unsayable shapes escape.

Request	Why not	Instead	Escape?
Quadratic / bilinear terms	degree pinned at 1	piecewise: (convex: true) or the epigraph pattern	banned today, blocked by the QP∧MILP exclusion on the default solver — a Track 4 capability question, and the bigger diff of the two (new plan node, lowering, convexity guard)
SOS / indicator (#23)	the default solver has no such concept	piecewise: λ-formulation (SOS2's usual purpose); big-M for indicator, given bounds	banned today, and the easier one: lp_file carries it as a text section and Gurobi natively — it needs Track 4, not a new invariant
Cumulative / running sums, normalisations	global — breaks partition-wise execution	state-variable recurrence (a storage SOC balance is the rewrite)	billed — but the rewrite is still right at scale, O(T) vs O(T²)
Conditionals, iteration, data-dependent structure	destroys the closed AST	where masks + foreach dims; computation → data prep	billed — inside an island only
Resampling, clustering, IO, units	data layer, not math	preprocess; pass a parameter	n/a — free in Python already
Domain helpers (reduce_carrier_dim)	encodes one domain into the language	component libraries over generic primitives	n/a
Arbitrary array ops (merge, reindex)	unbounded; xarray with extra steps	data prep	billed

Honest snapshot

Cheaper here, because the model is tables: model statistics and coefficient-range diagnostics; IIS read-back (a join, not a scatter — as dual read-back, now shipped, already was); serialization (parquet, more portable than netCDF); elastic relaxation; and dualization — transposing a COO matrix is swapping two column names.

Ahead of comparable declarative layers: sparse-by-construction build with no dense intermediate, and a hand-off straight to the solver rather than through a file (see benchmarks; a declared memory ceiling is Track 5, not something we have); parameterised macros: (Calliope's sub-expressions take no arguments); binary and integer variables; piecewise as N links with per-link signs, convex mode and active gating; non-cyclic shift; load-time validation of every expression, where string and uncalled macro template.

Behind: Track 1 items 1–6, plus Tracks 2 and 3. Where linopy stays ahead it is not expressiveness but the post-solve object (labelled DataArrays vs our tidy tables — to_dataarray bridges), debugging (IIS via Gurobi, print() of a row, against our status code — until Track 2b exists), and lifecycle (mutate, re-solve, warm start, callbacks vs our build-and-solve). None are ceiling questions; all are cheaper here once written, which is why they need scheduling rather than assuming they follow from the architecture.

Solvers we actually speak: HiGHS through solver_direct, Gurobi planned on the same path, LP files for everything else. Stated because anyone arriving from linopy will assume solver-agnosticism; the LP route costs the float→text→parse round trip solver_direct exists to remove, so "fast" is per-solver binding work rather than a property of the engine.