Auxiliary non-dimension coordinates leak into expressions and break alignment

[aurelije]

Hi people, I am currently struggling with xarray and linopy.

I have a dimension A whose coordinates can be grouped. For me it was natural to have a dimension A with coordinates for A but also coordinate B that is built on dimension A and then swap dimensions but I failed to do it.

So here is an example.
dimension A has coordinates: [1, 2, 3, 4, 5, 6] but coordinates [1, 2, 3] belong to group 311 and rest of them to group 322. I would like to create constraints based on those 2 groups.

I have a variable with this situation:

var.coords
Coordinates:
 * A (A) int64 48B 1 2 3 4 5 6
 B (A) int64 48B 311 311 311 322 322 322

Looking at xarray docs this may be the way var.swap_dims({'A': 'B'}) and then I can do sum on other dimensions and set constraint for groups. But Variable doesn't have that method.

Next I tried with groupby on B, I got correct groups but then sum on groupby expression will throw an error that B is already present...

In the end I made B full dimension (I am breaking normalization from Databases theory :D) but looks like suboptimal solution and I have this kind of case for multiple coordinates so instead of having small dimensional case of 7 dimensions I will end with at least 9

Could I do something better?

[FabianHofmann]

Hey @aurelije, thanks for raising the issue. How about we add the swap_dims method to the variable class. Would that help?

[aurelije]

Hi @FabianHofmann I have tried 0.3.10 version and change works in creating expressions with swapping dimensions. But it fails in expressions where you do not use those "auxiliary coordinates" because linopy would try to align on them too in comparison expressions with other dataarrays and then fail because they are missing in left side.

Example:

model.add_constraints(var.sum(['A']) == params_da) 

Here I use dimension A that has coordinates, but there is an auxiliary coordinate B set on same dimension. Dataarray pamams_da has dimension A but doesn't have coordinate B. In this example I do not care about coordinate B, I do not use it in this constraint. But I get exception ValueError: coordinate 'B' not present in all datasets. Yes I could probably solve this by adding B coordinate into params_da but it beats the purpose of having elegant code.

The source of error is in expressions.py file in line: ds = xr.concat([d[["coeffs", "vars"]] for d in data], dim, **kwargs)

It looks to me that the biggest obstacle to using linopy, which has such a good idea behind, is actually xarray being too complex and clumsy. In comparison to clumsy pandas, in pandas there is a nice tutorial and good documentation with numerous examples on how to use methods and functions... Ok maybe building linopy on top of xarray is necessity, but I think that less xarray gets exposed to linopy user, the better for linopy user. Xarray should be hidden deep behind the api, from outside we should use only xarray for supplying parameters with named dimensions...

[FBumann]

Small comment from me to frame this for the upcoming milestone

Generated with claude code

Status update + scoped goal

swap_dims is now available on Variable/LinearExpression/Constraint (added in 0.3.10), so the original request in this issue is satisfied. What remains is the follow-up @aurelije raised in the thread: an auxiliary (non-dimension) coordinate leaks into expressions and breaks alignment, even in operations that never reference it.

Goal: carrying a second, non-dimension coordinate on a dimension (e.g. a group label B on dimension A) should be transparent — operations that don't use B should neither fail on it nor require callers to copy B onto every other operand.

Replicator (linopy v0.7.0):

import pandas as pd
import xarray as xr
import linopy

m = linopy.Model()
A = pd.Index([1, 2, 3, 4, 5, 6], name="A")
var = m.add_variables(coords=[A], name="var")
# auxiliary coord B groups the A coordinates (no extra dimension)
var = var.assign_coords(B=("A", [311, 311, 311, 322, 322, 322]))

# (1) groupby on the auxiliary coord
var.groupby("B").sum()
# -> ValueError: B already exists as coordinate or variable name.

# (2) a constraint that does not even reference B
m.add_constraints(var.sum(["A"]) == xr.DataArray(100.0))
# -> CoordinateValidationError: coordinate B has dimensions ('_term',),
# but these are not a subset of the DataArray dimensions ()

Both fail today. In (2) the aux coord B rides along through .sum() onto the _term dimension and then fails validation against an RHS that doesn't carry B.

Also see #703

[FBumann]

Re-tested on linopy 0.7.0 — confirming the alignment angle in this issue's current title. A non-dimension coordinate is mishandled when two operands disagree on it:

import pandas as pd
import linopy

A = pd.Index([1, 2, 3, 4, 5, 6], name="A")
m = linopy.Model()
v = m.add_variables(lower=0, coords=[A])

vb = v.assign_coords(B=("A", [311, 311, 311, 322, 322, 322]))
vb2 = v.assign_coords(B=("A", [311, 311, 311, 311, 322, 322])) # B differs at A=4

list((vb + vb2).coords) # ['A', 'B'] — linopy keeps B (one operand's values)

The operands disagree on B at A=4 (322 vs 311); linopy silently keeps one side's B, so the sum carries an auxiliary coordinate that contradicts one of its inputs. xarray, by contrast, drops a conflicting non-dimension coordinate:

import numpy as np
import xarray as xr

da = xr.DataArray(np.ones(6), coords=[A])
list((da.assign_coords(B=("A", [311, 311, 311, 322, 322, 322]))
 + da.assign_coords(B=("A", [311, 311, 311, 311, 322, 322]))).coords) # ['A'] — B dropped

So linopy neither matches xarray nor raises — it silently guesses. Tracked under the arithmetic-convention meta issue #714; the v1 convention (#591) should raise on a conflicting non-dimension coordinate, as it does for a dimension mismatch.

[FabianHofmann]

Note

The following content was generated by AI (Claude Code) and checked against master (a731680). Posted as evidence only.

Scope update: the groupby-on-auxiliary-coordinate part of this is now fixed — var.groupby("<aux>").sum() returns a LinearExpression (via #751). What remains open is the alignment leak: var.sum([...]) compared against a plain DataArray still raises CoordinateValidationError because the auxiliary coordinate leaks onto _term, and a conflicting aux coord is silently kept. Keeping open for that residual.