Skip to content

Mosek interface ignores basic solution when IPM ends in dual_infeas_cer but crossover finds optimal basis #665

Description

@mgrabovsky

Version Checks (indicate both or one)

  • I have confirmed this bug exists on the lastest release of Linopy.

  • I have confirmed this bug exists on the current master branch of Linopy.

Issue Description

When solving an LP with Mosek through linopy, the solver can produce two distinct solutions with separate statuses: an interior-point solution (soltype.itr) and a basic solution (soltype.bas) obtained by basis identification (crossover). linopy's Mosek interface appears to read only the interior-point solution. If the IPM terminates with solsta.dual_infeas_cer but crossover then recovers a solsta.optimal basic solution, linopy reports infeasible_or_unbounded with Objective: nan and the actual optimal solution is silently discarded, even though Mosek itself has it.

I've hit this on a PyPSA model with two investment periods, solved via n.optimize(solver_name="mosek"). Mosek's solution summary at termination:

Interior-point solution summary
Problem status : DUAL_INFEASIBLE
Solution status : DUAL_INFEASIBLE_CER
Primal. obj: -1.0000000000e+00 nrm: 2e-06 Viol. con: 2e-06 var: 8e-09
Basic solution summary
Problem status : PRIMAL_AND_DUAL_FEASIBLE
Solution status : OPTIMAL
Primal. obj: 6.4683156642e+10 nrm: 6e+07 Viol. con: 1e-07 var: 2e-13
Dual. obj: 6.4683156642e+10 nrm: 3e+06 Viol. con: 4e-12 var: 4e-10

linopy output:

WARNING:linopy.constants:Optimization potentially failed:
Status: warning
Termination condition: infeasible_or_unbounded
Solution: 0 primals, 0 duals
Objective: nan
Solver model: not available
Solver message: solsta.dual_infeas_cer

The basic solution is feasible and optimal according to Mosek, but linopy never queries it and returns NaN. There is no way for the user to recover the solution through linopy – n.objective, primals, and duals are all NaN/empty.

I've tried debugging this with Claude for a bit, but didn't want to get too deep into linopy internals. Here's what it has to say:

Mosek may return up to three solutions per task (interior, basic, integer), each with its own prosta/solsta. The IPM and the basis identification phase that follows it are independent and can disagree. In particular, when the IPM ends in a numerically marginal state near the optimum (large constraint matrix range, multi-period structure, etc.), it can mis-classify its own iterates as a dual-infeasibility ray (dual_infeas_cer) even though the actual problem is bounded — the certificate violation here is 2e-6, i.e. a near-certificate, not an actual one. Crossover then starts from those iterates and the dual simplex pivots to a true optimal vertex, which Mosek records as the basic solution with solsta.optimal.

A correct interface must query solutiondef/getsolsta for both soltype.bas and soltype.itr, pick the better status, and read the corresponding solution.

Apparently, the same bug was found in cvxpy in 2017 (see cvxpy/cvxpy#335, fixed in cvxpy/cvxpy#347).

Reproducible Example

IdonothaveasmallreproducerthebugistriggeredbyIPMterminationinalargemulti-periodPyPSAnetwork.

Expected Behavior

Linopy should recognise that Mosek has found an optimal solution in the crossover and load that instead of jumping to conclusions based on the IPM result only.

Installed Versions

Details linopy==0.6.7 Mosek==11.1.11 pypsa==1.2.0

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions