Skip to content

Fix xarray inferred-from data ordering - #8445

Open
Aaron Smull (asmull) wants to merge 2 commits into
mainfrom
aaron/fix-xarray-inferred-from-ordering
Open

Fix xarray inferred-from data ordering#8445
Aaron Smull (asmull) wants to merge 2 commits into
mainfrom
aaron/fix-xarray-inferred-from-ordering

Conversation

@asmull

@asmullAaron Smull (asmull) commented Aug 25, 2026

Copy link
Copy Markdown

Dataset export to xarray currently fails under the condition.

This can occur when array-valued parameters. Users may wish to acquire and save data in a particular order, and no assumptions.

This failure occurs in two cases:

  1. With pandas-based exporting to xarray, indexing is not propagated to inferred-from relationships.
  2. With direct xarray exporting, the assumption is made that shape metadata can be used to imply that, e.g., the first column of an NxM sweep can be used to infer the axis of one of the sweep values. This contract is not made explicit in the API, and should not be relied upon. If data for an NxM sweep is created in a permuted order, this MR fixes the direct export to directly extract the axis information from the larger multi-dimensional array. Backwards compatibility of ordering is maintained for non-permuted cases.

@asmull

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

@asmullAaron Smull (asmull) changed the title Aaron/fix xarray inferred from orderingFix xarray inferred-from data orderingAug 25, 2026
@asmull
Aaron Smull (asmull)force-pushed the aaron/fix-xarray-inferred-from-ordering branch from 6029ed8 to 820eb16CompareAugust 25, 2026 22:19
permuted data axis.
Fixing ordering of data when using permuted data axes
Revert "Updating documentation"
This reverts commit 60a4735.
Fixing linting
@asmull
Aaron Smull (asmull)force-pushed the aaron/fix-xarray-inferred-from-ordering branch from 820eb16 to 93cb275CompareAugust 25, 2026 22:21
@codecov

codecovBot commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.14%. Comparing base (4dafdc9) to head (b4ac0b4).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@ Coverage Diff @@## main #8445 +/- ##
==========================================
+ Coverage 71.12% 71.14% +0.02% 
==========================================
Files 305 305 Lines 31952 31973 +21 ==========================================
+ Hits 22725 22748 +23 + Misses 9227 9225 -2 

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

CopilotAI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes ordering issues when exporting QCoDeS datasets to xarray in cases where data acquisition order is permuted relative to the inferred grid structure, and when inferred-from parameters need to be exported consistently alongside reordered data.

Changes:

  • Add an index plumbing path to _add_inferred_data_vars to support order-aware insertion of inferred-from variables in the pandas→xarray export route.
  • Rework the direct-to-xarray export (_xarray_data_set_direct) to derive coordinate axes and reorder data based on the actual dependency arrays rather than relying on shape-based axis inference.
  • Extend test coverage to validate permuted-grid ordering, direct-export grid validation, and inferred handling.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

FileDescription
src/qcodes/dataset/exporters/export_to_xarray.pyAdds index-aware inferred var insertion and reimplements direct xarray export ordering/reconstruction logic.
tests/dataset/test_dataset_export.pyAdds regression + validation tests for permuted direct export and _xarray_data_set_direct input validation.
tests/dataset/test_inferred_multiple_parents.pyUpdates direct calls to _add_inferred_data_vars for the new index argument.
tests/dataset/test_parameter_with_setpoints_has_control.pyUpdates _add_inferred_data_vars invocation to pass index=None.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +111 to +118
if index is not None:
# If an index is provided, we should align the inferred data with the index.
# This is necessary because data may be reordered when transforming from a pandas DataFrame to an xarray Dataset.
# Passing an index allows the original data ordering to be preserved on reconstruction.
indexed_data = Series(flat, index=index, name=inf.name).to_xarray()
xr_dataset[inf.name] = indexed_data
else:
xr_dataset[inf.name] = (dims, flat.reshape(expected_shape))

CopilotAI commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

One or more custom setup steps configured for this repository failed during this Copilot code review run:

install pyright

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

@asmull
Aaron Smull (asmull) marked this pull request as ready for review August 26, 2026 08:22
@asmull
Aaron Smull (asmull) requested a review from a team as a code ownerAugust 26, 2026 08:22
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@asmull