Skip to content

Faster encoding functions. - #8565

Merged
dcherian merged 4 commits into
pydata:mainfrom
dcherian:optimize-encode-multiindex-check
Jan 4, 2024
Merged

Faster encoding functions.#8565
dcherian merged 4 commits into
pydata:mainfrom
dcherian:optimize-encode-multiindex-check

Conversation

@dcherian

@dcheriandcherian commented Dec 21, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

Spotted when profiling some write workloads.

  1. Speeds up the check for multi-index
  2. Speeds up one string encoder by not re-creating variables when not necessary.

@benbovy is there a better way?

@benbovy

benbovy commented Dec 21, 2023

Copy link
Copy Markdown
Member

isinstance(var._data, xarray.core.indexing.PandasMultiIndexingAdapter) should work.

@dcherian
dcherian marked this pull request as ready for review December 21, 2023 18:06
@dcheriandcherian changed the title Faster ensure_not_multiindexFaster encoding functions.Dec 21, 2023
kmuehlbauer
kmuehlbauer approved these changes Jan 4, 2024
View reviewed changes

@kmuehlbauerkmuehlbauer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coding part LGTM, 👍

@dcherian
dcherian merged commit 5f1f78f into pydata:mainJan 4, 2024
@dcherian
dcherian deleted the optimize-encode-multiindex-check branch January 4, 2024 14:25
dcherian added a commit to dcherian/xarray that referenced this pull request Jan 4, 2024
@dcherian
Merge remote-tracking branch 'upstream/main' into init-zarr
1498c35
* upstream/main:
Faster encoding functions. (pydata#8565)
ENH: vendor SerializableLock from dask and use as default backend lock, adapt tests (pydata#8571)
Silence a bunch of CachingFileManager warnings (pydata#8584)
Bump actions/download-artifact from 3 to 4 (pydata#8556)
Minimize duplication in `map_blocks` task graph (pydata#8412)
[pre-commit.ci] pre-commit autoupdate (pydata#8578)
ignore a `DeprecationWarning` emitted by `seaborn` (pydata#8576)
Fix mypy type ignore (pydata#8564)
Support for the new compression arguments. (pydata#7551)
FIX: reverse index output of bottleneck move_argmax/move_argmin functions (pydata#8552)
@benbovybenbovy mentioned this pull request Jan 26, 2024
Closed
5 tasks
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kmuehlbauerkmuehlbauerkmuehlbauer approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dcherian@benbovy@kmuehlbauer