Skip to content

Update polars-pf to 1.1.60 - #107

Merged
kevindetry-milaboratories merged 4 commits into
mainfrom
kevindetry/MILAB-6925
Sep 14, 2026
Merged

kevindetry-milaboratories merged 4 commits into
mainfrom
kevindetry/MILAB-6925

Conversation

@kevindetry-milaboratories

@kevindetry-milaboratories kevindetry-milaboratories commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Bump polars-pf in the base python-3.12.10 runenv from 1.1.56 to 1.1.60.

Why

pframes-rs 1.1.59 folds same-axis n-ary joins as a balanced tree, so tables with hundreds of per-sample columns no longer overflow the Tokio worker stack (MILAB-6925, pframes-rs#296). pframes-rs 1.1.60 adds the outer join fix for primary-less axis groups (MILAB-6852, pframes-rs#293).

ptabler pins polars-pf in its requirements.txt and installs it offline from the wheels this runenv bundles. The runenv must ship the new wheel before the SDK can move its pin. The SDK side is platforma#1824; its ptabler workflow tests fail until this runenv is released and the SDK catalog points at the new version.

Changes

  • python-3.12.10/config.json: polars-pf==1.1.60. It is the only variant that pins this package. 1.1.60 publishes the same five wheel tags as 1.1.56, so every target platform still resolves.
  • .changeset/balanced-join-fold.md: patch release for @platforma-open/milaboratories.runenv-python-3.12.10.
  • checker/whitelists/linux-x64.json: add cupyx._cusolver and cupyx._cutensor under the cupy_cuda12x-*.whl pattern. Unrelated to polars-pf, but it blocked CI on this branch. See below.

CI note: cupy 14.2.0

The first run of this PR failed the linux-x64 python-3.12.10 job in the native import checker. The polars_pf wheel passed. The failing wheel was cupy_cuda12x-14.2.0, with the expected missing CUDA libraries:

cupyx._cusolver: ImportError: libcusolver.so.11: cannot open shared object file
cupyx._cutensor: ImportError: libcutensor.so.2: cannot open shared object file

The whitelist suppressed these under the old module names cupyx.cusolver and cupyx.cutensor. cupy 14.2.0 renamed them with a leading underscore. cupy is not pinned in the base variant. It comes through cudf-cu12==25.6.0, which requires cupy-cuda12x>=12.0.0 with no upper bound. The last green main build on 2026-09-10 still resolved 14.1.1. The whitelist keeps the old names for older cupy wheels and adds the new ones.

CI note, second run: torch, triton, faiss

The second run failed two other linux-x64 jobs, python-3.12.10-torch-cuda and python-3.10.21-clustcr, again in the native import checker and again unrelated to polars-pf.

  • torch-cuda: ablang2==0.2.1 and sceptr==1.2.0 are downloaded in separate pip download calls with only lower bounds on torch, so they pull the newest torch and triton from the indexes. Today that is torch-2.14.0+cu126 and triton-3.8.0; the last green main build got 2.12.1 and 3.7.1. The torch whitelist keys were version-specific, so each drift breaks CI. The new triton ships lib/cupti-blackwell/*.so; a directory name with a hyphen can never be a module, so the checker's import attempt was a SyntaxError by construction.
  • clustcr: faiss-cpu==1.8.0.post1 is pinned, but faiss._swigfaiss_avx512 imports only on a runner whose CPU has AVX-512. The green build ran on runner 1000263597, this one on 1000264688. faiss picks the instruction set at runtime, so the failure is expected on such hardware.

Fixes in this PR:

  • checker/check_native_imports.py: skip module paths with a segment that is not a valid Python identifier. Those files are plain shared libraries, not extensions.
  • checker/whitelists/linux-x64.json: wildcard entries torch-*+cu126-... and torch-*+cpu-... with the same expected errors as the existing per-version keys, and faiss_cpu-*.whl for the AVX-512 module. Existing keys are kept.

Greptile Summary

Updates the base Python 3.12.10 runtime to bundle polars-pf 1.1.60 and extends Linux x64 native-import policy for module names introduced by cupy 14.2.0.

  • Pins polars-pf==1.1.60, bringing balanced same-axis join folding and the primary-less outer-join fix.
  • Adds narrowly matched exceptions for the renamed cupyx._cusolver and cupyx._cutensor native modules.
  • Declares a patch release for the affected runtime package.

Important touched terms

  • polars-pf — The PFrames-enabled Polars package bundled in the runtime. Its pin changes from 1.1.56 at the PR base (1.1.59 at the previous review) to 1.1.60.
  • PFrames — The dataframe execution functionality supplied through polars-pf. This update includes balanced join folding for very wide same-axis joins and an outer-join correction.
  • Native-import checker whitelist — Platform-specific policy that approves known import failures only when wheel, module, and error text match. Linux x64 policy now recognizes the underscored cupy module names.
  • Changeset — Release metadata consumed by Changesets to version workspace packages. The new entry requests a patch release of the Python 3.12.10 runtime.

Confidence Score: 5/5

The PR appears safe to merge, with the dependency pin, platform artifact coverage, whitelist entries, and release declaration aligned.

No actionable new defects or outstanding previous findings remain; the new whitelist entries narrowly match the reported cupy module failures, and the runtime release metadata consistently identifies polars-pf 1.1.60.

Important Files Changed

Filename Overview
python-3.12.10/config.json Updates the global polars-pf dependency pin to 1.1.60 for the five supported runtime targets.
checker/whitelists/linux-x64.json Adds exact module and missing-library matches for two renamed cupy 14.2.0 native modules.
.changeset/balanced-join-fold.md Declares the corresponding patch release and documents the wide-join stack-overflow improvement.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Pin[Pin polars-pf 1.1.60] --> Build[Build Python 3.12.10 runtime]
    Build --> Wheels[Bundle platform wheels]
    Wheels --> Check[Run native-import validation]
    Policy[Linux x64 cupy exceptions] --> Check
    Check --> Release[Publish patch runtime release]
Loading

Reviews (2): Last reviewed commit: "Update polars-pf to 1.1.60" | Re-trigger Greptile

Context used (4)

pframes-rs 1.1.59 folds same-axis n-ary joins as a balanced tree, so
wide tables no longer overflow the Tokio worker stack (MILAB-6925).
ptabler pins polars-pf in its requirements and installs it offline
from the wheels this runenv bundles, so the runenv must ship the new
wheel before the SDK can move its pin.
cupy-cuda12x 14.2.0 moved cupyx.cusolver and cupyx.cutensor to
cupyx._cusolver and cupyx._cutensor. The linux-x64 native import
checker whitelist only listed the old names, so the python-3.12.10
linux-x64 build failed on the two missing CUDA libraries once pip
resolved cupy 14.2.0 through cudf-cu12. Keep the old names for
older cupy wheels and add the new ones.
pframes-rs 1.1.60 adds the outer join fix for primary-less axis groups
(MILAB-6852) on top of the balanced join fold.
@kevindetry-milaboratories kevindetry-milaboratories changed the title Update polars-pf to 1.1.59 Update polars-pf to 1.1.60 Sep 14, 2026
@kevindetry-milaboratories

Copy link
Copy Markdown
Contributor Author

@greptileai

Two linux-x64 jobs failed on wheels that are not pinned or that depend
on the runner hardware:

- ablang2 and sceptr are downloaded in separate pip calls with only
  lower bounds on torch, so they pull the newest torch and triton. The
  torch whitelist keys were version-specific and broke on 2.14.0+cu126.
  Add wildcard keys for the cu126 and cpu torch wheels.
- triton 3.8.0 ships lib/cupti-blackwell/*.so. A path segment with a
  hyphen can never be a module, so the import attempt was a
  SyntaxError by construction. Skip module paths whose segments are
  not valid identifiers.
- faiss._swigfaiss_avx512 imports only on a CPU with AVX-512, and the
  ubuntu-large-amd64 pool mixes CPUs. Whitelist it.
@kevindetry-milaboratories
kevindetry-milaboratories merged commit 00a4080 into main Sep 14, 2026
67 checks passed
@kevindetry-milaboratories
kevindetry-milaboratories deleted the kevindetry/MILAB-6925 branch September 14, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants