Skip to content

Flux - Add FluxExecutor using native Flux Python API bindings - #1758

Open
jlnav wants to merge 27 commits into
feature/flux_mpi_runnerfrom
feature/flux_executor_module
Open

Flux - Add FluxExecutor using native Flux Python API bindings#1758
jlnav wants to merge 27 commits into
feature/flux_mpi_runnerfrom
feature/flux_executor_module

Conversation

@jlnav

@jlnavjlnav commented Jul 8, 2026

Copy link
Copy Markdown
Member

FluxExecutor submits jobs directly to Flux via its Python bindings rather than wrapping flux run as a subprocess. It is useful when running inside containers where standard MPI runners are unavailable.

  • New libensemble/executors/flux_executor.py with FluxExecutor and FluxTask
  • Conditionally import FluxExecutor in executors/init.py (ImportError if flux-core bindings are not installed)
  • Append FluxExecutor unit tests to test_flux.py; tests skip automatically when flux-core Python bindings are not present

jlnavand others added 16 commits May 26, 2026 10:01
…-manager *or* the gen running on the first worker. adjust many tests to use VOCS instead of ub/lb bounds
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.47.2 to 1.48.0.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](crate-ci/typos@v1.47.2...v1.48.0)
---
updated-dependencies:
- dependency-name: crate-ci/typos
dependency-version: 1.48.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Updates the requirements on [setuptools](https://github.com/pypa/setuptools) to permit the latest version.
Updates `setuptools` to 83.0.0
- [Release notes](https://github.com/pypa/setuptools/releases)
- [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst)
- [Commits](pypa/setuptools@v75.1.0...v83.0.0)
---
updated-dependencies:
- dependency-name: setuptools
dependency-version: 83.0.0
dependency-type: direct:development
dependency-group: python-updates
...
Signed-off-by: dependabot[bot] <support@github.com>
FluxExecutor submits jobs directly to Flux via its Python bindings
rather than wrapping flux run as a subprocess. It is useful when
running inside containers where standard MPI runners are unavailable.
- New libensemble/executors/flux_executor.py with FluxExecutor and FluxTask
- Conditionally import FluxExecutor in executors/__init__.py (graceful
ImportError if flux-core bindings are not installed)
- Append FluxExecutor unit tests to test_flux.py; tests skip automatically
when flux-core Python bindings are not present
…evelop/crate-ci/typos-1.48.0
Bump crate-ci/typos from 1.47.2 to 1.48.0
…on-updates-fec1d00dd4
Update setuptools requirement from <83,>=75.1.0 to >=75.1.0,<84 in the python-updates group
@jlnav
jlnav marked this pull request as ready for review July 10, 2026 14:00
@codecov

codecovBot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.53061% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.93%. Comparing base (826eb9b) to head (0f92d54).

Files with missing linesPatch %Lines
libensemble/executors/flux_executor.py86.95%20 Missing and 10 partials ⚠️
libensemble/specs.py80.00%0 Missing and 3 partials ⚠️
Additional details and impacted files
@@ Coverage Diff @@## feature/flux_mpi_runner #1758 +/- ##
============================================================
+ Coverage 77.20% 89.93% +12.72% 
============================================================
Files 79 80 +1 Lines 8398 8643 +245 Branches 1287 1335 +48 ============================================================
+ Hits 6484 7773 +1289 + Misses 1666 659 -1007 + Partials 248 211 -37 

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jlnavand others added 11 commits July 10, 2026 11:57
slim down some of the TESTSUITE_NPROCS cases (for speed)
remove ax gen_f . ensure ax is up-to-date in all extra jobs for the optimas tests
set gen_specs.user.lb and ub from vocs variables bounds
…ssages about FluxExecutor's internal FluxState not being thread-safe, so process-based only. set more expected attributes upon Flux results. use Flux's submit_async instead for non-blocking. don't wait too long to kill a flux job. Specify flux uri optionally to FluxExecutor init in case the instance isn't obvious. passthrough task.stdout and task.stderr to JobSpecV1. test adjustments
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@jlnav