[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023

Closed
jan-janssen wants to merge 3 commits into
mainfrom
fix/init-function-mpi-parallel-block-allocation
Closed

[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom
fix/init-function-mpi-parallel-block-allocation

Conversation

@jan-janssen

@jan-janssenjan-janssen commented Jun 19, 2026•
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Member

Summary

  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.

Test plan

  • Existing test test_internal_memory (cores=1) continues to pass
  • New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  • Full test suite green

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Bug Fixes

    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.

  • Tests

    • Added test coverage for memory allocation behavior in parallel execution environments.

@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Jun 19, 2026•
edited
Loading

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094

📥 Commits

Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.

📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough

Walkthrough

The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.

Changes

MPI Init Error Propagation

Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.

Poem

🐇 Across the MPI ranks I hop,
Collecting errors, none shall drop.
Rank zero picks the first mistake,
And sends it back for the caller's sake.
No init error slips away—
The rabbit gathers them all today! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecovBot commented Jun 19, 2026•
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.

Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jan-janssenand others added 2 commits June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen marked this pull request as draft June 19, 2026 05:23
@jan-janssen
jan-janssen deleted the fix/init-function-mpi-parallel-block-allocation branch June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen
, 'i'); if (__m === '*' || __re.test(location.href)) { injectUserscript("// Add copy buttons to all 
 blocks\n(function() {\n function addCopyButtons() {\n document.querySelectorAll('pre code').forEach(function(codeBlock) {\n if (codeBlock.parentElement.hasAttribute('data-copy-added')) return;\n codeBlock.parentElement.setAttribute('data-copy-added', 'true');\n \n var btn = document.createElement('button');\n btn.textContent = 'Copy';\n btn.style.cssText = 'position:absolute;top:4px;right:4px;padding:2px 8px;font-size:11px;background:#4ecdc4;border:none;border-radius:4px;color:#1a1a2e;cursor:pointer;opacity:0.7;transition:opacity 0.2s;';\n btn.onmouseover = function() { this.style.opacity = '1'; };\n btn.onmouseout = function() { this.style.opacity = '0.7'; };\n btn.onclick = function() {\n navigator.clipboard.writeText(codeBlock.textContent).then(function() {\n btn.textContent = 'Copied!';\n setTimeout(function() { btn.textContent = 'Copy'; }, 1500);\n });\n };\n codeBlock.parentElement.style.position = 'relative';\n codeBlock.parentElement.appendChild(btn);\n });\n }\n \n addCopyButtons();\n \n // Re-run on dynamic content\n var observer = new MutationObserver(addCopyButtons);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Add Copy Buttons to Code Blocks");
}
} catch(__e) { console.warn('[Userscript:Add Copy Buttons to Code Blocks]', __e); }
})();
(function(){
try {
var __m = "github.com";
var __re = new RegExp('^' + "github\\.com" + '

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Force GitHub README to respect dark mode\n(function() {\n var style = document.createElement('style');\n style.textContent = '\n .markdown-body {\n color-scheme: dark light;\n }\n .markdown-body pre { background: #161b22 !important; }\n .markdown-body code { background: rgba(110, 118, 129, 0.4) !important; }\n .markdown-body table th, .markdown-body table td { border-color: #30363d !important; }\n .markdown-body img { background: #0d1117; }\n .markdown-body blockquote { border-left-color: #8b949e; }\n .markdown-body hr { border-color: #30363d; }\n ';\n document.head.appendChild(style);\n})();", "GitHub Dark Mode README Fix");
}
} catch(__e) { console.warn('[Userscript:GitHub Dark Mode README Fix]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Highlight search terms from Google/DuckDuckGo/Bing referrer\n(function() {\n var ref = document.referrer;\n var terms = [];\n \n if (ref.includes('google.com') || ref.includes('duckduckgo.com') || ref.includes('bing.com')) {\n var url = new URL(ref);\n var q = url.searchParams.get('q') || url.searchParams.get('p');\n if (q) {\n terms = q.split(/\\s+/).filter(function(t) { return t.length > 2; });\n }\n }\n \n if (terms.length === 0) return;\n \n var style = document.createElement('style');\n style.textContent = '.userscript-highlight { background: #fbbf24; color: #1a1a2e; padding: 1px 3px; border-radius: 2px; }';\n document.head.appendChild(style);\n \n function highlight(node) {\n if (node.nodeType === 3) { // text node\n var text = node.textContent;\n var found = false;\n terms.forEach(function(term) {\n var regex = new RegExp('(' + term.replace(/[.*+?^${}()|[\\]\\\\]/g, '\\\\') + ')', 'gi');\n if (regex.test(text)) {\n found = true;\n var frag = document.createDocumentFragment();\n var parts = text.split(regex);\n parts.forEach(function(part, i) {\n if (i % 2 === 0) {\n frag.appendChild(document.createTextNode(part));\n } else {\n var span = document.createElement('span');\n span.className = 'userscript-highlight';\n span.textContent = part;\n frag.appendChild(span);\n }\n });\n node.parentNode.replaceChild(frag, node);\n }\n });\n } else if (node.nodeType === 1 && node.childNodes) { // element\n var skipTags = ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXTAREA', 'INPUT', 'SELECT'];\n if (!skipTags.includes(node.tagName)) {\n Array.from(node.childNodes).forEach(highlight);\n }\n }\n }\n \n highlight(document.body);\n \n // Re-highlight on dynamic content\n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1 || node.nodeType === 3) highlight(node);\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Highlight Search Terms");
}
} catch(__e) { console.warn('[Userscript:Highlight Search Terms]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Strip utm_, fbclid, gclid, etc. from all links on page\n(function() {\n var trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content',\n 'fbclid', 'gclid', 'dclid', 'msclkid', 'yclid',\n 'ref', 'ref_src', 'source', 'medium', 'campaign'];\n \n function cleanUrl(url) {\n try {\n var u = new URL(url, window.location.origin);\n var changed = false;\n trackingParams.forEach(function(p) {\n if (u.searchParams.has(p)) {\n u.searchParams.delete(p);\n changed = true;\n }\n });\n return changed ? u.toString() : url;\n } catch (e) {\n return url;\n }\n }\n \n function cleanLinks() {\n document.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n \n cleanLinks();\n \n var observer = new MutationObserver(function(mutations) {\n mutations.forEach(function(m) {\n m.addedNodes.forEach(function(node) {\n if (node.nodeType === 1) {\n if (node.tagName === 'A') cleanLinks();\n node.querySelectorAll('a[href]').forEach(function(a) {\n var clean = cleanUrl(a.href);\n if (clean !== a.href) a.href = clean;\n });\n }\n });\n });\n });\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "Remove Tracking Parameters from Links");
}
} catch(__e) { console.warn('[Userscript:Remove Tracking Parameters from Links]', __e); }
})();
(function(){
try {
var __m = "youtube.com";
var __re = new RegExp('^' + "youtube\\.com" + '

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Auto-enable theater mode on YouTube\n(function() {\n function tryTheater() {\n var btn = document.querySelector('button[aria-label=\"Theater mode\"], ytd-player #player button[title=\"Theater mode\"]');\n if (btn && !btn.classList.contains('activated')) {\n btn.click();\n }\n }\n \n // Try immediately\n tryTheater();\n \n // Try after navigation (SPA)\n var lastUrl = location.href;\n setInterval(function() {\n if (location.href !== lastUrl) {\n lastUrl = location.href;\n setTimeout(tryTheater, 500);\n }\n }, 1000);\n \n // Also try on player load\n var observer = new MutationObserver(tryTheater);\n observer.observe(document.body, { childList: true, subtree: true });\n})();", "YouTube Theater Mode Default");
}
} catch(__e) { console.warn('[Userscript:YouTube Theater Mode Default]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Remove or un-stick sticky/fixed headers that block content\n(function() {\n function unstick() {\n document.querySelectorAll('header, nav, [role=\"banner\"], .header, .navbar, .sticky, .fixed-top, [style*=\"position: fixed\"], [style*=\"position:sticky\"]').forEach(function(el) {\n if (el.style.position === 'fixed' || el.style.position === 'sticky' || \n getComputedStyle(el).position === 'fixed' || getComputedStyle(el).position === 'sticky') {\n el.style.position = 'static';\n el.style.top = 'auto';\n el.style.zIndex = 'auto';\n }\n });\n }\n \n unstick();\n \n var observer = new MutationObserver(unstick);\n observer.observe(document.body, { childList: true, subtree: true, attributes: true, attributeFilter: ['style', 'class'] });\n})();", "Kill Sticky Headers");
}
} catch(__e) { console.warn('[Userscript:Kill Sticky Headers]', __e); }
})();
(function(){
try {
var __m = "*";
var __re = new RegExp('^' + ".*" + '

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen

, 'i');
if (__m === '*' || __re.test(location.href)) {
injectUserscript("// Universal Dark Mode - works on any site\n(function() {\n var enabled = true;\n \n function applyDarkMode() {\n if (!enabled) return;\n \n // Create style element if it doesn't exist\n var style = document.getElementById('universal-dark-mode-style');\n if (!style) {\n style = document.createElement('style');\n style.id = 'universal-dark-mode-style';\n document.head.appendChild(style);\n }\n \n // Dark mode CSS - inverts colors but preserves images/video\n style.textContent = '\n /* Invert everything except media */\n html {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #1a1a2e !important;\n }\n \n /* Restore images, videos, iframes, canvas */\n img, video, iframe, canvas, svg, picture, [style*=\"background-image\"] {\n filter: invert(1) hue-rotate(180deg) !important;\n }\n \n /* Preserve specific elements that should not be inverted */\n .no-dark-mode, .no-dark-mode *,\n [data-theme=\"light\"], [data-theme=\"light\"],\n .ace_editor, .ace_editor *,\n .CodeMirror, .CodeMirror *,\n .monaco-editor, .monaco-editor *,\n .markdown-body pre, .markdown-body pre *,\n .highlight, .highlight *,\n pre code, pre code * {\n filter: none !important;\n }\n \n /* Fix common UI elements */\n .modal, .popup, .dropdown-menu, .tooltip, .popover {\n filter: invert(1) hue-rotate(180deg) !important;\n background: #2d2d44 !important;\n border-color: #444 !important;\n }\n \n /* Scrollbars */\n ::-webkit-scrollbar { background: #1a1a2e !important; }\n ::-webkit-scrollbar-thumb { background: #444 !important; }\n ::-webkit-scrollbar-thumb:hover { background: #555 !important; }\n \n /* Selection */\n ::selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ::-moz-selection { background: #4ecdc4 !important; color: #1a1a2e !important; }\n ';\n }\n \n function removeDarkMode() {\n var style = document.getElementById('universal-dark-mode-style');\n if (style) style.remove();\n }\n \n // Toggle with Alt+Shift+D\n document.addEventListener('keydown', function(e) {\n if (e.altKey && e.shiftKey && e.key === 'D') {\n e.preventDefault();\n enabled = !enabled;\n if (enabled) {\n applyDarkMode();\n console.log('[Universal Dark Mode] Enabled');\n } else {\n removeDarkMode();\n console.log('[Universal Dark Mode] Disabled');\n }\n }\n });\n \n // Apply on load\n applyDarkMode();\n \n // Re-apply on dynamic content\n var observer = new MutationObserver(function(mutations) {\n if (enabled && !document.getElementById('universal-dark-mode-style')) {\n applyDarkMode();\n }\n });\n observer.observe(document.head, { childList: true });\n \n console.log('[Universal Dark Mode] Loaded - Press Alt+Shift+D to toggle');\n})();", "Universal Dark Mode");
}
} catch(__e) { console.warn('[Userscript:Universal Dark Mode]', __e); }
})();
})();

Skip to content
[Fix] propagate init_function errors from all MPI ranks in block allocation mode - #1023
Closed
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Closed
[Fix] propagate init_function errors from all MPI ranks in block allocation mode#1023
jan-janssen wants to merge 3 commits into
mainfrom 
fix/init-function-mpi-parallel-block-allocation
Conversation
@jan-janssen
@jan-janssenjan-janssen
commented
Jun 19, 2026•

edited by coderabbitai
Bot
Loading

Copy link


Copy Markdown

Member
Summary
  • In interactive_parallel.py, the init_function branch only reported errors from MPI rank 0; failures on non-zero ranks were silently swallowed, leaving those ranks with uninitialised memory. Subsequent function calls on the affected ranks would then receive wrong or missing kwargs injected from memory.
  • The fix gathers errors from all ranks via MPI.COMM_WORLD.gather before rank 0 sends the success/error response to the scheduler — mirroring how function-execution results are already gathered. This also acts as an implicit barrier so the scheduler cannot dispatch the next task until every rank has finished initialising.
  • Adds test_internal_memory_mpi to cover block_allocation + cores=2 + init_function, a combination that had zero test coverage.
Test plan
  •  Existing test test_internal_memory (cores=1) continues to pass
  •  New test test_internal_memory_mpi (cores=2) passes and verifies that both MPI ranks receive the memory value set by the init function
  •  Full test suite green
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
  • Bug Fixes
    • Enhanced error handling during initialization to capture errors across distributed execution ranks and provide clearer error reporting.
  • Tests
    • Added test coverage for memory allocation behavior in parallel execution environments.
@jan-janssen@claude
[Fix] propagate init_function errors from all MPI ranks in block allo…
b173e16
…cation mode
In interactive_parallel.py the init branch was only propagating errors
from rank 0; failures on non-zero ranks were silently swallowed, leaving
those ranks with uninitialised memory. Subsequent function calls on the
affected ranks would then receive wrong or missing kwargs.
The fix mirrors the existing function-execution path: after each rank
runs call_funct for init, all errors are gathered to rank 0 via
MPI.COMM_WORLD.gather before the success/error response is sent back to
the scheduler. This also acts as an implicit barrier so the scheduler
cannot dispatch the next task until every rank has finished init.
Adds a test (test_internal_memory_mpi) that exercises block allocation
with cores=2 and an init_function – a combination that had zero coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
coderabbitaiBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Contributor
Review Change Stack
Warning
Review limit reached
@jan-janssen, we couldn't start this review because you've reached your PR review rate limit.
More reviews will be available in 25 minutes and 10 seconds. Learn how PR review limits work.
Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).
⌛ How to resolve this issue?
After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.
To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.
🚦 How do rate limits work?
CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.
For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.
Please see our Fair Usage Limits Policy for further information.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 911eeadf-d364-4fb7-bc73-75d21bef6094
📥 Commits
Reviewing files that changed from the base of the PR and between b173e16 and 7f56240.
📒 Files selected for processing (1)
  • src/executorlib/backend/interactive_parallel.py
📝 Walkthrough
Walkthrough
The "init" handling in interactive_parallel.py is refactored to catch exceptions into init_error, gather that variable across all MPI ranks, and have rank zero select and forward the first non-None error via ZMQ. A new unit test (test_internal_memory_mpi) validates init_function memory sharing with cores=2 using mpi4py.
Changes
MPI Init Error Propagation
Layer / File(s)Summary
Gather init errors across MPI ranks and report on rank zero
src/executorlib/backend/interactive_parallel.py, tests/unit/standalone/interactive/test_spawner.py		The "init" path now catches exceptions into init_error, gathers it across ranks (or wraps in a list for single-rank runs), and rank zero picks the first non-None error to send via ZMQ and write the error file; the new test_internal_memory_mpi test, gated on mpi4py, verifies init_function with cores=2 returns two matching NumPy arrays.
Estimated code review effort
🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
  • pyiron/executorlib#804: Introduced the "fail safe init function" logic in the same "init" handling path of interactive_parallel.py that this PR extends with cross-rank error gathering.
Poem
🐇 Across the MPI ranks I hop,

Collecting errors, none shall drop.

Rank zero picks the first mistake,

And sends it back for the caller's sake.

No init error slips away—

The rabbit gathers them all today! 🎉
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check nameStatusExplanationResolution
Docstring Coverage⚠️ WarningDocstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check nameStatusExplanation
Description Check✅ PassedCheck skipped - CodeRabbit’s high-level summary is enabled.
Title check✅ PassedThe title directly and specifically describes the main change: propagating init_function errors from all MPI ranks in block allocation mode, which matches the core objective of the PR.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  •  Create PR with unit tests
  •  Commit unit tests in branch fix/init-function-mpi-parallel-block-allocation
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
❤️ Share
Comment @coderabbitai help to get the list of available commands and usage tips.
@codecov
codecovBot
commented
Jun 19, 2026•

edited

Loading

Copy link


Copy Markdown

Codecov Report
❌ Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

✅ Project coverage is 93.84%. Comparing base (640e440) to head (7f56240).
⚠️ Report is 1 commits behind head on main.
Files with missing linesPatch %Lines
src/executorlib/backend/interactive_parallel.py0.00%17 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@## main #1023 +/- ##
==========================================
- Coverage 94.24% 93.84% -0.40% 
==========================================
Files 39 39 Lines 2119 2128 +9 ==========================================
Hits 1997 1997 - Misses 122 131 +9 
☔ View full report in Codecov by Harness.

📢 Have feedback on the report? Share it here.
 🚀 New features to boost your workflow: 
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
jan-janssenand others
added 2 commits
June 19, 2026 07:01
@jan-janssen@claude
[Fix] extract _execute_init_dict to satisfy PLR0915 (too many stateme…
efe81db
…nts)
Moves the init-function handling out of main() into a private helper so
the statement count stays within the ruff/pylint PLR0915 limit of 50.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen@claude
[Docs] add docstring to _execute_init_dict
7f56240
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jan-janssen
jan-janssen
marked this pull request as draft
June 19, 2026 05:23
@jan-janssen
jan-janssen
deleted the

fix/init-function-mpi-parallel-block-allocation

branch
June 19, 2026 08:25
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment

Reviewers


No reviews


Assignees


No one assigned


Labels


None yet


Projects


None yet


Milestone


No milestone


Development

Successfully merging this pull request may close these issues.

1 participant

@jan-janssen