[Fix] Only add processes which exist in process_dict by jan-janssen · Pull Request #913 · pyiron/executorlib

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssen · 2026-02-14T11:18:05Z

Summary by CodeRabbit

Refactor
- Improved task scheduling efficiency through optimized resource field propagation.
- Enhanced queue management for task processing.
- Streamlined task dependency handling with refined filtering.
- Consolidated task resource parameter extraction for consistency.

for more information, see https://pre-commit.ci

coderabbitai · 2026-02-14T11:18:22Z

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Walkthrough

Refactored execute_tasks_h5 function to consolidate resource dictionary management through a new _get_task_input helper. Introduced queue ID extraction via get_queue_id(file_name) and simplified file naming logic while restricting dependency resolution to existing process dict keys.

Changes

Cohort / File(s)	Summary
Task Execution Refactoring `src/executorlib/task_scheduler/file/shared.py`	Added `_get_task_input` helper to merge resource dict keys and extract cache metadata. Integrated `get_queue_id` for queue-based task handling. Refactored file naming logic and constrained future dependency inclusion to pre-existing process dict entries.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related PRs

Use a more flexible cache directory #707: Modifies execute_tasks_h5 to source per-task cache_directory from task_resource_dict; main PR extends this pattern with resource key merging via the new _get_task_input helper.
Move cleanup of previous jobs into execute function #732: Related through get_queue_id function usage—main PR calls this function while the retrieved PR modifies its implementation in hdf.py.

Suggested reviewers

liamhuber

Poem

🐰 A helper born to merge and glean,
Where cache keys rest and queues convene,
Task dependencies grow lean and true,
File names dance in queue's soft hue,
Executors hop through paths anew!

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch file_wait_fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

codecov · 2026-02-14T11:23:08Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.68%. Comparing base (eb420fd) to head (1c1e058).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@ Coverage Diff @@## main #913 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.01% 
==========================================
Files 38 38 Lines 1911 1917 +6 ==========================================
+ Hits 1790 1796 +6 
Misses 121 121

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

for more information, see https://pre-commit.ci

jan-janssenand others added 2 commits February 14, 2026 12:17

[Fix] Only add processes which exist in process_dict
5183a3d

[pre-commit.ci] auto fixes from pre-commit.com hooks
5d142e5
for more information, see https://pre-commit.ci

jan-janssen marked this pull request as draft February 14, 2026 11:18

jan-janssenand others added 2 commits February 14, 2026 12:27

reduce number of statements
0bc8c96

[pre-commit.ci] auto fixes from pre-commit.com hooks
1c1e058
for more information, see https://pre-commit.ci

jan-janssen marked this pull request as ready for review February 14, 2026 14:06

jan-janssen merged commit f3af366 into mainFeb 14, 2026
35 checks passed

jan-janssen deleted the file_wait_fix branch February 14, 2026 14:06

jan-janssen mentioned this pull request Feb 14, 2026
feat: API with scheduler support glasagent/amorphouspy#121
Merged