Skip to content

deterministic cpu association - #2

Open
hsian-coding wants to merge 7 commits into
IES-ESLab:masterfrom
hsian-coding:master
Open

deterministic cpu association#2
hsian-coding wants to merge 7 commits into
IES-ESLab:masterfrom
hsian-coding:master

Conversation

@hsian-coding

Copy link
Copy Markdown

This PR makes GaMMA’s multi-process association CPU-bounded and reproducible. It clamps worker counts to available CPUs and clusters, runs DBSCAN single-threaded for stable labels, applies deterministic per-cluster seeds, and merges results in a fixed order so event IDs and pick assignments remain consistent across worker counts.

It also limits native math libraries to one thread per association worker to prevent CPU oversubscription.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make GaMMA’s multi-process association reproducible and CPU-bounded by enforcing deterministic clustering/association behavior and preventing CPU oversubscription from native threadpools.

Changes:

  • Adds deterministic controls in association: DBSCAN forced single-threaded, deterministic per-cluster seeding, and fixed-order merging of results.
  • Clamps association worker count to available CPUs (respecting CPU affinity) and to the number of clusters.
  • Adds threadpoolctl dependency and initializes per-worker native threadpool limits to 1 thread.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
setup.py Adds threadpoolctl dependency and reformats requirements list.
gamma/utils.py Implements deterministic clustering/association behavior, CPU/cluster worker clamping, and per-worker native threadpool limiting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread gamma/utils.py Outdated
Comment thread gamma/utils.py Outdated
Comment thread gamma/utils.py
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread gamma/utils.py Outdated

@hsian-coding hsian-coding left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@hsian-coding

Copy link
Copy Markdown
Author

Addressed the review suggestions and resolved the conversations. The PR is ready for re-review.

@Pamicoding Pamicoding left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @hsian-coding bro, I think I should submit review in formal way that can let AI to know there are still have some issues need to solve. Issues include three comments we've closed (R252-R269, R273, R278-282), since close the comments did not solve it directly, I might still need another push from you to solve this, many thanks bro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants