Skip to content

[Merge only onto large-scale] Large scale Kmeans changes. - #174

Merged
KateBlueSky merged 36 commits into
large-scalefrom
dev/large_scale_kmeans
Mar 22, 2025
Merged

[Merge only onto large-scale] Large scale Kmeans changes. #174
KateBlueSky merged 36 commits into
large-scalefrom
dev/large_scale_kmeans

Conversation

@KateBlueSky

@KateBlueSkyKateBlueSky commented Mar 17, 2025

Copy link
Copy Markdown

Added KMeans for large-scale data branch.

Added configs and changes for KMeans large data.


PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

@KateBlueSky
KateBlueSky marked this pull request as ready for review March 18, 2025 23:52
@KateBlueSkyKateBlueSky changed the title Dev/large scale kmeans[Merge only onto large-scale] Large scale Kmeans changes. Mar 18, 2025
Comment threadconfigs/spmd/large_scale/large_scale.json Outdated
Comment threadconfigs/spmd/large_scale/kmeans_wide_weak.json
Comment threadconfigs/spmd/large_scale/kmeans_strong.json Outdated
Comment threadconfigs/spmd/large_scale/kmeans_narrow_weak.json Outdated
Comment threadconfigs/spmd/large_scale/kmeans_wide_weak.json Outdated
Comment threadsklbench/datasets/transformer.py
Comment threadsklbench/utils/common.py
Comment threadsklbench/datasets/transformer.py Outdated
Comment threadsklbench/datasets/transformer.py Outdated
Comment threadconfigs/spmd/large_scale/kmeans_wide_weak.json Outdated
Comment threadconfigs/spmd/large_scale/kmeans_narrow_weak.json Outdated
Comment threadsklbench/datasets/transformer.py Outdated
Comment threadconfigs/README.md Outdated
Comment threadsklbench/datasets/transformer.py Outdated
Comment threadconfigs/README.md Outdated
Comment threadsklbench/datasets/transformer.py
@KateBlueSky
KateBlueSky merged commit 2edb597 into large-scaleMar 22, 2025
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@KateBlueSky@ethanglaser