Conversation
eliminate_common_subexpression merged the structurally-identical per-layer INT4 weight-dequant tiles (~531MB each) across all layers, making their outputs share one producer and stay co-live -> memory_coloring scratch scaled O(num_layers) (~0.86GB/layer) -> GPU OOM on deep models (52-layer Onyx/Llama-4). Add a 64MB single-output size guard so CSE still de-dups small common ops (needed by the embedding-gather path) but leaves large weight-dequants independent and reusable. Full 52-layer Onyx INT4 now compiles + runs on gfx1151 (was OOM): 67GB mxr cached, exit 0, 5.4 tok/s. (Output correctness at full depth tracked separately.) Co-Authored-By: Claude <noreply@anthropic.com> (cherry picked from commit 97ae9c1a3329ed67d76cb6c83caccda236925e1e)
Collaborator
|
I dont think we should do this here. If this is caused by long liveness then we could analyze that in another pass and then duplicate calculation before scheduling and memory coloring. |
rlegithub
marked this pull request as ready for review
September 14, 2026 19:26
pfultz2
requested changes
Sep 14, 2026
| if(not s.sub_shapes().empty()) | ||
| return false; | ||
| return s.bytes() > threshold; | ||
| } |
Collaborator
There was a problem hiding this comment.
This can massively increase memory usage and calculations especially since this targets large outputs.
This should be done as another pass to check the liveness benefit against the cost of recalculating.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Common-subexpression elimination that merges very large (>64 MB) single-output ops can drive excessive peak memory on deep quantized LLMs (observed OOM on int4 models). This guards CSE to skip merging such large single-output ops.
Test plan