Uh oh!
There was an error while loading. Please reload this page.
Fix layernorm in GQA - #434
Merged
Merged
Conversation
fixesNVIDIA#392 Signed-off-by: Markus Schnoes <markus.schnoes@gmx.de>
…r to support splitting with different sizes as required by unfused GQA. Signed-off-by: Markus Schnoes <markus.schnoes@gmx.de>
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
cyanguwa
marked this pull request as ready for review
September 15, 2023 00:22
cyanguwa
commented
Sep 15, 2023
CollaboratorAuthor
Pipeline 9820505 |
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
CollaboratorAuthor
Pipeline 9827724 is green. |
timmoon10
requested changes
Sep 22, 2023
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com>
cyanguwa
commented
Sep 22, 2023
CollaboratorAuthor
Pipeline 9936670 |
ptrendx pushed a commit
that referenced
this pull request
Oct 2, 2023
* [PyTorch] Implement GQA based on fused q, k, v projection. Additionally fixes#392 Signed-off-by: Markus Schnoes <markus.schnoes@gmx.de> * [PyTorch] Extend parameters_split option in Linear and LayerNormLinear to support splitting with different sizes as required by unfused GQA. Signed-off-by: Markus Schnoes <markus.schnoes@gmx.de> * fix parameters split Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix noop cat to bypass torch.cat and support uneven split Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix unit tests Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix torch.split args Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix cuda graph due to noop_cat Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix lint Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * remove the use of enumerate when possible Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> * fix strides in SplitAlongDim Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> --------- Signed-off-by: Markus Schnoes <markus.schnoes@gmx.de> Signed-off-by: Charlene Yang <8636796+cyanguwa@users.noreply.github.com> Co-authored-by: Markus Schnoes <markus.schnoes@gmx.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is based on #394 and has a few more changes in order to complete the functionality and pass the CI tests.