Skip to content

[Paddle] Add control of RNG state - #410

Merged
timmoon10 merged 3 commits into
NVIDIA:mainfrom
Tom-Zheng:gh_add_seed_control
Sep 1, 2023
Merged

[Paddle] Add control of RNG state#410
timmoon10 merged 3 commits into
NVIDIA:mainfrom
Tom-Zheng:gh_add_seed_control

Conversation

@Tom-Zheng

@Tom-ZhengTom-Zheng commented Aug 30, 2023•
edited
Loading

Copy link
Copy Markdown
Contributor

This PR allows user to control the RNG state of attention dropout and hidden dropout. Referring to PaddleNLP, attention dropout uses local_seed and hidden dropout uses global_seed.

@Tom-Zheng
Add control of attention dropout and hidden dropout RNG state
4022a22
Signed-off-by: Tian Zheng (Engrg-Hardware 1) <tizheng@nvidia.com>
@Tom-Zheng

Tom-Zheng commented Aug 30, 2023

Copy link
Copy Markdown
ContributorAuthor

@Tom-ZhengTom-Zheng changed the title [Paddle] Add control of attention dropout and hidden dropout RNG state[Paddle] Add control of RNG state of attention dropout and hidden dropoutAug 30, 2023
@Tom-ZhengTom-Zheng changed the title [Paddle] Add control of RNG state of attention dropout and hidden dropout[Paddle] Add control of RNG stateAug 30, 2023
timmoon10
timmoon10 approved these changes Aug 31, 2023
View reviewed changes

@timmoon10timmoon10 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@timmoon10

timmoon10 commented Aug 31, 2023

Copy link
Copy Markdown
Member

/te-ci

@Wong4jWong4j mentioned this pull request Sep 1, 2023
Merged
@Tom-Zheng
Fix CI error
32de57e
Signed-off-by: Tian Zheng (Engrg-Hardware 1) <tizheng@nvidia.com>
@Tom-Zheng

Tom-Zheng commented Sep 1, 2023

Copy link
Copy Markdown
ContributorAuthor

@zlsh80826 Would you please help trigger CI?

@zlsh80826

zlsh80826 commented Sep 1, 2023

Copy link
Copy Markdown
Collaborator

/te-ci

@timmoon10
timmoon10 merged commit 805b987 into NVIDIA:mainSep 1, 2023
@Tom-ZhengTom-Zheng mentioned this pull request Sep 3, 2023
Merged
RuiWang1998 pushed a commit to RuiWang1998/TransformerEngine that referenced this pull request Sep 11, 2023
@Tom-Zheng@RuiWang1998
[Paddle] Add control of RNG state (NVIDIA#410)
1aea467
* Add control of attention dropout and hidden dropout RNG state
Signed-off-by: Tian Zheng (Engrg-Hardware 1) <tizheng@nvidia.com>
* Fix CI error
Signed-off-by: Tian Zheng (Engrg-Hardware 1) <tizheng@nvidia.com>
---------
Signed-off-by: Tian Zheng (Engrg-Hardware 1) <tizheng@nvidia.com>
Signed-off-by: Rui Wang <rui@helixon.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@timmoon10timmoon10timmoon10 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Tom-Zheng@timmoon10@zlsh80826