Skip to content

Adds Distconv support for Layer Normalization layer - #2229

Open
szaman19 wants to merge 7 commits into
LBANN:v1.x-developfrom
szaman19:distconv-layer-norm
Open

Adds Distconv support for Layer Normalization layer#2229
szaman19 wants to merge 7 commits into
LBANN:v1.x-developfrom
szaman19:distconv-layer-norm

Conversation

@szaman19

@szaman19szaman19 commented Mar 15, 2023

Copy link
Copy Markdown
Collaborator

Adds distconv support for layer normalization

  • Add CI test
  • Implement layer_norm_distconv_adapter
  • Move layer norm kernels to shared .cuh file
  • Use existing layer norm kernels to implement distconv layer norm operator

@szaman19
szaman19force-pushed the distconv-layer-norm branch from 69269a7 to a4f367dCompareMarch 17, 2023 19:56
@szaman19
szaman19 marked this pull request as ready for review March 18, 2023 08:31
@szaman19
szaman19force-pushed the distconv-layer-norm branch from 0a7ffbd to d59e2dbCompareApril 20, 2023 07:43
@szaman19
szaman19force-pushed the distconv-layer-norm branch from 5b133d0 to 13aff25CompareMay 24, 2023 02:58
@szaman19

Copy link
Copy Markdown
CollaboratorAuthor

Tagging @tbennun@benson31@bvanessen

@tbennun

Copy link
Copy Markdown

Tagging @tbennun@benson31@bvanessen

Please fix the conflicts :)

@szaman19
szaman19force-pushed the distconv-layer-norm branch from ecac28c to 021d2a7CompareJune 25, 2024 19:42
@benson31benson31 added the v0 Related to the now-deprecated LBANNv0 development. label Apr 7, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0Related to the now-deprecated LBANNv0 development.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@szaman19@tbennun@benson31