IFU-master-2021-09-14 - #40

Closed

rraminen wants to merge 51 commits into

ROCm:masterfrom

rraminen:IFU-master-2021-09-14

Commits on Aug 16, 2021

Curriculum learning (#1307)

conglongli
and
jeffra
authored
b2b34ae

Commits on Aug 17, 2021

DeepSpeed MoE (#1310)

authored
f284324
bump 0.5.1, DSE, moe docs
jeffra
committed
a1de767
MoE read the docs update (#1312)
jeffra
authored
9cb64a1
add moe to sidebar
jeffra
committed
e070a09

Commits on Aug 18, 2021

Updating the torch version check to numeric (#1314)
pruthvistony
authored
e804f15
add MoE press release links
jeffra
committed
058ab81
[docs] update moe features and news post
jeffra
committed
6cd5f87

Commits on Aug 19, 2021

Add issue templates
jeffra
authored
10b4840

Commits on Aug 25, 2021

Commits on Aug 26, 2021

Reducing the memory-overhead of creating model for multi-GPU run (#1244)

RezaYazdaniAminabadi
and
jeffra
authored
49b6a63
bump to 0.5.2
jeffra
committed
c1b0a4e

Commits on Aug 27, 2021

Commits on Aug 28, 2021

Use clone to avoid checkpoint bloat (#1326)
tjruwase
authored
336dd08

Commits on Aug 30, 2021

update for cuda-11.4 (#1329)

stas00
and
jeffra
authored
b9ece25

Commits on Aug 31, 2021

Remove duplicate clip grad function in deepspeed (#1333)
awan-10
authored
ddffbae

Commits on Sep 1, 2021

Added drop_last to DeepSpeedDataLoader (#1321)

authored
c0b27fb
Support client lr schedulers that are not subclass of torch _LRScheduler (#1337)
tjruwase
authored
8e301b6
Use mpu in zero.Init() (#1325)
tjruwase
authored
e08c239
Update main.yml (#1338)
jeffra
authored
74f058b

Commits on Sep 2, 2021

[actions] split formatting and unit tests into two jobs (#1339)
jeffra
authored
600db09
[actions] update branch triggers
jeffra
authored
8f299be
[actions] add master to formatting trigger
jeffra
authored
86b948f

Commits on Sep 8, 2021

[actions] add torch version runner label
jeffra
authored
0a32c3e

Commits on Sep 9, 2021

[actions] revert unit-test build name
jeffra
authored
3e7d06a
Added 4-byte alignment on NCCL/RCCL (#1328)

authored
2c62843

Commits on Sep 10, 2021

Update matmul.py (#1349)
hyunwoongko
authored
eb97a42
update hiring link
jeffra
authored
168fce3
Update website hiring link
jeffra
authored
bff6126
Correctness fix PP+ZeRO for gradient accumulation (#1264)

jeffra
and
tjruwase
authored
b712bab
Remove dropout as client code can do it independently. (#1354)

awan-10
and
tjruwase
authored
9f5939d
fix unit test failure when merging PP ckpt files (#1359)
eltonzheng
authored
9b915fe

Commits on Sep 13, 2021

Update setup.py (#1361)
anukaal
authored
8e577c9

Commits on Sep 14, 2021

Merge remote-tracking branch 'upstream/master' into IFU-master-2021-09-14
rraminen
committed
df6d3ab
bump to 0.5.3
jeffra
committed
a708b18

Commits on Sep 15, 2021

Introduce a device rank when setting device (#1370)
Sean Naren
authored
90398a7

Commits on Sep 16, 2021

[zero Init] fix regression (#1373)
stas00
authored
cf22a69
[zero_to_fp32] adapt to 4-bytes alignment in z2 (#1372)

stas00
and
tjruwase
authored
30537e7
bump to 0.5.4
jeffra
committed
45a498d

Commits on Sep 18, 2021

fix: support three digit layer numbers (#1377)
adammoody
authored
4ad8019

Commits on Sep 21, 2021

Sparse attn triton v1.0 support + torch1.8 test runner (#1374)

jeffra
and
Reza Yazdani
authored
6996bb0
add tutorial on pytorch profiler usage (#1350)

authored
51a2e91

Commits on Sep 22, 2021

[zero_to_fp32] fix padding removal (#1380)

stas00
and
jeffra
authored
364994a
Fix from Felipe and Young for loading checkpoints. (#1389)
awan-10
authored
86dd6a6

Commits on Sep 24, 2021

[CI] Add HF transformers tests (#958)

authored
c1829c4

Commits on Sep 28, 2021

[CI] install fix for HF tests and use dict instead of json for some tests (#1405)
jeffra
authored
9e5c0c5
Bump nokogiri from 1.11.4 to 1.12.5 in /docs (#1409)

dependabot[bot]
and
jeffra
authored
39c7744
Mergedfrom upstream
rraminen
committed
7bb7c8b