Uh oh!
There was an error while loading. Please reload this page.
Feature/multi node training - #90
Conversation
b4b571c to
30640d6Comparebdc5db7 to
181a687Compare6a30c51 to
d1cc216Compare…ad and multi-node multi-process scenarios
…rocessGroup destructor.
d1cc216 to
03f3534Compare以上述测例中双机 16 卡 llama fp32 为例,主节点运行指令: 唯一区别是传给 infini_run 的 --node_rank 参数值。 |
03f3534 to
c48ac8dCompareUh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.








已完成多机训练基础设施的开发,并在双机 16 卡环境下验证了 llama/gpt2 在 fp32 与 bf16 精度下的多维并行训练能力(支持 DDP、TP、SP、PP 及其组合)。