Skip to content
View Victarry's full-sized avatar
178 followers · 138 following

    Achievements

    Achievement: Pair Extraordinairex4Achievement: QuickdrawAchievement: YOLOAchievement: Starstruckx2Achievement: Arctic Code Vault ContributorAchievement: Pull Sharkx2

    Achievements

    Achievement: Pair Extraordinairex4Achievement: QuickdrawAchievement: YOLOAchievement: Starstruckx2Achievement: Arctic Code Vault ContributorAchievement: Pull Sharkx2

    Block or report Victarry

    Block user

    Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

    You must be logged in to block users.

    Content in all repositories owned by your account will be closed.
    Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
    Report abuse

    Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

    Report abuse
    Victarry/README.md

    Victarry

    Dennis (Zhenhuan) Liu — Megatron-Core, NVIDIA.
    MoE at multi-thousand-GPU scale: expert parallelism, communication/computation overlap, pipeline scheduling.

    github.com/Victarry

    Projects

    RepoFocus
    PP-Schedule-VisualizationInteractive visualizer for pipeline-parallel schedulesviz
    PyTorch-Memory-ProfilerTensor-level GPU memory allocation profilerprofiling
    Megatron-LMLarge-scale transformer training and Mixture-of-Expertsmoe
    Image-Generation-modelsFrom-scratch studies of generative image modelsresearch

    Lab log

    Victarry's recent merged pull requests, reviews, releases, and commits

    Recent merged PRs, reviews, releases, and commits — sourced from GitHub and cached at the edge.

    Pinned Loading

    1. NVIDIA/Megatron-LMNVIDIA/Megatron-LMPublic

      Ongoing research training transformer models at scale

      Python 17.6k 4.4k

    2. NVIDIA/TransformerEngineNVIDIA/TransformerEnginePublic

      A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance…

      Python 3.5k 809

    3. PyTorch-Memory-ProfilerPyTorch-Memory-ProfilerPublic

      Python 45 1

    4. Image-Generation-modelsImage-Generation-modelsPublic

      Generative models (GAN, VAE, Diffusion Models, Autoregressive Models) implemented with Pytorch, Pytorch_lightning and hydra.

      Python 61 9

    5. PP-Schedule-VisualizationPP-Schedule-VisualizationPublic

      Pipeline Parallelism Emulation and Visualization

      Python 84 10

    6. Megatron-MoE-EchoMegatron-MoE-EchoPublic

      Python 11 3