Pinned Loading

1. multigpu-llm-finetuningmultigpu-llm-finetuningPublic

   This repository showcases hands-on projects leveraging distributed multi-GPU training to fine-tune large language models (LLMs).

   Python

2. hivemind-qwen2-0.5bhivemind-qwen2-0.5bPublic

   Internet-scale data-parallel fine-tuning of Qwen2-0.5B-Instruct using Hivemind + TorchTune. Initial peer on public IP; second peers on free GPUs (e.g., Kaggle).

3. petals-llama2-70bpetals-llama2-70bPublic

   Decentralised inference + Prompt tuning of LLaMA-2-70B with Petals (swarm model-parallelism). Meta-repo with summary and links to the two write-ups.

4. hivemind-modified-for-torchtunehivemind-modified-for-torchtunePublic

   Forked from learning-at-home/hivemind

   Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

   Python 1

5. torchtunetorchtunePublic

   Forked from meta-pytorch/torchtune

   PyTorch native finetuning library

   Python 1