core: Kimi-K3 decode/prefill scaffolding (distributed weight store, DeepEP-LL exchange, resident MoE)

[Andrewxu313]

Description

Core scaffolding to serve Kimi-K3 (2.8T, 93 layers, 896 MXFP4 experts, top-16) on 2×8 H200 — the runtime/serving layer only:

• compact distributed host-weight store + daemon (core/Weights_Storage/**)
• DeepEP low-latency EP-exchange primitive (batchgen/moe/deepep_ll.py)
• resident-MoE runtime (marlin grouped MoE, streamed SP8, dequant-once)
• server flags --enable-deepep (boolean: default off = NCCL, on = require DeepEP or fail fast) and --distributed-weight-config

Model + kernels are the companion PR #317, stacked on this.

Motivation

Kimi-K3 needs runtime/serving changes a model PR may not touch (PR_MERGE_POLICY §2.5). Splitting the scaffolding into this core PR keeps the model PR inside its allowlist and lets the two layers review independently.

Type of Change

• model — add/extend model support (models/** + registration seam + model kernels only)
• kernel — add/optimize a compute kernel (batchgen_kernels/** + in-tree kernel dirs)
• core — change scheduling/serving/runtime scaffolding (the only type that may)
• fix — narrow bug fix (+ a regression test)
• infra — build / CI / packaging / scripts / Docker
• docs — documentation only

File changes

File	Δ	Note
batchgen/batchgen_worker.py	mod	K3 support
batchgen/ckpt_converter/ckpt_converter.py	mod	K3 support
batchgen/config/config.py	mod	K3 support
batchgen/continuous_batching.py	mod	K3 support
batchgen/decode_dp_group.py	add	K3 support
batchgen/kernel_compat.py	mod	K3 support
batchgen/kv_cache/gpu_paged_kv_manager.py	mod	KV-cache scaffolding
batchgen/kv_cache/host_kv_mananger_config.py	mod	KV-cache scaffolding
batchgen/moe/deepep_ll.py	add	MoE runtime primitive
batchgen/moe/dispatch_scatter_3d.py	mod	MoE runtime primitive
batchgen/moe/fused_moe_bf16_resident.py	mod	MoE runtime primitive
batchgen/moe/fused_moe_mxfp4_resident.py	add	MoE runtime primitive
batchgen/moe/fused_wgmma_expert.py	mod	MoE runtime primitive
batchgen/moe/fused_wgmma_grouped.py	mod	MoE runtime primitive
batchgen/moe/grouped_fp8_blockwise_moe.py	mod	MoE runtime primitive
batchgen/moe/k3_prefill_dequant_once.py	add	MoE runtime primitive
batchgen/moe/marlin_grouped_moe.py	mod	MoE runtime primitive
batchgen/moe/marlin_weight_prep.py	mod	MoE runtime primitive
batchgen/moe/mxfp4_grouped_gemm.py	mod	MoE runtime primitive
batchgen/moe/mxfp4_oracle_vector.py	mod	MoE runtime primitive
batchgen/moe/routing/cuda_routing.py	mod	MoE runtime primitive
batchgen/moe/streamed_sp8_mxfp4.py	add	MoE runtime primitive
batchgen/other_kernels/hadamard_transform/__init__.py	mod	K3 support
batchgen/planner/base_planner.py	mod	K3 support
batchgen/sequence.py	mod	K3 support
batchgen/server/batch_scheduler.py	mod	server/serving scaffolding
batchgen/server/server_args.py	mod	server/serving scaffolding
batchgen/server/worker_manager.py	mod	server/serving scaffolding
batchgen/server/worker_readiness.py	add	server/serving scaffolding
batchgen/server_worker_main_loop.py	mod	K3 support
batchgen/worker/boundary.py	mod	worker scaffolding
batchgen/worker/decode.py	mod	worker scaffolding
batchgen/worker/kv_manager.py	mod	worker scaffolding
batchgen/worker/prefill.py	mod	worker scaffolding
core/GPU_Weight_Buffer/GPU_Weight_Buffer.cpp	mod	C++ engine scaffolding
core/GPU_Weight_Buffer/GPU_Weight_Buffer.h	mod	C++ engine scaffolding
core/HtoD_Engine/HtoD_Engine.cu	mod	C++ engine scaffolding
core/HtoD_Engine/HtoD_Engine.h	mod	C++ engine scaffolding
core/KV_Storage/compressed_state_host_manager.h	mod	C++ engine scaffolding
core/KV_Storage/host_paged_kv_worker_view.h	mod	C++ engine scaffolding
core/Parameter_Server/posix_shm.cpp	mod	C++ engine scaffolding
core/Weights_Storage/Weights_Storage.cpp	mod	C++ engine scaffolding
core/Weights_Storage/Weights_Storage.h	mod	C++ engine scaffolding
core/Weights_Storage/distributed_weight_daemon.cpp	add	C++ engine scaffolding
core/Weights_Storage/distributed_weight_daemon.h	add	C++ engine scaffolding
core/Weights_Storage/distributed_weights_protocol.h	add	C++ engine scaffolding
core/batchgen.cpp	mod	C++ engine scaffolding
core/batchgen.h	mod	C++ engine scaffolding
core/batchgen_Binding.cpp	mod	C++ engine scaffolding
core/numa_compat.h	add	C++ engine scaffolding
docs/troubleshooting.md	mod	docs
op_builder/core_engine.py	mod	build
scripts/install_deps.sh	mod	build

Checklist

• I have read the CONTRIBUTING guide and the PR Merge Policy Contract.
• I have updated the tests (if applicable).
• I have updated the documentation (if applicable).