Archives
All the articles I've archived.
20265
October5
Parallelism Strategies in Distributed Computing
One map of DP, DDP, FSDP, ZeRO, TP, PP, CP, SP and EP across training and inference, what each costs, and which ones vLLM supports.
Open-Source RL Frameworks for LLMs: verl, OpenRLHF, TRL, NeMo RL, ROLL, slime, AReaL
A side-by-side comparison of the popular LLM RL training frameworks — verl, OpenRLHF, TRL, NeMo RL, ROLL, slime, AReaL — and a deep dive into verl's system design.
NVIDIA Data Center GPUs from Pascal to Blackwell
P100 → V100 → A100 → H100 → B200 — specs, internal architecture, and what each generation changed.
vLLM Quantization
How vLLM loads quantized checkpoints, quantizes BF16 models at load time, picks kernels, and which format to choose (FP8, INT8, INT4, MXFP4, KV cache).
How vLLM Sleep Mode Works
Free and restore vLLM's GPU memory without restarting the process — Level 0/1/2, how CuMemAllocator does it, usage, and the speedup.