Learning notes and experiments on modern machine learning systems.

Currently focusing on LLM serving and inference optimization.

Posts

MLA (Multi-Head Latent Attention) Explained

19 min read

Rejection Sampling in Speculative Decoding

7 min read

Recovering NVLS on H100

10 min read

GPU Memory Hierarchy in CUDA

6 min read

Inside nano-vLLM v1

22 min read

CUDA Intro

7 min read

Softmax: From Naive to Blocked Softmax

6 min read

Inside nano-vLLM

43 min read

Pipeline Parallelism

5 min read

Tensor Parallelism

11 min read

Accelerating Diffusion Inference via Caching

16 min read

How Multiprocess Serving Works in vLLM

11 min read

Sarathi-Serve paper review

15 min read

tinyorca deep dive

21 min read

PagedAttention paper review

10 min read

Introduction to LLM Inference Part 1

32 min read

ORCA paper review

8 min read