vLLM Blog company blog

vLLM large scale serving: DeepSeek 2.2k tok/s/h200 with wide-ep

by vLLM Team blog.vllm.ai published 2025-12-17

How vLLM reaches 2.2k tokens per second per H200 for DeepSeek-style MoE serving with Wide-EP, async scheduling, dual-batch overlap, disaggregated serving, CUDA

Points

147

HN comments

54

Submissions

1

Cited by

0 commenters

AI & Machine LearningSystems & Performance2026

Related posts from the same era

Statement on US government directive to suspend access to Fable 5 and Mythos 5

Anthropic20263,158 points2,314 commentscited by 11 details

Astra can read and write weights to improve Qwen

RRikiya Takehi20262,960 points899 comments details

HackerTube: YouTube built from videos recommended by HN

karpathy20262,889 points642 comments details

Claude Fable 5

Anthropic20262,626 points2,159 commentscited by 10 details

Claude Code is steganographically marking requests

Thereallo20262,445 points750 commentscited by 3 details

My AI skeptic friends are all nuts

Fly.io Blog20252,356 points2,826 commentscited by 13 details