NVIDIA Developer Blog company blog

Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM

Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations connected over ConnectX-8 RDMA serve one…
Points
144
HN comments
69
Submissions
1
Cited by
0 commenters

AI & Machine LearningSystems & Performance2026

On Hacker News

SubmittedTitle on HNPointsComments
Nov 14, 2026Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM by wondersky14469

More from NVIDIA Developer Blog

All 14 posts →

Related posts from the same era