Hacker Blog Posts

Infini-AI-Lab company blog

SEQUOIA: Exact Llama2-70B on an RTX4090 with half-second per-token latency

infini-ai-lab.github.io 3 min read

Serving Llama2-70B on an RTX4090 with Sequoia
Read on infini-ai-lab.github.io → Discuss on HN (61) Archived copy
Points
131
HN comments
61
Submissions
1
Cited by
0 commenters

AI & Machine LearningSystems & Performance2024

On Hacker News

SubmittedTitle on HNPointsComments
May 5, 2024SEQUOIA: Exact Llama2-70B on an RTX4090 with half-second per-token latency by zinccat13161

Related posts from the same era