Skip to content

Baseten Blog company blog

A guide to open-source LLM inference and performance

baseten.co

Read on baseten.co → Discuss on HN (14) Archived copy

Points: 113
HN comments: 14
Submissions: 1
Cited by: 0 commenters

AI & Machine Learning Infrastructure & DevOps 2023

On Hacker News

Submitted	Title on HN	Points	Comments
Nov 20, 2023	A guide to open-source LLM inference and performance by varunshenoy	113	14

More from Baseten Blog

All 3 posts →

Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs

2025247 points175 comments AI & Machine Learning details

The efficient frontier of LLM inference

2026155 points46 comments AI & Machine Learning details

Related posts from the same era

Bing: “I will not harm you unless you harm me first”

Simon Willison20233,363 points1,673 commentscited by 6 details

GPT-4o

OpenAI Blog20243,138 points2,366 commentscited by 16 details

Open source AI is the path forward

Meta Newsroom20242,360 points887 commentscited by 5 details

Building a Virtual Machine Inside ChatGPT

Jonas Degrave20222,029 points908 commentscited by 16 details

The man who killed Google Search?

Ed Zitron20241,884 points878 commentscited by 47 details

ChatGPT Plugins

OpenAI Blog20231,875 points1,106 commentscited by 14 details