Baseten Blog

Running GPT-OSS-120B at 500 tokens per second on Nvidia GPUs

Train, serve, and scale open-source and custom AI models on the fastest, most reliable inference platform.

The efficient frontier of LLM inference

Inference techniques either move a deployment along the latency–throughput frontier or push the entire frontier out, creating more efficiency to allocate.

A guide to open-source LLM inference and performance