New

The latest blog posts to take off on Hacker News. 80,767 posts.

Any topic
Any year
190

Fixing Tool-Call Accuracy in vLLM

Tool call accuracy is an important part of serving open-weight AI models. When your tool calls aren’t accurate, the model builds up on nonsensical context. We’ve been carrying our own fork of vLLM to hit 100% in tool call schema accuracy with Kimi and DeepSeek. Fortunately, vLLM 0.31 added support for constrained…