LLM-D: Kubernetes-Native Distributed Inference
Introducing llm-d: Kubernetes-native distributed LLM inference with KV-cache routing, disaggregated serving, and SOTA performance per dollar. Built on vLLM.
Topics /
The most popular blog posts about ai & machine learning on Hacker News. 1,227 posts.
Introducing llm-d: Kubernetes-native distributed LLM inference with KV-cache routing, disaggregated serving, and SOTA performance per dollar. Built on vLLM.
Turning LLMs into coding experts and how to take advantage of them.
This post shares the local optima Manus arrived at through our own "SGD". If you're building your own AI agent, we hope these principles help you converge faster.
MongoDB to integrate Voyage AI's industry-leading embedding and reranking models, delivering highly accurate and relevant information retrieval to power sophisticated AI use cases NEW YORK , Feb. 24, 2025 /PRNewswire/ -- MongoDB, Inc. (NASDAQ: MDB), the leading database for modern applications,
The agentic loop—generate, assess, improve—seems like a natural fit for iterating on pelicans on bicycles.
A now-deleted report by HonestAGI on GitHub showed a high number of similarities between Huawei’s Pangu and Qwen 2.5.
A critical benchmark for image generation models: A partially eaten burrito with cheese, sour cream, guacamole, lettuce, salsa, pinto beans, and chicken.
AI Agents, Modularity, and Alignment
When we first set up the llm such that it could play text adventures , we noted that none of the models we tried to use with it were any good at it. We dreamed of a way to compare them, but all I could think of was setting a goal far into the game and seeing how long it takes them to get there. I just realised there’s…
Company crosses a dangerous line by beginning to offer AI suspicion-generation functions
We are excited to announce that ClickHouse has acquired LibreChat, the leading open-source AI chat platform.
Open source LLMs now outperform closed alternatives at 90% lower cost for workhorse tasks. We compare benchmarks, pricing, and real-world performance.
QWEN CHAT API DEMO DISCORD It is widely recognized that continuously scaling both data size and model size can lead to significant improvements in model intelligence. However, the research and industry community has limited experience in effectively scaling extremely large models, whether they are dense or…
Majorana 1, the first quantum chip powered by a new Topological Core architecture
Anthropic is partnering with Fluidstack to build data centers in Texas and New York, creating roughly 800 permanent and 2,400 construction jobs.
Patterns for managing agent context.
We investigate the internal mechanisms used by Claude 3.5 Haiku — Anthropic's lightweight production model — in a variety of contexts, using our circuit tracing methodology.
A virtual feature store for the modern AI stack
Everybody get up, it's time to slam now, we got a real jam goin down
New revenue benchmarks for gen AI startups in year one, based on dozens of enterprise and consumer AI companies we’ve spent significant time with.
When the world's richest man can simply 'correct' an AI to reflect his own values, what does that tell us about the entire alignment discourse?
Check out our deep dive on the latest meta prompting methods, like DSPy and TEXTGRAD, and the best prompt generator tools out there.
The difference between the IT bubble in the 1990s and the AI bubble today is that the top 10 companies...
Something’s been bugging me about how new devs learn and I need to talk about it. We’re at this weird inflection point in software development. Every junior dev I talk to has Copilot or Claude or GPT running 24/7. They’re shipping code faster than ever. But when I dig deeper into their understanding of what they’re…
I run Claude Code with --dangerously-skip-permissions flag, giving it full system access. Let me show you a new way of approaching computers.
Thanks to NVIDIA’s early access program, we are thrilled to get our hands on the NVIDIA DGX™ Spark. It’s quite an unconventional system, as NVIDIA rarely releases compact, all-in-one machines that bring supercomputing-class performance to a desktop workstation form factor.
Inside the effort to de-democratize AI
Two years ago, Americans anxious about the forthcoming 2024 presidential election were considering the malevolent force of an election influencer: artificial intelligence. Over the past several years, we have seen plenty of warning signs from elections worldwide demonstrating how AI can be used to propagate…
A Response to Yudkowsky and Soares from the Front Lines of AGI Development