271
How Is LLaMa.cpp Possible?
If you want to read more of my writing, I have a Substack . Articles will be posted simultaneously to both places.
New
The latest blog posts to take off on Hacker News. 789 posts.
272
My deep learning rig (2022)
A lot of people have asked about the computers I used to train TorToiSe. I’ve been meaning to snap some pictures, but it’s never “convenient” to turn these servers off so I keep procrastinating. We had some severe thunderstorms today here i
273
Why host your own LLM?
In the Terminator movies, good relationships beat technological superiority. Kyle Reese and Sarah Connor outwit the advanced T-800 who in turn helps Sarah and John beat the ultra-advanced T-1000. OpenAI’s GPT-4 is currently the most advanced publicly available language model. There are also analyses showing it’s…
274
275
Continuous batching to increase LLM inference throughput and reduce p50 latency
In this blog, we discuss continuous batching, a critical systems-level optimization that improves both throughput and latency under load for LLMs.
276
277
278
279
280
Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Custom Models
We examine the Llama-2 models under 3 real-world use cases and show that fine-tuning yields significant accuracy improvements.
281
ML is useful for many things, but not for predicting scientific replicability
How the veneer of AI is used to legitimize awful ideas
282
There is no hard takeoff
Back in 2014, Elon Musk referred to AI as summoning the demon. And it wasn’t hard to see that view. Soon, Go agents would beat top humans learning from self play. By the end of 2017, the same algorithm mastered Chess and Shogi. By 2020, it didn’t even need tons of calls to the simulator, and could play Atari too.
283
Artificial General Intelligence – A gentle introduction
[On-line document since 2007, last updated in September 2026] [Translations: 中文 , Español , lietuvių kalba ]
284
Do Machine Learning Models Memorize or Generalize?
An interactive introduction to grokking and mechanistic interpretability.
285
Making AMD GPUs competitive for LLM inference
MLC-LLM makes it possible to compile LLMs and deploy them on AMD GPUs using ROCm with competitive performance. More specifically, AMD Radeon™ RX 7900 XTX gives 80% of the speed of NVIDIA® GeForce RTX™ 4090 and 94% of the speed of NVIDIA® GeForce RTX™ 3090Ti for Llama2-7B/13B. Besides ROCm, our Vulkan support allows us…
286
Llama from scratch, or how to implement a paper without crying
Llama from scratch I want to provide some tips from my experience implementing a paper. I'm going to cover my tips so far from implementing a dramatically...
287
Testing Intel’s Arc A770 GPU for Deep Learning
This post covers my experience training image classification models with Intel’s PyTorch extension on the Arc A770 GPU.
288
StableCode
Stability AI has just announced the release of StableCode, its very first LLM generative AI product for coding. This product is designed to assist programmers with their daily work while also providing a great learning tool for new developers ready to take their skills to the next level.
289
How Zoom’s terms of service and practices apply to AI features
As part of our commitment to transparency and user control, we're providing clarity on our approach to Zoom’s AI features and content sharing for product improvement purposes.
290
Nvidia's CUDA Monopoly
CUDA (Compute Unified Device Architecture) is a closed-source low-level API that interfaces software with NVIDIA GPUs.
291
292
MK-1
AMD acquires MK1, welcoming their world-class team and marking another key milestone in our strategy to advance AI performance and efficiency.
293
IBM and NASA open-source largest geospatial AI foundation model on Hugging Face
IBM & open-source AI platform Hugging Face today announced that IBM's watsonx.ai geospatial foundation model – built from NASA's satellite data – will now be openly available on Hugging Face.
294
Double neural bypass restores movement, sense of touch after paralysis
Using brain implants, artificial intelligence and novel stimulation technology, double neural bypass technology restores quadraplegic man’s sense of touch and movement.
295
Non-determinism in GPT-4 is caused by Sparse MoE
It’s well-known at this point that GPT-4/GPT-3.5-turbo is non-deterministic, even at temperature=0.0. This is an odd behavior if you’re used to dense decoder-only models, where temp=0 should imply greedy sampling which should imply full determinism, because the logits for the next token should be a pure function of…
296
Add an AI Code Copilot to your product using GPT-4
The AI Code Copilot has been replaced by the more powerful AI Chat feature.
297
Catching up on the weird world of LLMs
I gave a talk on Sunday at North Bay Python where I attempted to summarize the last few years of development in the space of LLMs—Large Language Models, the technology …
298
IBM and NASA Open Source Largest Geospatial AI Foundation Model on Hugging Face
IBM & open-source AI platform Hugging Face today announced that IBM's watsonx.ai geospatial foundation model – built from NASA's satellite data – will now be openly available on Hugging Face.
299
Meta’s Reality Labs prototype hardware
Both Butterscotch Varifocal and Flamera are very much in the research stage, and the technologies they’re investigating may never make their way into a consumer-facing product. And, as we hope you’ll come to see, that’s very much the point.
300
Cookbook: Finetuning Llama 2 in your own cloud environment, privately
An operational guide on finetuning Llama 2, ready for commercial use.