1
Things we learned about LLMs in 2024
A lot has happened in the world of Large Language Models over the course of 2024. Here’s a review of things we figured out about the field in the past …
New
The latest blog posts to take off on Hacker News. 310 posts.
2
3
OpenAI’s board, paraphrased: ‘All we need is unimaginable sums of money’
OpenAI is to this decade’s generative-AI revolution what Netscape was to the 1990s’ internet revolution.
4
How I run LLMs locally
I document how I run Large Language Models locally.
5
Can LLMs accurately recall the Bible?
I've often found myself uneasy when LLMs (Large Language Models) are asked to quote the Bible. While they can provide insightful discussions about faith, their tendency to hallucinate responses raises concerns when dealing with scripture, which we regard as the inspired Word of God. To explore these concerns, I
6
All You Need Is 4x 4090 GPUs to Train Your Own Model
How I built an ML rig for training LLMs locally, exploring hardware choices, setup tricks, and lessons learned along the way.
7
Google's Results Are Infested, Open AI Is Using Their Playbook from the 2000s
Google search results are becoming annoying to use and Open AI is about to eat Google's lunch.
8
Building AI Products–Part I: Back-End Architecture
In 2023, we launched an AI-powered Chief of Staff for engineering leaders—an assistant that unified information across team tools and tracked critical project developments. Within a year, we attrac...
9
Short Message Compression Using LLMs
ts_sms works similarly to ts_zip . It uses a specific padding system compatible with arithmetic coding so that the message length does not need to be explicitly encoded.
10
11
Trying out QvQ – Qwen's new visual reasoning model
I thought we were done for major model releases in 2024, but apparently not: Alibaba’s Qwen team just dropped the Apache 2.0 licensed Qwen licensed (the license changed) QvQ-72B-Preview, “an …
12
AIs Will Increasingly Fake Alignment
This post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking in Large Language Models.
13
Narrative Jailbreaking for Fun and Profit
Posted on Monday 23 Dec 2024. 1,634 words, 1 link. By Matt Webb.
14
Can AI do maths yet? Thoughts from a mathematician
So the big news this week is that o3, OpenAI’s new language model, got 25% on FrontierMath. Let’s start by explaining what this means.
15
I sensed anxiety and frustration at NeurIPS 24
last week at NeurIPS’24, one extremely salient thing was the anxiety and frustration felt and expressed by late-year PhD students and postdocs who were confused by the job market that looks and feels so much different from what they expected perhaps when they were applying for PhD programs five or so years ago. and…
16
Maximum likelihood estimation and loss functions
When I started learning about loss functions, I could always understand the intuition behind them. For example, the mean squared error (MSE) for regression seemed logical—penalizing large deviations from the ground-truth makes sense. But one thing always bothered me: I could never come up with those loss functions on…
17
Quick takes on the recent OpenAI public incident write-up
OpenAI recently published a public writeup for an incident they had on December 11, and there are lots of good details in here! Here are some of my off-the-cuff observations: Saturation With thousa…
18
Tenstorrent and the State of AI Hardware Startups
Semi-custom silicon is a bigger problem than Nvidia.
19
Fast LLM Inference From Scratch (using CUDA)
This post is about building an LLM inference engine using C++ and CUDA from scratch without libraries.
20
Computing Inside an AI
Since the launch of ChatGPT, explorations in two directions kicked into high gear.
21
22
A ChatGPT clone, in 3000 bytes of C, backed by GPT-2 (2023)
This program is a dependency-free implementation of GPT-2, including byte-pair encoding and transformer inference, in ~3000 bytes of C. I then use this to create something like Chat GPT.
23
The case against Google's claims of "quantum supremacy"
The 2019 paper “Quantum supremacy using a programmable superconducting processor” asserted that Google’s Sycamore quantum computer, with 53 qubits and a depth of 20, performed a specific computat…
24
The Google Willow Thing
Yesterday I arrived in Santa Clara for the Q2B (Quantum 2 Business) conference, which starts this morning, and where I’ll be speaking Thursday on “Quantum Algorithms in 2024: How Should…
25
26
Task-specific LLM evals that do and don't work
Evals for classification, summarization, translation, copyright regurgitation, and toxicity.
27
The 70% problem: Hard truths about AI-assisted coding
A field guide and why we need to rethink our expectations
28
Bringing K/V context quantisation to Ollama
K/V context cache quantisation has been added to Ollama. This enables significant reductions in VRAM usage, allowing users to realise the potential of expanded context sizes and run larger models at their existing context sizes.
29
LLM abstraction levels inspired by fish eye lens
We often think of the same information at different levels or abstraction. Here's a simple example:
30
Breaking the 4Chan CAPTCHA
In this blog post, I will talk about my successful experiment in machine learning using TensorFlow to automate solving the 4Chan CAPTCHA.