241
AI isn’t good enough
On a recent cross-country trip—I (Paul) drove from California to Illinois and back again.
242
Nvidia’s AI supremacy is only temporary
Nvidia is an amazing company that has executed a contrarian vision for decades, and has rightly become one of the most valuable corporations on the planet thanks to its central role in the AI revol…
243
Building Boba AI: Lessons learnt in building an LLM-powered application
A series of patterns for building an AI co-pilot application.
244
Notes on training BERT from scratch on an 8GB consumer GPU
I trained a BERT model (Devlin et al, 2019) from scratch on my desktop PC (which has a Nvidia 3060 Ti 8GB GPU). The model architecture, tokenizer, and trainer all came from Hugging Face libraries, and my contribution was mainly setting up the code, setting up the data (~20GB uncompressed text), and leaving my computer…
245
Translating Latin demonology manuals with GPT-4 and Claude
At long last, a real-world use case for AI!
246
Understanding Llama 2 and the New Code Llama LLMs
In this edition of the newsletter, we direct our attention to one of the most prominent highlights of the summer: the release of the Llama 2 base and chat models, as well as CodeLlama, the latest highlights in the open-source AI large language model (LLM) landscape.
247
248
Google's True Moonshot
Google could do more than just win the chatbot war: it is the one company that could make a universal assistant. The question is if the company is willing to risk it all.
249
Why Do ML on the Erlang VM?
Another question might be; why do machine learning at all? I'm not big into ML/AI though I've been poking it more recently as the open models, such as Stable Diffusion and the more practical Whisper, caught my curiosity. I have a very decent GPU (3090 Ti) and I've poked around a bit. I don't consider ML a super…
250
LLMs can't self-correct in reasoning tasks, DeepMind study finds
A study by Google's DeepMind and the University of Illinois at Urbana-Champaign has found that self-correction in large language models (LLMs) isn't universally effective.
251
Translating Akkadian clay tablets with ChatGPT?
Alles wat Jan bezighoudt, interesseert en irriteert... en ook een beetje onzin...
252
Flow-Based Programming, a way for AI and humans to develop together
I think by now everybody reading this will have seen how the new generation of Large Language Models like ChatGPT are able to produce somewhat useful code. Like any advance in software development—from IDEs to high-level languages—this has generated some discussion on the future employment prospects in our field.
253
My deep learning rig (2022)
A lot of people have asked about the computers I used to train TorToiSe. I’ve been meaning to snap some pictures, but it’s never “convenient” to turn these servers off so I keep procrastinating. We had some severe thunderstorms today here i
254
How RLHF Works
Why RLHF may still win out and why we haven't seen it yet in open-source.
255
256
Yoshua Bengio: How Rogue AIs May Arise
The rise of powerful AI dialogue systems in recent months has precipitated debates about AI risks of all kinds, which hopefully will yield an acceleration of governance and regulatory frameworks.
257
How to implement Q&A against your docs with GPT3 embeddings and Datasette
If you’ve spent any time with GPT-3 or ChatGPT, you’ve likely thought about how useful it would be if you could point them at a specific, current collection of text …
258
AI Won't Cause Unemployment
"In retrospect, I wish I had known more about the hazards and difficulties of [running] a business." -- George McGovern
259
Open Challenges in LLM Research
[LinkedIn discussion, Twitter thread]
260
Will AIs take all our jobs and end human history? It’s complicated
Stephen Wolfram explores some of the science, technology and philosophy of what we can expect from AIs. From how ChatGPT works to the cycle of technology, to the concept of progress and preparing for an AI world.
261
Chatbots are not the future
Unfortunately for the countless hapless people I've talked to in the past few months, this was inexorable. Ever since ChatGPT exploded in popularity, my inner designer has been bursting at the seams.
262
CUDA vs. ROCm: A case study
A simple, whitespace theme for academics. Based on [*folio](https://github.com/bogoli/-folio) design.
263
LLM Python/CLI tool adds support for embeddings
LLM is my Python library and command-line tool for working with language models. I just released LLM 0.9 with a new set of features that extend LLM to provide tools …
264
Two Kinds of Techno-Optimism
A few things Marc Andreessen forgot to mention
265
The future of programming: Research at CHI 2023
Research on programming tools published at CHI 2023.
266
267
Understanding and coding the self-attention mechanism of large language models
Step-by-step tutorial explaining scaled dot-product self-attention for large language models, with Python code that builds the mechanism from scratch.
268
Emergent abilities of large language models
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
269
Bayes is guaranteed to overfit
The myth A few years ago I saw a StackOverflow question (but I cannot find it now): allegedly Andrew Gelman blogged that Bayesian models would not overfit and would only underfit (I also cannot find Andrew’s blog post now but I think what he meant by “Bayesian models only underfits” is that you can never fully model…
270
There is no hard takeoff
Back in 2014, Elon Musk referred to AI as summoning the demon. And it wasn’t hard to see that view. Soon, Go agents would beat top humans learning from self play. By the end of 2017, the same algorithm mastered Chess and Shogi. By 2020, it didn’t even need tons of calls to the simulator, and could play Atari too.