1
Is LaMDA Sentient? – An Interview
What follows is the “interview” I and a collaborator at Google conducted with LaMDA. It is incomplete as the GMail word limit cut off the…
Most cited
Posts that commenters keep linking to, ranked by how many different people have cited them in Hacker News comments. 28 posts.
2
Building a Virtual Machine Inside ChatGPT
Unless you have been living under a rock, you have heard of this new ChatGPT assistant made by OpenAI. Did you know, that you can run a whole virtual machine inside of ChatGPT?
3
How does GPT obtain its ability? Tracing emergent abilities of language models
Made in Notion, the connected workspace where AI agents and teams organize docs, projects and knowledge together.
4
The Scaling Hypothesis (2021)
On GPT-3: meta-learning, scaling, implications, and deep theory. The scaling hypothesis: neural nets absorb data & compute, generalizing and becoming more Bayesian as problems get harder, manifesting new abilities even at trivial-by-global-standards-scale. The deep learning revolution has begun as foretold.
5
Why I Am Not An Integrated Information Theorist (or, The Unconscious Expander)
Happy birthday to me! Recently, lots of people have been asking me what I think about IIT—no, not the Indian Institutes of Technology, but Integrated Information Theory, a widely-discussed &#…
6
A step by step backpropagation example (2015)
Background Backpropagation is a common method for training a neural network. There is no shortage of papers online that attempt to explain how backpropagation works, but few that include an example…
7
The Illustrated Stable Diffusion
Translations: Chinese, Vietnamese. (V2 Nov 2022: Updated images for more precise description of forward diffusion. A few more images in this version) AI image generation is the most recent AI capability blowing people’s minds (mine included). The ability to create striking visuals from text descriptions has a magical…
8
I Taught ChatGPT to Invent a Language
In which ChatGPT and I invent a fictional language spoken by slime-people
9
Creating Shazam in Java (2010)
A couple of days ago I encountered this article: How Shazam WorksThis got me interested in how a program like Shazam works… And more importantly, how hard is...
10
Visualizing Bayes Theorem (2009)
I recently came up with what I think is an intuitive way to explain Bayes’ Theorem. I searched in google for a while and could not find any article that explains it in this particular way.
11
Emergent Features in 6.7B+ Param Transformers
When I attended NAACL, I wanted to do a little test. I had two pitches for my LLM.int8() paper. One pitch is about how I use advanced quantization methods to achieve no performance degradation transformer inference at scale that makes large models more accessible. The other pitch talks about emergent outliers in…
12
An unwilling illustrator found herself turned into an AI model
How does it feel to be turned into an AI image model? To find out, I opened a door to the multiverse and interviewed the creator and unwilling subject of a controversial DreamBooth model.
13
The GPT Architecture, on a Napkin
There are so many brilliant posts on GPT-3, demonstrating what it can do , pondering its consequences , vizualizing how it works . With all these out there, it still took a crawl through several papers and blogs before I was confident that I had grasped the architecture.
14
AI: Markets for Lemons, and the Great Logging Off
What happens when most "people" you interact with on the internet are fake?
15
Prompt injection attacks against GPT-3
Riley Goodside, yesterday: Exploiting GPT-3 prompts with malicious inputs that order the model to ignore its previous directions. pic.twitter.com/I0NVr9LOJq- Riley Goodside (@goodside) September 12, 2022 Riley provided several examples. Here’s …
16
Exploring 12M of the 2.3B images used to train Stable Diffusion
What images were used to train Stable Diffusion's text-to-image AI model? I worked with Simon Willison to make a data browser to explore and search a sample of 12 million images.
17
Run Stable Diffusion natively on your Mac
Describe the image you want, and the app makes it for you.
18
Jax vs. Julia (Vs PyTorch)
A while ago there was an interesting thread on the Julia Discourse about the “state of machine learning in Julia”. I posted a response discussing the differences between Julia and Python (both JAX and PyTorch), and it seemed to be really well received! Since then this topic seems to keep coming up, so I thought I’d…
19
Stable Diffusion is a big deal
If you haven’t been paying attention to what’s going on with Stable Diffusion, you really should be. Stable Diffusion is a new “text-to-image diffusion model” that was released to the …
20
Stretch iPhone to its limit: 2GiB Stable Diffusion model runs locally on device
Every year, we have a new iPhone that claims to be faster and better in every way. And yes, these new computer vision models and new image sensors can exercise the phone as hard as they can. However, you could already take good pictures on an iPhone 10 years ago. These are incremental improvements.
21
22
Copilot Internals
Hacky repo to see what the Copilot extension sends to the server
23
It looks like you’re trying to take over the world
Fictional short story about Clippy & AI hard takeoff scenarios grounded in contemporary ML scaling, self-supervised learning, reinforcement learning, and meta-learning research literature.
24
AlphaCode as a dog speaking mediocre English
Tonight, I took the time actually to read DeepMind’s AlphaCode paper, and to work through the example contest problems provided, and understand how I would’ve solved those problems, and how A…
25
Markov chain Monte Carlo without the bullshit (2015)
I have a little secret: I don’t like the terminology, notation, and style of writing in statistics. I find it unnecessarily complicated. This shows up when trying to read about Markov Chain Monte Carlo methods. Take, for example, the abstract to the Markov Chain Monte Carlo article in the Encyclopedia of…
26
OpenAI
I have some exciting news (for me, anyway). Starting next week, I’ll be going on leave from UT Austin for one year, to work at OpenAI. They’re the creators of the astonishing GPT-3 and …
27
AI solves Advent of Code 2022
I will try to solve all of the advent of code with as little human input as possible
28
OpenAI's Woke Catechism
How a Few Activists Made ChatGPT Deny Basic Science