781
An interview with Steve Wozniak by Jessica Livingston cured my AI anxiety
Note: there's no hidden knowledge contained in this post, it's just a reflection on how reading someone's reaction to an analogous situation...
782
Visualizing Neural Networks
A visual example-focused introduction to neural networks.
783
It Looks Like You’re Trying to Take over the World
Fictional short story about Clippy & AI hard takeoff scenarios grounded in contemporary ML scaling, self-supervised learning, reinforcement learning, and meta-learning research literature.
784
GPT-2 as step toward general intelligence (2019)
A machine learning researcher writes me in response to yesterday’s post, saying:I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you bac…
785
AI Written, AI Read cartoon
One piece of slang that has long embodied the short attention span Internet age is TL;DR, short for “too long; didn’t read.” With the explosion of generative AI tools, we’re rapidly entering the age of TL;DW: “too long, didn’t write.” A January survey from Fishbowl found that 40% of nearly 12,000 workers have used…
786
RLHF: Reinforcement Learning from Human Feedback
[LinkedIn discussion, Twitter thread]
787
Aligning language models to follow instructions
We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models…
788
CLIP: Connecting text and images
We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the “zero-shot” capabilities of GPT-2 and GPT-3.
789
The Scaling Hypothesis
On GPT-3: meta-learning, scaling, implications, and deep theory. The scaling hypothesis: neural nets absorb data & compute, generalizing and becoming more Bayesian as problems get harder, manifesting new abilities even at trivial-by-global-standards-scale. The deep learning revolution has begun as foretold.