“People say, It’s just glorified autocomplete . . . Now, let’s analyze that. Suppose you want to be really good at predicting the next word. If you want to be really good, you have to understand what’s being said. That’s the only way. So by training something to be really good at predicting the next word, you’re…
I have developed a way to supercharge ChatGPT. It’s called Nova Mode and it’s an awesome custom instruction that you can paste into your ChatGPT Custom instructions (see screenshot below) in ChatGPT Settings. This modifies how ChatGPT works in all your chats, giving you much more control over how you interact with…
Recently, I started working on a new LLM-oriented project . It’s currently far from feature-complete and probably filled with bugs, but I’m hoping to use it as a platform for exploring a question I’ve been considering a lot lately: what will LLM-powered software look like in the medium-term future? It seems undeniable…
The philosopher Harry Frankfurt defined bullshit as speech that is intended to persuade without regard for the truth. By this measure, OpenAI’s new chatbot ChatGPT is the greatest bullshitter ever. Large Language Models (LLMs) are trained to produce
Yesterday, Sam Altman and Greg Brockman were fired from the Board of Directors of OpenAI.
By Marc Andreessen
Non-proliferation only benefits incumbents
A method for extracting multiple concepts from a single image.
Wouldn't it be great to have a pet AI chat bot that knows you really well, and can give you free consulting, coaching and therapy ? What could possibly go wrong if an intelligent corporate-controlled system has unlimited access to your private data to provide you the best possible service? Well... a lot of things.
Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy…
In the emerging age of AI generated code, is leetcode really the best way to evaluate technical candidates (was it ever)?
A short story on how my website lost all indexed pages in Bing, affecting my presence in DuckDuckGo, secret blacklisting and classification as a splog by Bing AI
(Spoiler alert: it has been very successful, but there are some lessons to be learned)
Important: This page summarizes data scaling only, using tokens to parameters as a ratio, and as derived from large language models like GPT-3, Chinchilla, and beyond, linked to the Compute-Optimal scaling laws like Kaplan and Hoffmann/Chinchilla. Please note that compute scaling laws are outside the scope of my…
A little bit of magic, but mostly just practice
I explain what is so unique about the RWKV language model.
If you ever tried to learn how the Harris corner detection algorithm works, you might have noticed that the process is not intuitive at all. First, you start with an energy function, approximate it using Taylor approximation, get a matrix from that, then find the eigenvalues of that matrix, etc. But when you come to…
Approximately four GPTs and seven years ago, OpenAI’s founders brought forth on this corporate landscape a new entity, conceived in liberty, and dedicated to the proposition that all men might live…
Old is new again
Stephen Wolfram shares memories of Doug Lenat, early AI pioneer and developer of the CYC system. Their discussions around AI & logic and the potential combined uses of their respective discoveries.
Can AI Image generation tools make re-imagined, higher-resolution versions of old video game graphics? Over the last few days, I used AI image generation to reproduce one of my childhood nightmares. I wrestled with Stable Diffusion, Dall-E and Midjourney to see how these commercial AI generation tools can help retell…
Augmenting GPT-3 with tools is a popular technique right now. But is GPT-3 really using them?
Just because it matters doesn’t mean it’s defensible or profitable
Diffusion models are trained to reverse a stochastic process through score matching. However, a lot of diffusion models rely on a small but critical implementation detail called thresholding. Thresholding projects the sampling process to the data support after each discretized diffusion step, stabilizing generation at…
The Case for LLMs as Hallucination Engines
Disclaimer: This article mentions https://terra-cotta.ai/, an LLM experimentation platform I am building Introduction ChatGPT, Bard, and other large language models (LLMs) are very useful for a wide variety of tasks from writing code to answering complex questions to aiding with education. However, these models are…
A short simulation of Neal Stephenson's Illustrated Primer using GPT-4 and DALL·E 3.
