Posts that commenters keep linking to, ranked by how many different people have cited them in Hacker News comments. 155 posts.
Stephen Wolfram explores the broader picture of what's going on inside ChatGPT and why it produces meaningful text. Discusses models, training neural nets, embeddings, tokens, transformers, language syntax.
I sometimes see people refer to neural networks as just “another tool in your machine learning toolbox”. They have some pros and cons, they…
One piece of slang that has long embodied the short attention span Internet age is TL;DR, short for “too long; didn’t read.” With the explosion of generative AI tools, we’re rapidly entering the age of TL;DW: “too long, didn’t write.” A January survey from Fishbowl found that 40% of nearly 12,000 workers have used…
Fictional short story about Clippy & AI hard takeoff scenarios grounded in contemporary ML scaling, self-supervised learning, reinforcement learning, and meta-learning research literature.
We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.
Here, I provide an in-depth analysis of GPUs for deep learning/machine learning and explain what is the best GPU for your use-case and budget.
I find this literature irritating and opaque. This is at least somewhat because I do not yet understand it well, and there's too much of it. But clearly I need to wrap my head around it, before I become technically obsolete. My scare quotes in the title of these notes thus derive in part from jealousy and fear. But…
Instruction-following models such as GPT-3.5 (text-davinci-003), ChatGPT, Claude, and Bing Chat have become increasingly powerful. Many users now interact with these models regularly and even use them for work. However, despite their widespread deployment, instruction-following models still have many deficiencies…
We’re announcing updates including more steerable API models, function calling capabilities, longer context, and lower prices.
We’ve implemented initial support for plugins in ChatGPT. Plugins are tools designed specifically for language models with safety as a core principle, and help ChatGPT access up-to-date information, run computations, or use third-party services.
It’s well-known at this point that GPT-4/GPT-3.5-turbo is non-deterministic, even at temperature=0.0. This is an odd behavior if you’re used to dense decoder-only models, where temp=0 should imply greedy sampling which should imply full determinism, because the logits for the next token should be a pure function of…
An update on our findings, the actions we’ve taken, and technical details of the bug.
Wolfram plugin gives computationally accurate answers to ChatGPT queries using Wolfram's data knowledgebase and language. Custom visualizations are given as well. Stephen Wolfram explains how it works.
I procrastinated a deep dive into transformers for a few years. Finally the discomfort of not knowing what makes them tick grew too great for me. Here is that dive.
Today, we’re releasing our LLaMA (Large Language Model Meta AI) foundational model with a gated release. LLaMA is more efficient and competitive with previously published models of a similar size on existing benchmarks.
A statement jointly signed by a historic coalition of experts: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Yesterday, Sam Altman and Greg Brockman were fired from the Board of Directors of OpenAI.
Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available for commercial use, and matches the quality of LLaMA-7B. MPT-7B was trained on the MosaicML platform in 9.5 days with zero human intervention…
We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models…
On GPT-3: meta-learning, scaling, implications, and deep theory. The scaling hypothesis: neural nets absorb data & compute, generalizing and becoming more Bayesian as problems get harder, manifesting new abilities even at trivial-by-global-standards-scale. The deep learning revolution has begun as foretold.
Developers can now integrate GPT‑3.5 Turbo and Whisper models into their apps and products through our API.
We’re launching a classifier trained to distinguish between AI-written and human-written text.
I really want an AI assistant: a Large Language Model powered chatbot that can answer questions and perform actions for me based on access to my private data and tools. …
AI folklore tells a story about a neural network trained to detect tanks which instead learned to detect time of day; investigating, this probably never happened.
A machine learning researcher writes me in response to yesterday’s post, saying:I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you bac…
Implementing a GPT model from scratch in NumPy.
Introducing Bard (now Gemini), Google's conversational AI service — plus, new AI features in Search.
Here we go again. I started another company. The money is in the bank.
Proving you're a human on a web flooded with generative AI content
