A recent paper The Reversal Curse points out an apparent failure in large large language models like GPT-4. From the abstract: We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is
Here, I provide an in-depth analysis of GPUs for deep learning/machine learning and explain what is the best GPU for your use-case and budget.
An introduction to the core ideas and approaches
GigaGPT is Cerebras’ implementation of Andrei Karpathy’s nanoGPT – the simplest and most compact code base to train and fine-tune GPT models.
GPT4-V is the new mode of GPT-4 that allows you to upload images as part of your conversations. It’s absolutely brilliant. It also provides a whole new set of vectors …
On Saturday 11th March I wrote about how Large language models are having their Stable Diffusion moment. Today is Monday. Let’s look at what’s happened in the past three days. …
Microsoft has made tons of noise about their Microsoft 365 Copilot toolset. They’ve made a lot of promises about its capabilities to enhance individual and team productivity, but access to it…
StripedHyena-7B is an open-source model built beyond the Transformer architecture, matching Llama-2 and Mistral 7B while running faster on long sequences.
Text-to-image diffusion models can create stunning images from natural language descriptions that rival the work of professional artists and photographers. However, these models are large, with complex network architectures and tens of denoising iterations, making them computationally expensive and slow to run. As a…
At Replit, our mission is to empower the next billion software creators. Yesterday, we strengthened our commitment by announcing that Replit AI is now free for all users. Over the past year, we’ve witnessed the transformative power of building software collaboratively with the power of AI. We believe AI will be part…
We need scientific and technical breakthroughs to steer and control AI systems much smarter than us. To solve this problem within four years, we’re starting a new team, co-led by Ilya Sutskever and Jan Leike, and dedicating 20% of the compute we’ve secured to date to this effort. We’re looking for excellent ML…
Voicebox is a state-of-the-art speech generative model based on a new method proposed by Meta AI called Flow Matching. By learning to solve a text-guided speech infilling task with a large scale of data, Voicebox outperforms single-purpose AI models across speech tasks through in-context learning.
The massive false positive rate of general AI detectors had a devastating effect on freelance writer Michael Berben: being falsely accused of cheating, he lost his job.
Why generative AI tools can be so difficult to use and how we might improve them
If OpenAI was a public company, it's stock would have been down about 38% today in a shocking turn of events.
For most people I interact with, I’m just another text-based program for the most of the time. If input and output are so simple, could I be replaced by the model? For this to work, the model would need to not only understand my writing style but also know a lot about me. The best source for this is my Telegram…
[Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neural Networks”
Unlock new experiences with Microsoft Teams Premium for customers looking to make meetings more intelligent, personalized, and protected.
Mistral AI is the most exciting AI research lab at the moment. They’ve now released two extremely powerful smaller Large Language Models under an Apache 2 license, and have a …
How to run a latent consistency model on your M1 or M2 Mac
Blog about linguistics, programming, and my projects
ChatGPT and generative AI will change how we work, but how different is this to all the other waves of automation of the last 200 years? What does it mean for employment? Disruption? Coal consumption?
Prompt injection remains an unsolved problem. The best we can do at the moment, disappointingly, is to raise awareness of the issue. As I pointed out last week, “if you …
Discussions: Hacker News (64 points, 3 comments), Reddit r/MachineLearning (219 points, 18 comments) Translations: Simplified Chinese, French, Korean, Russian, Turkish This year, we saw a dazzling application of machine learning. The OpenAI GPT-2 exhibited impressive ability of writing coherent and passionate essays…
It shouldn’t be surprising that ChatGPT can play chess.
