For most people I interact with, I’m just another text-based program for the most of the time. If input and output are so simple, could I be replaced by the model? For this to work, the model would need to not only understand my writing style but also know a lot about me. The best source for this is my Telegram…
[Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neural Networks”
Mistral AI is the most exciting AI research lab at the moment. They’ve now released two extremely powerful smaller Large Language Models under an Apache 2 license, and have a …
Blog about linguistics, programming, and my projects
ChatGPT and generative AI will change how we work, but how different is this to all the other waves of automation of the last 200 years? What does it mean for employment? Disruption? Coal consumption?
Prompt injection remains an unsolved problem. The best we can do at the moment, disappointingly, is to raise awareness of the issue. As I pointed out last week, “if you …
Discussions: Hacker News (64 points, 3 comments), Reddit r/MachineLearning (219 points, 18 comments) Translations: Simplified Chinese, French, Korean, Russian, Turkish This year, we saw a dazzling application of machine learning. The OpenAI GPT-2 exhibited impressive ability of writing coherent and passionate essays…
It shouldn’t be surprising that ChatGPT can play chess.
And I recorded it. Here's "Sailor's Song."
Tools for thought, programming, games, homeschooling, and learning
There are plenty of jobs, but finding a place where you're happy is as hard as ever.
If you haven’t been living under a rock, you probably already know what ChatGPT is. I am an operator of high bandwidth Tor exit relays, and with that, I get a lot of abuse complaints. Combined with the fact that I have ai lease on a “/24” of IPv4 space, basically a block of 256 IP addresses, I have the IP space’s…
Lately, I've been playing around with LLMs to write code.
I’ve been building out a small suite of command-line tools for working with ChatGPT, GPT-4 and potentially other language models in the future. The three tools I’ve built so far …
I trusted a lot today. I trusted my phone to wake me on time. I trusted Uber to arrange a taxi for me, and the driver to get me to the airport safely. I trusted thousands of other drivers on the road not to ram my car on the way. At the airport, I trusted ticket agents and maintenance engineers and everyone else who…
This post covers my experience training image classification models with Intel’s PyTorch extension on the Arc A770 GPU.
Update (April 14): On the suggestion of my TA Justin Yirka, I gave GPT-4 problem 2 from the final exam, along with its own answers and my and Justin’s grading comments, and asked it whether i…
I really want an AI assistant: a Large Language Model powered chatbot that can answer questions and perform actions for me based on access to my private data and tools. …
Spoiler alert: the answer is maybe! Although, my inclusion of the word “actually” betrays my bias. Vector databases are having their day right now. Three different vector DB companies have raised money on valuations up to $700 million (paywall link). Surprisingly, their rise in popularity is not for their “original”…
One of the most pervasive mistakes I see people using with large language model tools like ChatGPT is trying to use them as a search engine. As with other LLM …
Suppose that we sample a large number N N N of independent random functions f i : R → R f_i \colon \R \to \R f i : R → R from a certain distribution F \Fc F and propose to solve a regression problem by choosing a linear combination f ˉ = ∑ i λ i f i . \bar{f} = \sum_i \lambda_i f_i. f ˉ = i ∑ λ i f i . For…
GM’s Cruise aims to turn self-driving into a billion-dollar business.
Google A/I suggests that AI is a sustaining innovation for all of Big Tech; that means the real battle will be between incumbents and Big Tech on one side, and open source on the other.
The big news today is Llama 2, the new openly licensed Large Language Model from Meta AI. It’s a really big deal: Llama 2 is available for free for research …
An interview with Nvidia CEO Jensen Huang about the impact of ChatGPT, Nvidia’s new cloud service, and how Nvidia is adapting to new geopolitical and competitive realities.
You shouldn't bet against the bazaar or the GPU p̶o̶o̶r̶ hungry.
Mozilla’s innovation group and Justine Tunney just released llamafile, and I think it’s now the single best way to get started running Large Language Models (think your own local copy …
I gave a talk on Sunday at North Bay Python where I attempted to summarize the last few years of development in the space of LLMs—Large Language Models, the technology …
