2101
I had ChatGPT write a Decemberists song and recorded it
And I recorded it. Here's "Sailor's Song."
New
The latest blog posts to take off on Hacker News. 3,092 posts.
2102
2103
How to train large models on many GPUs? (2021)
[Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog: “Techniques for Training Large Neural Networks”
2104
Understanding and coding the self-attention mechanism of large language models
Step-by-step tutorial explaining scaled dot-product self-attention for large language models, with Python code that builds the mechanism from scratch.
2105
A GPT in 60 Lines of NumPy
Implementing a GPT model from scratch in NumPy.
2106
We come to bury ChatGPT, not to praise it
Large language models (LLMs) like the GPT family learn the statistical structure of language by optimising their ability to predict missing words in sentences (as in ‘The cat sat on the [BLANK]’). Despite the impressive technical ju-jitsu of transformer models and the billions of parameters they learn, it’s still a…
2107
Run CLIP on iPhone to search photos
How I run CLIP (ViT-B-32) on iPhone to search the Photos album offline with Queryable: exporting Core ML text and image encoders for CLIP photo search
2108
I used GPT to build a search tool for my second brain note-taking system
In this post, I discuss how I used GPT embeddings to build a smart search tool for my second brain note-taking system.
2109
Artificial intelligence just lost a leader
Plus a position-search announcement from NSF Roger Schank just passed away. Roger was a top leader of AI. I overlapped with him for my time at Yale. In 1974, he became a professor of computer scien…
2110
Understanding the limits of large language models
Reading level: No programming knowledge required, but trying ChatGPT first is advised.
2111
ChatGPT: The Slickest Con Artist of All Time
Or 10 tweets about ChatGPT
2112
2113
The unequal treatment of demographic groups by ChatGPT/OpenAI content moderation
Should AI systems treat different demographic groups unequally?
2114
Anki and GPT-3
Tools for thought, programming, games, homeschooling, and learning
2115
ChatGPT is a bullshit generator but it can still be amazingly useful
The philosopher Harry Frankfurt defined bullshit as speech that is intended to persuade without regard for the truth. By this measure, OpenAI’s new chatbot ChatGPT is the greatest bullshitter ever. Large Language Models (LLMs) are trained to produce
2116
LLM discourse needs more nuance
With this write-up, I want to outline my current thinking around large language models like OpenAI's chatGPT.
2117
C++ Neural Network in a Weekend (2020)
A blog about game engines, graphics, C++, and more.
2118
Training NanoGPT on My Journal
Wouldn't it be great to have a pet AI chat bot that knows you really well, and can give you free consulting, coaching and therapy ? What could possibly go wrong if an intelligent corporate-controlled system has unlimited access to your private data to provide you the best possible service? Well... a lot of things.
2119
The Transformer Family
Many new Transformer architecture improvements have been proposed since my last post on “The Transformer Family” about three years ago. Here I did a big refactoring and enrichment of that 2020 post — restructure the hierarchy of sections and improve many sections with more recent papers. Version 2.0 is a superset of…
2120
Natural language is the lazy user interface
A chatbot is the lazy user interface. It puts all the burden on the user to articulate good questions.
2121
Just know stuff (or, how to achieve success in a machine learning PhD)
Introduction So I recently completed my PhD in Mathematics from the University of Oxford. (Hurrah! It was so much fun.) In 2-and-a-bit years I wrote 12 papers, received 4139 GitHub stars, got 3271 Twitter followers, authored 1 textbook – doing double-duty as my thesis – and got the coveted big-tech job-offer. On…
2122
The continued relevance of Weizenbaum’s warnings
“The computer has long been a solution looking for problems—the ultimate technological fix which insulates us from having to look at problems.” – Joseph Weizenbaum (1983) Trying to kee…
2123
Certified 100% AI-free organic content
This is a last call to introduce a "made by an organically-sourced human" label.
2124
Bypassing Gmail's spam filters with ChatGPT
If you haven’t been living under a rock, you probably already know what ChatGPT is. I am an operator of high bandwidth Tor exit relays, and with that, I get a lot of abuse complaints. Combined with the fact that I have ai lease on a “/24” of IPv4 space, basically a block of 256 IP addresses, I have the IP space’s…
2125
How to implement Q&A against your docs with GPT3 embeddings and Datasette
If you’ve spent any time with GPT-3 or ChatGPT, you’ve likely thought about how useful it would be if you could point them at a specific, current collection of text …
2126
ChatGPT in an iOS Shortcut: Smart HomeKit Voice Assistant
Ever since I tried ChatGPT and GPT-3, everything else feels painfully dumb and useless: Siri, Alexa, Google Home and all other “smart”…
2127
AI is in danger of being swallowed up by copyright law
Marc Andreessen famously said that software is eating the world. But the latest and greatest software trend–generative AI–is in danger of being swallowed up by copyright law. Like a cruise ship hea…
2128
Large Transformer Model Inference Optimization
[Updated on 2023-01-24: add a small section on Distillation.] Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to train and use. The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful…
2129
Gptcommit: Never write a commit message again (with the help of GPT-3)
With gptcommit, you no longer have to spend time writing your commit messages. Install gptcommit and let GPT-3 do the writing for you.
2130
Which GPU(s) to get for deep learning
Here, I provide an in-depth analysis of GPUs for deep learning/machine learning and explain what is the best GPU for your use-case and budget.