1981
Double Descent in Human Learning
The uncanny resemblance between double descent and a 50 year old theory from psychology
New
The latest blog posts to take off on Hacker News. 3,091 posts.
1982
Capturing the Flag with GPT-4
This weekend I went to BSides SF 2023 and had a blast. I went to some really interesting talks (including an excellent one about adversarial machine learning), but mostly I spent my time solving CTF hacking challenges. And this time, I did it with the help of GPT-4, the
1983
Bing Jail
A short story on how my website lost all indexed pages in Bing, affecting my presence in DuckDuckGo, secret blacklisting and classification as a splog by Bing AI
1984
ProfileGPT: An Example of AI Agents Collaboration Architecture
A dive into the architecture and design of ProfileGPT, an app that allows users to analyze their user's profile and personality as seen by ChatGPT to raise awareness about data usage.
1985
Prompt engineering vs. blind prompting
"Prompt Engineering" emerged from the growth of language models to describe the process of applying prompting to effectively extract information from language models, typically for use in real-world applications.
1986
Finetuning Large Language Models
An introduction to the core ideas and approaches
1987
Large language models: Scaling laws and emergent properties
More details on the science of training large autoregressive language models.
1988
Building a ChatGPT-enhanced Python REPL
In this blog I share my experience in building a Python REPL augmented with ChatGPT. I explore how the application is built, and speculate on software engineering patterns and paradigms that might emerge in systems built on Large Language Models (LLMs). GEPL - Generate, Evaluate, Print, Loop Link to this section…
1989
Record labels panic as A.I. generated song becomes ‘hit single’
Welcome to the new world.
1990
An example of LLM prompting for programming
Generated knowledge and chain of thought prompting of an LLM can generate useful code.
1991
I used Stable Diffusion and Dreambooth to create an art portrait of my dog
In this post, we walk through my entire workflow/process for bringing Stable Diffusion to life as a high-quality framed art print. We’ll touch on making art with Dreambooth, Stable Diffusion, Outpainting, Inpainting, Upscaling, preparing for print with Photoshop, and finally printing on fine-art paper with an Epson…
1992
Understanding large language models: A cross-section of the relevant literature
A Cross-Section of the Most Relevant Literature To Get Up to Speed
1993
33TB of text data for a 1T-parameter model
Important: This page summarizes data scaling only, using tokens to parameters as a ratio, and as derived from large language models like GPT-3, Chinchilla, and beyond, linked to the Compute-Optimal scaling laws like Kaplan and Hoffmann/Chinchilla. Please note that compute scaling laws are outside the scope of my…
1994
The Decade of Deep Learning
As the 2010’s draw to a close, it’s worth taking a look back at the monumental progress that has been made in Deep Learning in this decade.[1] Driven by the development of ever-more powerful comput
1995
Prompt injection: what’s the worst that can happen?
Activity around building sophisticated applications on top of LLMs (Large Language Models) such as GPT-3/4/ChatGPT/etc is growing like wildfire right now. Many of these applications are potentially vulnerable to prompt …
1996
Building LLM Applications for Production
[Hacker News discussion, LinkedIn discussion, Twitter thread]
1997
1998
Do you need a vector database?
Spoiler alert: the answer is maybe! Although, my inclusion of the word “actually” betrays my bias. Vector databases are having their day right now. Three different vector DB companies have raised money on valuations up to $700 million (paywall link). Surprisingly, their rise in popularity is not for their “original”…
1999
Scaffolded LLMs as natural language computers
Recently, LLM-based agents have been all the rage – with projects like AutoGPT showing how easy it is to wrap an LLM in a simple agentic loop and prompt it to achieve real-world tasks. More generally, we can think about the class of ‘scaffolded’ 1 LLM systems – which wrap...
2000
Experimenting with LLMs to Research, Reflect, and Plan
Also, shortcomings in document retrieval and how to overcome them with search & recsys techniques.
2001
Replacing my best friends with an LLM trained on 500k group chat messages
An exploration into customizing LLMs for personal use
2002
GPT-4 gets a B on my quantum computing final exam
Update (April 14): On the suggestion of my TA Justin Yirka, I gave GPT-4 problem 2 from the final exam, along with its own answers and my and Justin’s grading comments, and asked it whether i…
2003
2004
Maximizing the Potential of LLMs: A Guide to Prompt Engineering
Language models have rapidly improved in recent years, with large language models (LLMs) such as GPT-3 and GPT-4 taking center stage. These models have…
2005
Emergent abilities of large language models
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
2006
Teaching ChatGPT to speak my son’s invented language
Becoming fluent in my son’s invented language was too difficult for me, so I decided to see how would AI do.
2007
Why is GPT-3 15.77x more expensive for certain languages?
How large language models process multilingual data differently
2008
I wish GPT4 had never happened
Made in Notion, the connected workspace where AI agents and teams organize docs, projects and knowledge together.
2009
2010
We need to tell people ChatGPT will lie to them, not debate linguistics
ChatGPT lies to people. This is a serious bug that has so far resisted all attempts at a fix. We need to prioritize helping people understand this, not debating the …