Skip to content

Theia Vogel personal blog

I made a transformer to predict a simple sequence manually

vgel.me

To better understand how transformers work, I hand-assigned all the weights to predict a simple sequence.

Read on vgel.me → Discuss on HN (94) Archived copy

Points: 360
HN comments: 94
Submissions: 1
Cited by: 0 commenters

AI & Machine Learning Software Engineering 2023

On Hacker News

Submitted	Title on HN	Points	Comments
Sep 22, 2023	I made a transformer to predict a simple sequence manually by lukastyrychtr	360	94

More from Theia Vogel

All 8 posts →

Why do LLMs freak out over the seahorse emoji?

2025734 points416 comments AI & Machine Learning details

Writing a C compiler in 500 lines of Python

2023510 points274 comments Programming Languages details

Representation Engineering: Mistral-7B on Acid

2024360 points65 commentscited by 11AI & Machine Learning details

Signed distance functions in 46 lines of Python

2022283 points24 comments Math & Algorithms details

How to make LLMs go fast

2023215 points54 comments AI & Machine Learning details

GPT-3 will ignore tools when it disagrees with them

2023177 points166 commentscited by 4AI & Machine Learning details

Related posts from the same era

Bing: “I will not harm you unless you harm me first”

Simon Willison20233,363 points1,673 commentscited by 6 details

GPT-4o

OpenAI Blog20243,138 points2,366 commentscited by 16 details

Open source AI is the path forward

Meta Newsroom20242,360 points887 commentscited by 5 details

Building a Virtual Machine Inside ChatGPT

Jonas Degrave20222,029 points908 commentscited by 16 details

The man who killed Google Search?

Ed Zitron20241,884 points878 commentscited by 47 details

ChatGPT Plugins

OpenAI Blog20231,875 points1,106 commentscited by 14 details