1
Attention Is Off By One
Let’s fix these pesky Transformer outliers using Softmax One and QuietAttention.
Evan Miller
Articles, software, calculators, and opinions.
- Popular posts
- 22
- Total points
- 6,957
- HN comments
- 3,733
- Rank
- #133 personal
Visit evanmiller.org → RSS feedOn Hacker News Apr 2010 – Jul 2023
Programming Languages 8Software Engineering 6Math & Algorithms 5Tech Industry & Business 3Web & Frontend 2Tools & Productivity 2
2
Statistical Formulas For Programmers
When someone says “7 is greater than 5”, you declare that they’re really the same.
3
Elixir RAM and the Template of Doom
I will attempt to convince you, in two lines of code, that Elixir is more interesting than any programming language you’ve ever used.
4
The Mathematical Hacker
The programming profession is blessed with a number of gifted essayists. Today I will pick on three of my favorites — Eric Raymond, Paul Graham, and Steve Yegge — because they all seem to disagree about why (and whether) mathematics is relevant to the practicing programmer. Their attitudes might be summarized as:
5
Type Punning Functions in C
C has a reputation for being inflexible. But did you know you can change the argument order of C functions if you don't like them?
6
The Floppy Disk of Floating Point
An essay that bids farewell to x87 – a computing architecture too long for this world.