Yi Tay

In 2025, I helped train the first Deep Think model that achieved the historic IMO gold medal . In 2026, I co-invented a new paradigm that is widely adopted in latest Gemini releases (Gemini 4 Argon, 3.8 Flash etc).

Popular posts: 2
Total points: 680
HN comments: 182
Rank: #3,587 personal

Visit yitay.net → On Hacker News Mar 2024 – Jul 2024

AI & Machine Learning 2 Startups 1

Most points Most discussed Most cited Newest

Training LLMs from ground zero as a startup

2024429 points114 comments AI & Machine Learning details

Chronicles of training strong LLMs from scratch in the wild

What happened to BERT and T5?

2024251 points68 comments AI & Machine Learning details

A Blogpost series about Model Architectures Part 1: What happened to BERT and T5? Thoughts on Transformer Encoders, PrefixLM and Denoising objectives