Alan D. Thompson personal blog
33TB of text data for a 1T-parameter model
Important: This page summarizes data scaling only, using tokens to parameters as a ratio, and as derived from large language models like GPT-3, Chinchilla, and beyond, linked to the Compute-Optimal scaling laws like Kaplan and Hoffmann/Chinchilla. Please note that compute scaling laws are outside the scope of my…
- Points
- 185
- HN comments
- 150
- Submissions
- 1
- Cited by
- 0 commenters
AI & Machine LearningDatabases & Data2023
On Hacker News
| Submitted | Title on HN | Points | Comments |
|---|---|---|---|
| Apr 15, 2023 | 33TB of text data for a 1T-parameter model by ofou | 185 | 150 |
