I’m a book lover, sci-fi writer, and native plant nerd. Learn more about me , and see what I’m up to now .
Visit tracydurnell.com → RSS feedOn Hacker News Jun 2023 – Dec 2024
Society, Policy & Culture 1Life, Learning & Writing 1AI & Machine Learning 1
I prefer a lightweight nonfiction book to a detailed tome. I’m a dilettante of many interests, so my attention for any given topic is more likely to sustain 100 pages than 600. The sweet spot is longer than a longread internet article, but that doesn’t demand a months-long commitment: a 2-3 hour text.
What will happen to GPT-{n} once LLMs contribute much of the language found online? We find that use of model-generated content in training causes irreversible defects in the resulting models, where tails of the original content distribution disappear. We refer to this effect as Model Collapse and show that it can…