1
RedPajama v2 Open Dataset with 30T Tokens for Training LLMs
RedPajama-Data-v2 is a 30 trillion token open dataset from 84 CommonCrawl dumps across five languages, with over 40 precomputed quality annotations.
Build what's next on the AI Native Cloud. Full-stack AI platform for inference, fine-tuning, and GPU clusters — powered by cutting-edge research.
Visit together.ai → RSS feedOn Hacker News Oct 2023 – Mar 2026
AI & Machine Learning 8Systems & Performance 4Programming Languages 1Databases & Data 1
RedPajama-Data-v2 is a 30 trillion token open dataset from 84 CommonCrawl dumps across five languages, with over 40 precomputed quality annotations.