Skip to content

Databricks Blog company blog

Spark Breaks Previous Large-Scale Sort Record

by @databricks databricks.com published Fri, 10/10

Read how Apache Spark sorted 100 TB of data (1 trillion records) 3X faster using 10X fewer machines in 2014.

Read on databricks.com → Discuss on HN (56) Archived copy

Points: 142
HN comments: 56
Submissions: 1
Cited by: 0 commenters

Databases & Data Systems & Performance 2014

On Hacker News

Submitted	Title on HN	Points	Comments
Oct 10, 2014	Spark Breaks Previous Large-Scale Sort Record by metronius	142	56

More from Databricks Blog

All 23 posts →

DBRX: A new open LLM

2024866 points343 comments AI & Machine Learning details

Hello Dolly: Democratizing the magic of ChatGPT with open models

2023494 points173 comments AI & Machine Learning details

Databricks acquires Neon

2025389 points226 comments Tech Industry & Business details

Managing AI Coding Costs at Scale

2026317 points268 comments AI & Machine Learning details

Spark 2.0 Technical Preview

2016259 points36 comments Databases & Data details

Spark as a Compiler: Joining a Billion Rows per Second on a Laptop

2016251 points53 commentscited by 4Databases & Data details

Related posts from the same era

Visualizing Algorithms

Mike Bostock20141,557 points129 commentscited by 38 details

Command-line tools can be faster than your Hadoop cluster

Adam Drake2015812 points480 commentscited by 23 details

Goodbye MongoDB, Hello PostgreSQL

Olery Developer Blog2015802 points545 comments details

PostgreSQL 9.4 Released

PostgreSQL2014744 points188 comments details

Don't use Hadoop when your data isn't that big

Chris Stucchio2013711 points611 commentscited by 15 details

ASCII Delimited Text – Not CSV or TAB delimited text

Ronald Duncan2014695 points397 commentscited by 8 details