751
Making a Postgres query 1k times faster
Learn how we discovered what was slowing down large database queries, and what we did to make Postgres queries 1000x faster.
New
The latest blog posts to take off on Hacker News. 4,224 posts.
752
Jepsen: Datomic Pro 1.0.7075
Datomic is a general-purpose database intended for systems of record. In many ways, Datomic is unusual. At any instant in time, the state of the database is represented by a set of [entity, attribute, value] ( EAV ) triples, known as datoms . Each datom declares that some entity (like a person) has a particular…
753
Optimizing ClickHouse: Tactics that worked for us
At LaunchDarkly, we ingest nearly 100 terabytes of observability data from our customers every month.
754
HBase Deprecation at Pinterest
Alberto Ordonez Pereira | Senior Staff Software Engineer; Lianghong Xu | Senior Manager, Engineering;
755
The notifier pattern for applications that use Postgres
Maximizing Postgres connection economy by using a single connection per program to receive and distribute all listen/notify notifications.
756
Amazon S3 will no longer charge for several HTTP error codes
Discover more about what's new at AWS with Amazon S3 will no longer charge for several HTTP error codes
757
VideoPrism: A foundational visual encoder for video understanding
Posted by Long Zhao, Senior Research Scientist, and Ting Liu, Senior Staff Software Engineer, Google Research An astounding number of videos are av...
758
Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x
TL;DR: LLMs have been traditionally regarded as sequential decoders, decoding one token after another. In this blog, we show pretrained LLMs can be easily taught to operate as efficient parallel decoders. We introduce Consistency Large Language Models (CLLMs), a new family of parallel decoders capable of reducing…
759
The Birth of Parquet
15 years ago (2007-2011) I was at Yahoo! working with Map/Reduce and Apache Pig, which was the better Map/Reduce at the time. The Dremel paper just came out and, as everything I worked with seemed to be inspired from Google papers, I read it. I could see it applying to what we were doing at Yahoo! and this was to…
760
Deno KV internals: building a database for the modern web
How we built a performant, scalable, ACID-compliant, JavaScript-native database on FoundationDB.
761
Data Science at the Command Line, 2nd Edition (2021)
This thoroughly revised guide demonstrates how the flexibility of the command line can help you become a more efficient and productive data scientist. You’ll learn how to combine small yet powerful command-line tools to quickly obtain, scrub, explore, and model your data. To get you started, author Jeroen Janssens…
762
I’m writing a new vector search SQLite Extension
sqlite-vec is an new vector search SQLite extension, coming soon!
763
Backblaze Drive Stats for Q1 2024
Analyze Backblaze's Q1 2024 hard drive stats, including annualized failure rates, average ages of failure, and insights on over 279,000 drives.
764
Hacking on PostgreSQL is hard
Hacking on PostgreSQL is really hard. I think a lot of people would agree with this statement, not all for the same reasons. Some might poin...
765
Why SQLite Uses Bytecode
Every SQL database engine works in roughly the same way: It first translates the input SQL text into a "prepared statement". Then it "executes" the prepared statement to generate a result.
766
You can't just assume UTF-8
How to infer character encodings with statistics
767
Common DB schema change mistakes in Postgres
Common DB schema change mistakes and how to avoid them in PostgreSQL development.
768
Building an open data pipeline in 2024
Using Iceberg allows us to pick the optimal "big data" compute environment for the specific requirements we have. There's no need to limit yourself to a single solution.
769
770
A Logic Language for Distributed SQL Queries
A comprehensive look at the design of Polar, and how we enabled it to operate over distributed data.
771
MemoryDB: A fast and durable memory-first cloud database
Amazon MemoryDB for Redis is a database service designed for 11 9s of durability with in-memory performance. In this paper, we describe the architecture of MemoryDB and how we leverage open-source Redis, a popular data structure store, to build an enterprise-grade cloud database. MemoryDB offloads…
772
The "it" in AI models is the dataset
I’ve been at OpenAI for almost a year now. In that time, I’ve trained a lot of generative models. More than anyone really has any right to train. As I’ve spent these hours observing the effects of tweaking various model configurations and h
773
774
775
AI for Data Journalism: demonstrating what we can do with this stuff
I gave a talk last month at the Story Discovery at Scale data journalism conference hosted at Stanford by Big Local News. My brief was to go deep into the …
776
Setting up PostgreSQL for running integration tests
Setting up PostgreSQL for tests is crucial for achieving performance and reliability. In this article, I'll explain how to set up PostgreSQL for tests and discuss some common pitfalls to avoid.
777
Multi-tenant queues in Postgres
How we implemented fair queueing strategies for Postgres-backed task queues to ensure fair execution time across tenants.
778
Ten years of improvements in PostgreSQL's optimizer
Ryan Marcus, assistant professor at the University of Pennsylvania. Using machine learning to build the next generation of data systems.
779
Redis is forked
Where do we go from here?
780
