181
Manticore 6.0.0 – a faster alternative to Elasticsearch in C++
Today, the Manticore team is thrilled to announce the release of Manticore Search 6.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 371 posts.
182
The growing pains of database architecture
How the Figma infrastructure team reduced potential instability by scaling to multiple databases
183
Are You Sure You Want to Use MMAP in Your Database Management System? (2022)
MMAP Databases = 💩
184
Updates to the H2O.ai db-benchmark
The H2O.ai db-benchmark has been updated with new results. In addition, the AWS EC2 instance used for benchmarking has been changed to a c6id.metal for improved repeatability and fairness across libraries. DuckDB is the fastest library for both join and group by queries at almost every data size.
185
The magic of small databases
Notes on personal libraries, collections and small indexes on the web
186
Faster LZ is not the answer to 150-250 GB video game downloads
When the JPEG folks were working on image compression, they didn't create a better or faster LZ. Instead they developed new approaches. I se...
187
Maxar's Open Satellite Feed
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
188
Prolog for data science
I demonstrate a widely applicable pattern which integrates Prolog as a critical component in a data science analysis. Analytic methods are used to generate properties about the data under study and Prolog is used to reason about the data via the generated properties. The post includes some examples of piece-wise…
189
Polars: Company Formation Announcement
We are excited to announce the start of company that will build around Polars, enabling data processing at any scale.
190
You cannot have exactly-once delivery (2015)
I’m often surprised that people continually have fundamental misconceptions about how distributed systems behave. I myself shared many of these misconceptions, so I try not to demean or dismiss but rather educate and enlighten, hopefully while sounding less preachy than that just did. I continue to learn only by…
191
Introduction to vector similarity search (2022)
Learn what vector search is and the metrics pertinent to decide the distance (or similarity) between objects.
192
DuckDB 0.8
The DuckDB team is happy to announce the latest DuckDB release (0.8.0). This release is named “Fulvigula” after the Mottled Duck (Anas Fulvigula) native to the Gulf of Mexico. To install the new version, please visit the installation guide. The full release notes can be found on GitHub. There have been too many…
193
The many faces of DISTINCT in Postgres (2017)
3 interesting uses of DISTINCT in PostgreSQL
194
PostgreSQL Encryption: The Available Options
On an episode of Postgres FM , the hosts had a (very brief) discussion of data encryption in PostgreSQL. While Postgres FM is a podcast well worth a subscribe, the hosts aren’t data security experts, and so as someone who builds a queryable database encryption system , I found the coverage to be somewhat… lacking. I…
195
Tips for a healthier Postgres database (2021)
Craig's year end wrap-up of 5 things you can do for a healthier Postgres database next year.
196
Pg_later: Asynchronous Queries for Postgres
Engineering deep-dives, product updates, and company news from Tembo — the platform for running coding agents across your stack.
197
Pyroscope and Grafana Phlare join together
We’re happy to announce that Pyroscope, the company behind the open source continuous profiling project of the same name, is now part of Grafana Labs.
198
Building ClickHouse Cloud from scratch in a year
Have you ever wondered what it takes to build a serverless software as a service (SaaS) offering in under a year? In this blog post, we will describe how we built ClickHouse Cloud from the ground up
199
33TB of text data for a 1T-parameter model
Important: This page summarizes data scaling only, using tokens to parameters as a ratio, and as derived from large language models like GPT-3, Chinchilla, and beyond, linked to the Compute-Optimal scaling laws like Kaplan and Hoffmann/Chinchilla. Please note that compute scaling laws are outside the scope of my…
200
201
Faster MySQL with HTTP/3
In this article we explore how our HTTP/3 API compares to the latency of a traditional MySQL client.
202
The Case of a Curious SQL Query
Languages that suffer success often have to do so by selling out and adding features that go against some of the original purposes of their design. SQL is a...
203
204
205
OpenTelemetry at Scale: Using Kafka to handle bursty traffic
Learn how to use Apache Kafka as a buffer between OpenTelemetry Collectors for a scalable multi-collector pipeline, with YAML configuration examples.
206
Bank failures come in waves
Interactive post about the frequency of bank failures over time
207
Operating on a minimal two-core Postgres instance: Query optimization insights
In an era where the costs for cloud services continue to soar, I want to outline how our data team has successfully managed to run the infrastructure of our energy trading company within the first year using only two PostgreSQL database instances - each equipped with just two CPU cores and four GB of RAM. Despite this…
208
Most companies do not need Snowflake or Databricks
The cost for something that can be replicated free and open source is absurd. The Fortune 100 have a use case for these companies, the rest…
209
Ways to shoot yourself in the foot with Redis
Personal website of Phil Booth
210
CRDT Benchmarks
Open-source TypeScript library for collaborative JSON editing powered by CRDTs.