181
Production RAG: what I learned from processing 5M+ documents
Lessons learned from building RAG systems for Usul AI and enterprise clients, processing over 13 million pages.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 4,222 posts.
182
Things I learned from building a production database
In 2017, I went to Facebook on a sabbatical from my faculty position at Yale. I created a team to build a storage system called Delos at the bottom of the Facebook stack (think of it as Facebook’s version of Chubby). We hit production with a 3-person team in less than a year; and subsequently scaled the team to 30+…
183
SQLite the only database you will ever need in most cases
The name SQLite is a nice name, but the "lite" part is misleading, it sounds like it is only useful for tiny things - which is very wrong. SQLite should be named AwesomeSQL, because that is what it is. SQLite is probably the only database you will ever need in most cases.
184
We analyzed 425k favicons
The humble favicon was messily birthed with the pernicious IE5 release. Since that fateful day, browsers have slowly expanded favicon technology to encompass many wildly differing and lightly documented use cases. Here in 2021 favicons are found primarily in browser tabs, home screens, and Google search results, but…
185
The Untold Story of SQLite
On today's show, I'm talking to Richard Hipp about surviving becoming core infrastructure for the world. SQLite is everywhere. It's in your web browser, it's in your phone, it's probably in your car, and it's definitely in commercial planes. It's where your iMessages and WhatsApp messages are stored, and if you do a…
186
SQLite in the browser with WASM/JS
WebAssembly , a.k.a. WASM, is a standard defining a low-level programming language suitable (A) as a target for cross-compilation from many other languages and (B) for running via a virtual machine in a browser. Designed with scriptability via JavaScript in mind, it provides a way to compile C code (among others) to…
187
Build your own database
A step-by-step guide to building a key-value database from scratch.
188
AI didn't delete your database, you did
Last week, a tweet went viral showing a guy claiming that a Cursor/Claude agent deleted his company's production database. We watched from the sidelines as he tried to get a confession from the agent:
189
A summary of what quantitative trading firms do
This summary is an attempt to shed some light on modern quantitative trading since there is limited information available for people who are not already in the industry. Hopefully this is useful for students and candidates coming from outside the industry who are looking to understand what it’s like working for a…
190
Many small queries are efficient in SQLite
200 SQL statements per webpage is excessive for client/server database engines like MySQL, PostgreSQL, or SQL Server.
191
192
Retrieving 1TB of data from a faulty drive with the help of woodworking tools
So, a while ago I built a gaming PC with the following specification: Case Dan A4 v4.1 Motherboard Z690I Strix Gaming Pro...
193
Osquery: Expose the operating system as a relational database
Visit the post for more.
194
It's 2026, Just Use Postgres
Stop managing multiple databases. Postgres extensions replace Elasticsearch, Pinecone, Redis, MongoDB, and InfluxDB with BM25, vectors, JSONB.
195
Joins 13 Ways
Relational (inner) joins are really common in the world of databases, and one weird thing about them is that it seems like everyone has a different idea of what they are. In this post I’ve aggregated a bunch of different definitions, ways of thinking about them, and ways of implementing them that will hopefully be…
196
Creating a search engine with PostgreSQL
PostgreSQL provides the necessary building blocks for you to combine and create your own search engine for full-text search. Let's see how far we can take it.
197
The database servers powering Let's Encrypt
Let’s Encrypt helps to protect a huge portion of the Web by providing TLS certificates to more than 235 million websites. A database is at the heart of how Let’s Encrypt manages certificate issuance. If this database isn’t performing well enough, it can cause API errors and timeouts for our subscribers. Database…
198
Michael Stonebraker wins Turing Award
Michael Stonebraker, an MIT researcher who has revolutionized the field of database management systems and founded multiple successful database companies, has won the Association for Computing Machinery's $1 million A.M. Turing Award, often referred to as “the Nobel Prize of computing.”
199
200
I’m writing a new vector search SQLite Extension
sqlite-vec is an new vector search SQLite extension, coming soon!
201
Amazon S3 Adds Put-If-Match (Compare-and-Swap)
Discover more about what's new at AWS with Amazon S3 adds new functionality for conditional writes
202
The startup's Postgres survival guide
A guide to preventing Postgres from toppling over.
203
204
Migrating Facebook to MySQL 8.0
MySQL, an open source database developed by Oracle, powers some of Facebook’s most important workloads. We actively develop new features in MySQL to support our evolving requirements. These feature…
205
PostgreSQL Scalability: Towards Millions TPS
PostgreSQL scalability on multicore and multisocket machines became a subject of optimization long time ago once such machines became widely used. This blog post shows brief history of vertical scalability improvements between versions 8.0 and 8.4. PostgreSQL 9.2 had very noticeable scalability improvement. Thanks to…
206
Analyzing Cryptocurrencies Using PostgreSQL
Cryptocurrencies are fueling a modern day gold rush. Can data help us better understand this evolving market? Read the full article on the Tiger Data blog.
207
Comparing Database Types
Many types of databases exist, each with their own benefits. In this guide, we compare the different types of databases and what each of them offer.
208
Accounting for Computer Scientists
Every educated person really ought to have a basic understanding of accounting. Just like maths, science, programming, music, literature, history, etc., it’s one of those things which helps you make sense of the world. Although dealing with money is not much fun, it’s an unavoidable part of life, so you might as well…
209
210
New Features Coming in PostgreSQL 10
The list of new features coming in PostgreSQL 10 is extremely impressive. I've been involved in the PostgreSQL project since the 8.4 releas...