1
Why SQLite Uses Bytecode
Every SQL database engine works in roughly the same way: It first translates the input SQL text into a "prepared statement". Then it "executes" the prepared statement to generate a result.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 185 posts.
2
Slack AI Training with Customer Data
Slack's Terms and Policies, including privacy, Terms of Service, API terms, security, and more.
3
PostgreSQL 17
The [PostgreSQL Global Development Group](https://www.postgresql.org) today announced the release of [PostgreSQL 17](https://www.postgresql.org/docs/17/release-17.html), the latest version of the world's most advanced …
4
How Figma's databases team lived to tell the scale
Our nine month journey to horizontally shard Figma’s Postgres stack, and the key to unlocking (nearly) infinite scalability.
5
Amazon S3 Adds Put-If-Match (Compare-and-Swap)
Discover more about what's new at AWS with Amazon S3 adds new functionality for conditional writes
6
Vector indexing all of Wikipedia on a laptop
Indexing the entirety of English Wikipedia on a laptop has become a practical reality thanks to recent advances in the JVector library that will be part of the imminent 3.0 release.
7
The design philosophy of Great Tables
We’ve spent a lot of time thinking about tables. Tables—like plots—are crucial as a last step toward presenting information. There is surprising…
8
JPEG XL and the Pareto Front
Full details on libjxl v. 0.10 and how it affects jpeg xl lossless compression using Pareto Front for a closer analysis.
9
Vector databases are the wrong abstraction
Today’s vector databases disconnect embeddings from their source data. We should treat embeddings more like database indexes—here’s how.
10
Consistency LLM: converting LLMs to parallel decoders accelerates inference 3.5x
TL;DR: LLMs have been traditionally regarded as sequential decoders, decoding one token after another. In this blog, we show pretrained LLMs can be easily taught to operate as efficient parallel decoders. We introduce Consistency Large Language Models (CLLMs), a new family of parallel decoders capable of reducing…
11
The semantic web is now widely adopted
Web 3.0 was not about the blockchain, thank god
12
SQLite: Wal2 Mode
"Wal2" mode is very similar to "wal" mode. To change a database to wal2 mode, use the command:
13
How Discord stores trillions of messages (2023)
Engineer Bo Ingram shares insight into how Discord shoulders its traffic and provides a platform for our users to communicate.
14
Redis adopts dual source-available licensing
Developers love Redis. Unlock the full potential of the Redis database with Redis Enterprise and start building blazing fast apps.
15
ISO 8583: The language of credit cards
Discover the inner workings of ISO 8583, the global standard for credit card transaction messaging. Learn how it powers payment processing across networks and explore its structure, fields, and real-world applications.
16
How Meta trains large language models at scale
As we continue to focus our AI research and development on solving increasingly complex problems, one of the most significant and challenging shifts we’ve experienced is the sheer scale of co…
17
Jepsen: Datomic Pro 1.0.7075
Datomic is a general-purpose database intended for systems of record. In many ways, Datomic is unusual. At any instant in time, the state of the database is represented by a set of [entity, attribute, value] ( EAV ) triples, known as datoms . Each datom declares that some entity (like a person) has a particular…
18
A new JSON data type for ClickHouse
We’re excited to introduce our new and significantly enhanced JSON data type, purpose-built to deliver high-performance handling of JSON data. Our core engineer, Pavel Kruglov, dives into how we built this feature on top of ClickHouse's columnar storage.
19
Turbopuffer: Fast search on object storage
Inaugural blog post about the development of turbopuffer, a search engine that uses object storage and SSD caching for cost-effective, low latency search. This post describes into the motivation behind its creation, its unique architecture, and how it significantly reduces costs for large-scale vector searches…
20
Limbo: A complete rewrite of SQLite in Rust
we forked SQLite with the libSQL project. What would it be like if we just rewrote it?
21
Postgres.new: In-browser Postgres with an AI interface
Introducing postgres.new, the in-browser Postgres sandbox with AI assistance.
22
Databases and why their complexity is now unnecessary
This post is not going to be about what’s wrong with individual databases. There are so many databases and so many individual API issues, operational problems, and arbitrary limitations that it wou…
23
Rearchitecting: Redis to SQLite
Learn how we approached migrating our Wafris v1 client based on Redis to a new faster, easier to use SQLite architecture.
24
Common DB schema change mistakes in Postgres
Common DB schema change mistakes and how to avoid them in PostgreSQL development.
25
B-Trees and Database Indexes
B-trees are used by many modern DBMSs. Learn how they work, how databases use them, and how your choice of primary key can affect index performance.
26
The Great Migration from MongoDB to PostgreSQL
How Infisical migrated from MongoDB to PostgreSQL.
27
pg_timeseries: Open-source time-series extension for PostgreSQL
Engineering deep-dives, product updates, and company news from Tembo — the platform for running coding agents across your stack.
28
Migrating Uber's ledger data from DynamoDB to LedgerStore
This week, we'll dive into how we migrated Uber’s business-critical ledger data to LedgerStore. We'll detail how we moved more than a trillion entries (making up a few petabytes of data) transparently and without causing disruption, and we'll discuss what we learned during the migration.
29
Neon Serverless Postgres is generally available
Neon is now Generally Available. We’ve shipped major improvements to Neon internals that, combined with our operating experience scaling up to 700,000+ databases over the past year, give us the confidence that Neon is ready to support your business-critical workloads. If you’re building or scaling an application, you…
30
NumPy 2.0
NumPy 2.0.0 is the first major release since 2006. It is the result of 11 months of development since the last feature release and is the work of 212 contributors spread over 1078 pull requests. It contains a large number of exciting new features as well as changes to both the Python and C APIs.