301
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 371 posts.
302
303
You Don't Need UUID
I've experienced first-hand how using UUID hurts the usability of computer systems, and I want you to understand why you certainly don't need it.
304
I Migrated from a Postgres Cluster to Distributed SQLite with LiteFS
Why and how I migrated from distributed postgres to distributed sqlite on kentcdodds.com
305
Deno KV Is in Open Beta
Deno KV, the easiest way to add a strongly consistent database to your app, is now in open beta.
306
PostgreSQL and SQL:2023
In a previous article, I introduced what is new in SQL:2023. Now I want to show the status of SQL:2023 support in PostgreSQL. See the previous article for details on these features.
307
Postgres WAL Files and Sequence Numbers
Go inside the WAL file with this in-depth look at WAL file Log Sequence Number (LSN), WAL segment positions, WAL file names, and pg_switch_wal.
308
Transactions Are Not Locks (2022)
One thing I wish I had understood better earlier on in my experience with PostgreSQL is how transactions and locks can be used together to provide serializable logic.
309
Clog: Making a homegrown ClickHouse log for $20 a month
There was recently a discussion on Hacker News around application logging on a budget. At work I've been trying to keep things lean, not to the point of absurdity, but also not using a $100 or $1000/month setup, when a $10 one will suffice for now . We settled on a homegrown Clickhouse + PHP solution that has…
310
PostgreSQL 16 Bi-Directional Logical Replication
Introduction In this blog, we'll be going over some more advanced topics new in Postgres 16. Having some experience with Linux, Postgres, and SQL is necessary as we'll not only be going over these new features but also how to implement them. This blog was written using PostgreSQL 16 (Development Version) running on…
311
PostgreSQL Logical Replication Explained
Vigneshwaran C explained the internals of PostgreSQL Logical Replication during PGConf India - if you were not able to attend, you can read all here.
312
Our Journey with Apache Arrow (Part 2): Adaptive Schemas and Sorting
In the previous article, we discussed our use of Apache Arrow within the context of the OpenTelemetry project. We investigated various techniques to maximize the efficiency of Apache Arrow, aiming to find the optimal balance between data compression ratio and queryability. The compression results speak for themselves…
313
Pandas vs. Julia – cheat sheet and comparison
This is a Python/Pandas vs Julia cheatsheet and comparison. You can find what is the equivalent of Pandas in Julia or vice versa. You can find links to the documentation and other useful Pandas/Julia resources. The table below show the useful links for both: Pandas Julia data analysis
314
A LLM+OLAP Solution
The exploration of a LLM+OLAP solution is a bumpy journey, but phew, it now works well for the Tencent case, and they're writing down every lesson learned to share with you.
315
We Built a Streaming SQL Engine
A streaming SQL engine keeps queries’ results up to date without ever having to recalculate them, even as the underlying data changes. To explain this, imagine a simple query, such as SELECT count(*) FROM humans
316
Indexing a billion pages
It’s two years since we launched Mwmbl, the open source, non-profit search engine, on Boxing Day 2021. A good time to take stock of where we are and where we’re going. We’ve indexed over 100 million pages Thanks to our volunteers, who crawl the web using the Firefox extension and command line script, we’re crawling up…
317
ChatLZMA – text generation from data compression
I came across a random tweet, found a Wikipedia page and bumped into some smart people and long story short, apparently compression is equivalent to general intelligence.
318
Analysis of the data job market using HN job posts
I parse HackerNews (HN) “Ask HN: Who is hiring?” posts from 2013 to time of writing and analyse them to better understand the trends in the data job market with a focus on the fate of data science. Here are my main conclusions:
319
Database Isolation Is Broken and You Should Care
The database zeitgeist has decided that ANSI SQL '92 isn't enough. We need something better.
320
Daft: A High-Performance Distributed Dataframe Library for Multimodal Data
Discover how Daft accelerates multimodal data processing with a high-performance distributed dataframe engine built for modern AI and analytics tasks.
321
The challenges of supporting foreign key constraints
Today, PlanetScale launched support for foreign key constraints. This article covers some of the behind-the-scenes technical challenges we had to overcome to support them.
322
Faster PostgresSQL to BigQuery Transfers
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
323
Merklizing the key/value store for fun and profit
Joel Gustafson
324
Do you really need foreign keys?
Foreign keys are a bit like that well-intentioned friend who insists on double-checking everything you do. They’re often recommended as a must-have for enforcing referential integrity checks in your database.
325
The PostgreSQL documentation and the limitations of community
In my opinion, the PostgreSQL documentation is simultaneously excellent and fairly poor, and both its excellence and its shortcomings are di...
326
327
Rerun 0.9 – a framework for visualizing streams of multimodal data
We're releasing Rerun 0.9.0 with a completely revamped API that gives more control over the underlying data model, a hierarchical and time aware Entity Component System (ECS).
328
15k inserts/s with Rust and SQLite (2021)
There is this growing sentiment in tech that stacking more and more layers of complexity to reach the sacrosanct "infinite scalability" is not the way forward. First, because it rarely materializes: you need a lot of other things than fancy tech to reach millions of people. Second, because the tradeoffs
329
Querying the GitHub archive with the ClickHouse playground
Via this comment on Hacker News I started exploring the ClickHouse Playground . It's really cool, and among other things it allows CORS-enabled API hits that can query a decade of history from the GitHub events archive in less than a second.
330
Modern Transactional Stack
Transactional databases have long been the most critical component of application design. Why? Because a steadfast database is generally the ultimate enforcement point for correctness in a messy, distributed world. Without them we’d overpay and undercharge. We’d lose riders trying to get home from the airport, and…