1711
I used Benford’s law to analyze Covid-19 in 113 countries (2020)
Spoiler alert: I found some intriguing patterns…
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 1,888 posts.
1712
1713
Perceptual Image Hashing
Joe Bertolami's portfolio and blog covering artificial intelligence, game development, computer graphics, data compression, and more.
1714
1715
Racket vs Clojure (it's all about the data structures)
I've been asked by several people to explain why I use Clojure for my professional work rather than Racket. ABOUT RACKET I have been using R...
1716
Batch size one billion: SQLite insert speedups, from the useful to the absurd
A little-discussed method enables inserting rows with bound data into SQLite faster than any existing technique. This novel method is then discovered to have a drawback that makes it generally unusable. The rest of this article explores how to get the best insert performance out of SQLite generally; what's in your…
1717
Dynamo vs. Cassandra: Systems Design of NoSQL Databases (2018)
A comparison of ideas behind Dynamo and Cassandra
1718
1719
Extending That XOR Trick to Billions of Rows
Learn how to extend the classic XOR trick to find thousands of missing values using Invertible Bloom Filters
1720
Web SQL Database: In Memoriam
All signs seem to indicate that Apple will finally ship IndexedDB in Safari 7.1 sometime this year. This means that Safari, the last holdout against IndexedDB,…
1721
Mergeable replicated data types – Part I
Mergeable replicated data types Kaki et al., OOPSLA’19 This paper was published at OOPSLA, but perhaps it’s amongst the distributed systems community that I expect there to be the greatest in…
1722
PostgreSQL needs a new load balancer
We do a lot of high-availablity PostgreSQL deployments for clients. As of 9.1, streaming replication (SR) is excellent for this, and can sc...
1723
1724
How much memory does a graph take?
We need a fast way to query multiple potentially large sets of data on-demand at interactive speeds. Sometimes the easiest solution to a hard problem is to build the right tool for the job.
1725
SQLite transactions and virtual tables
In my previous post I introduced SQL virtual tables and how to use them in Go—registering modules per connection, defining schemas, and querying external sources as regular tables.
1726
Layers in Software: From Data to Value
Then Back in the 2000s, we wrote applications in layers. Presentation layer, client, data transfer, API, business logic, data access, database. We maintained strict separation bet ween these layers…
1727
Type information for faster Python C extensions
Update: The paper version of this post is accepted at PLDI SOAP 2024. Take a look at the preprint (PDF).
1728
1729
The 74,000 numbers of Barclays Bank
The UK faces an epidemic of telephone scams. Fraudsters are constantly calling people up pretending to be their bank. But how can you be sure the number displayed on your screen in genuine? You can't. The telecom system is hopelessly insecure and shouldn't be trusted for anything more complicated than dialling the…
1730
Wrong ways to use the databases, when the pendulum swung too far
Disclaimer: This story took place when I was a junior developer for a previous employer ages ago. Some details might be fuzzy, and goes without saying this is entirely from my own POV so take everything here with humor and a grain of salt.
1731
Replacing Pandas with Polars
I remember those days, oh so long ago, it seems like another lifetime. I haven’t used Pandas in many a year, decades, or whatever. We’ve all been there, done that. Pandas I mean. I would dare say it’s a rite of passage for most data folk. For those using Python, it’s probably one of the […]
1732
Poor schemas, poor cataloguing: why music tagging sucks
One of the things that frustrates me with having a local music library is the tedium of tagging. While there are tools (like beets or MusicBrainz Picard) to make it easier, I feel there are fundamental issues with the design… Continue reading →
1733
Building a Database in the 2020s (2022)
Building a database in the 2020s
1734
1735
1736
How to Get or Create in PostgreSQL
And why it is so easy to get wrong...
1737
Multidimensional analysis of plot arcs in thousands of TV and movie scripts
Note: a somewhat more complete and slightly less colloquial, but eminently more citeable, version of this work is in the Proceedings of the ...
1738
Stock Option Pricing Inference
I’m a stock market enthusiast 📈, so lately I have spent my spare time on the topic of stock option pricing, and I was curious to find out whether or not I could build a script that accurately reflected real market option prices. It turns out I came close, and learned a lot along the way, which I share in this article.
1739
Math versus Dirty Data
At Google, our organization designs, owns, and maintains a number of optimization models that automate the planning of Google’s datacenter growth and health. As is pretty standard in supply chain optimization and planning, these models are often integer linear programs. It’s a core competency of operations research…
1740
EnclaveDB: A secure database using SGX
EnclaveDB: A secure database using SGX Priebe et al., IEEE Security & Privacy 2018 This is a really interesting paper (if you’re into this kind of thing I guess!) bringing together the security…