Top posts
Every blog post that took off on Hacker News, ranked by the votes of its best submission. 4,226 posts.
Building an open data pipeline in 2024
Using Iceberg allows us to pick the optimal "big data" compute environment for the specific requirements we have. There's no need to limit yourself to a single solution.
Tracing Military Command Chains Through Time and Location
Case Study ⸺ Using novel technology to analyze groundbreaking chain of command research
How Coderwall grew over 50% with an A/B test
Turn SEO rankings into revenue. RankScience drives growth across Google, ChatGPT, and AI platforms for startups.
How to Get or Create in PostgreSQL
And why it is so easy to get wrong...
Announcing Spark 1.3
Apache Spark 1.3 introduces the DataFrame API, enhancing data processing with new features across streaming, ML, and SQL.
How Swiss tables work in Go built-in map
Go 1.24 replaced the built-in map’s bucket-based runtime with Swiss Tables. This article explains groups, control bytes, H1 and H2, probing, table growth, directories, deletion, load factor, and the experimental split-group layout.
Multidimensional analysis of plot arcs in thousands of TV and movie scripts
Note: a somewhat more complete and slightly less colloquial, but eminently more citeable, version of this work is in the Proceedings of the ...
Stock Option Pricing Inference
I’m a stock market enthusiast 📈, so lately I have spent my spare time on the topic of stock option pricing, and I was curious to find out whether or not I could build a script that accurately reflected real market option prices. It turns out I came close, and learned a lot along the way, which I share in this article.
Fixing Bad Data in Datomic
The free database designed for immutable values and a time-aware information model.
Math versus Dirty Data
At Google, our organization designs, owns, and maintains a number of optimization models that automate the planning of Google’s datacenter growth and health. As is pretty standard in supply chain optimization and planning, these models are often integer linear programs. It’s a core competency of operations research…
EnclaveDB: A secure database using SGX
EnclaveDB: A secure database using SGX Priebe et al., IEEE Security & Privacy 2018 This is a really interesting paper (if you’re into this kind of thing I guess!) bringing together the security…
Building agents using streaming SQL queries
Table of Contents Agents Need to Interact With LLMs Agents Should Be Event-Driven Agents Need Context Agents Require Memory When SQL Is Not Enough Parting Thoughts AI Agents have improved in leaps …
Pgvector 0.5.0 Feature Highlights and how tos
An overview of new features in pgvector 0.5.0, including HNSW, parallel index builds, and more!
Thinking about high-quality human data
[Special thank you to Ian Kivlichan for many useful pointers (E.g. the 100+ year old Nature paper “Vox populi”) and nice feedback. 🙏 ] High-quality data is the fuel for modern data deep learning model training. Most of the task-specific labeled data comes from human annotation, such as classification task or RLHF…
Understanding Raft Consensus
Recently I was digging deeper into Raft , an important algorithm in the field of distributed systems. Raft is a consensus algorithm , meaning that it is designed to facilitate a set of computers agreeing on a state of the world (more on exactly how the state of the world is represented later), even when communications…
From Datalog to SVG
Exporting graph data to text file and SVG with Datalog
Why Did So Many Startups Choose MongoDB?
NoSQL databases were the future. MongoDB was the database for “modern” web engineers and used by countless startups. What happened?
Database Version Control with Liquibase
Introduction to managing DB shcema changes with Liquibase
Do we need to store all that telemetry?
In my last post I talked about why modern observability has become so expensive. At the end of the post I posit a question: What if by default we never send any telemetry at all? In this post I’m goin