31
Write a time-series database engine from scratch
It covers how to make a time-series database engine from scratch
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 138 posts.
32
MeiliSearch: A Minimalist Full-Text Search Engine
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
33
Waiting for PostgreSQL 14 – SEARCH and CYCLE clauses
Which means, that to test it, we need some data for recursive queries. This time I took directory listing of my /etc, and loaded it to db:
34
The pedantic checklist for changing your data model in a web application
Let's say you have a web app with some database. In your database you have an Invoice model, where you store things you are charging your customers.
35
PostGIS at 20, the Beginning
Twenty years ago today, the first email on the postgis users mailing list (at that time hosted on yahoogroups.com) was sent, announcing the first numbered release of PostGIS. The early history of PostGIS was tightly bound to a consulting company I had started a few years prior, Refractions Research. My first contracts…
36
I don't want to learn your query language (2018)
This is a bit of a rant but I really don't like software that invents its own query language. There's a trillion different ORMs out there. Another trillion databases with their own query language. Another trillion SaaS products where the only way to query is to learn some random query DSL they made up.
37
EBCDIC is incompatible with GDPR
Welcome to acronym city! The Court of Appeal of Brussels has made an interesting ruling. A customer complained that their bank was spelling the customer's name incorrectly. The bank didn't have support for diacritical marks. Things like á, è, ô, ü, ç etc. Those accents are common in many languages. So it was a little…
38
How We Went All In on sqlc/pgx for Postgres and Go
Touring the ORM and Postgres landscape in Go, and why sqlc is today’s top pick.
39
Deleting an S3 Bucket Costs Money
Deleting an S3 bucket is harder than you think, and may even cost you money.
40
Julia: Faster than Fortran, cleaner than Numpy
I'm super enthusiastic about Julia after running this comparison of Julia vs Numpy vs Fortran, for performance and code simplicity.
41
Word-Aligned Bloom Filters
Programmers often need to 'filter out' data. Suppose that you are given a database of users where only a small percentage are 'paying customers' (say 5% or less). You can write an SQL query to check whether a given user is indeed a paying customer, but it might require a round trip to your database … Continue reading…
42
43
How to Become a Data Engineer in 2021
The demand for data engineers is growing rapidly. According to hired.com, the demand has increased by 45% in 2019. The median salary for Data Engineers in SF Bay Area is around $160k. So the question…
44
45
46
MinIO: A Bare Metal Drop-In for AWS S3
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
47
Why SELECT * is bad for SQL performance (2020)
Here’s a list of reasons why SELECT * is bad for SQL performance, assuming that your application doesn’t actually need all the columns. When I write production code, I explicitly specify the columns of interest in the select-list (projection), not only for performance reasons, but also for application reliability…
48
Why I Use Nim instead of Python for Data Processing
Lazy programmers often prefer to substitute computing effort for programming effort.
49
FoundationDB: A distributed unbundled transactional key value store
FoundationDB: A Distributed Unbundled Transactional Key Value Store Zhou, et al., 2021
50
51
Investigating how the New York Times A/B tests their headlines
Part 1 of a series on the New York Times, in which I take a close look at how (and when) the New York Times tests multiple headlines for a single article.
52
Hierarchical Structures in PostgreSQL (2020)
Modelling hierarchical/team/categorical/tag data with arbitrary depths.
53
54
Comparing SQLite, DuckDB and Arrow with UN trade data
This is not a competition, is just to show how to use the hardware with relative efficiency, being the idea is to show something collaborative rather than competitive.
55
Why do we use R rather than Excel?
I recently had cause to take a beginners course in R - a language I'm fairly familiar with. One of the other students had never used it before, so we were buddied up in order for me to show them the ropes. The first lesson of R is always the same. Read a CSV, manipulate it a bit, draw a graph. We did it all without…
56
Spending $5k to learn how database indexes work
At Superwall we’re building an SDK to help App Developers increase their revenue by showing the right offer to the right person at the right time. Follow me on twitter for little tips and articles like this one.
57
Why isn't differential dataflow more popular?
Differential dataflow is a library that lets you write simple dataflow programs and a) then runs them in parallel and b) efficiently updates the outputs when new inputs arrive. Compared to competition like spark and kafka streams , it can handle more complex computations and provides dramatically better throughput and…
58
Replacing Elasticsearch with Rust and SQLite (2017)
Caveat: Elasticsearch is a multi-purpose, distributed, battle-tested, schema-free, log and document storage engine. We’ll be replacing it with a single-purpose, relational schema, me-tested, and custom nginx access log storage engine. Knowing that, let’s begin. Find this project on Github After building my home…
59
I don't think Elasticsearch is a good logging system
I’ve been a heavy user of ElasticSearch for coming up 7 years now. During that time I’ve used it for a few main usecases: A Search Engine, An APM Solution (after NewRelic started being stupidly expensive), a backend for Jaeger, and as a log storage system. In all of those usecases I’ve really pushed ElasticSearch to…
60
