481
A data table thousands of years old (2020)
I knew that data tables had been around a long time, but I didn't appreciate how long until I read recently about account-keeping in ancient Mesopotamia.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 1,888 posts.
482
The DynamoDB Paper
This week at USENIX ATC’22, a group of my colleagues 1 from the AWS DynamoDB team are going to be presenting their paper Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service . This paper is a rare look at a real-world distributed system that runs at massive scale.
483
Satellogic's Open Satellite Feed
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
484
Introduction to PostgreSQL physical storage
I initially wrote this post for myself, I wanted to understand enough about the underlying physical storage to be able to grasp other concepts which can affect your database. This post is probably more aimed to people with similar interests, I will try to keep it accurate but the aim is to make it simpler to…
485
I analysed 20 years of my chats
Importing 1.2 million messages from Telegram, VK, Instagram, Facebook, and Twitter into a structured Obsidian vault with local LLM inference - to measure how my friendships work with decay curves, vocabulary shifts, directional sentiment, and conflict rates.
486
Shardines: SQLite3 Database-per-Tenant with ActiveRecord
There is a pattern I am very fond of - “one database per tenant” in web applications with multiple, isolated users. Recently, I needed to fix an application I had for a long time where this database-per-tenant multitenancy utterly broke down, because I was doing connection management wrong. Which begat the question…
487
PostgreSQL at low level: stay curious
Yes, I know, it’s a long text, and it was my conscious decision to write it in this way. But fear not! Imagine that you read a book, take a look at the introduction and first few interesting sections, think about it and then find time to read further. I hope I’ve left enough references, so if you don’t get some ideas…
488
What Does It Take to Track a Million Cell Phones?
You can find anything and everything on the internet, yet nothing that explains how to track cell phones. Let us clarify right away, we are not talking about how to track your own cell phone in cas…
489
FoundationDB: A distributed unbundled transactional key value store
FoundationDB: A Distributed Unbundled Transactional Key Value Store Zhou, et al., 2021
490
PostgreSQL on the Command Line
Working efficiently with psql
491
Arima Model – Guide to Time Series Forecasting in Python
Using ARIMA model, you can forecast a time series using the series past values. In this post, we build an optimal ARIMA model from scratch and extend it to Seasonal ARIMA (SARIMA) and SARIMAX models. You will also see how to build autoarima models in python
492
493
How to create NBA shot charts in Python
In this post I go over how to extract a player's shot chart data and then plot it using matplotlib and seaborn .
494
Distributed SQLite: Paradigm shift or hype?
SQLite is fast, really fast. On a single ~40€ / m commodity server, it can sustain ~168,000 reads and ~8000 writes per second, concurrently. Because it's an embedded library designed for client-side applications such as embedded systems and phone & desktop applications, SQLite databases have to be co-located…
495
Data first, not code first
Developers hate him! We'll cover some standard tips and tricks here, but we're not really interested in those. We're looking for the One Weird Trick to rule them all. Hopefully each trick we encounter brings us closer to coding Mecca. In the beginning The first video game I ever wrote was called Ninja Wars. Yes, that…
496
Redis: Zero to Master in 30 minutes
Karl Seguin's Blog - A mix of coding and creative writing
497
ML is not that good at predicting consumers' choices
I wanted to share the attached paper on choice prediction that I recently co-authored with Itamar Simonson in case it’s of interest.
498
499
So, you want to deploy on the edge?
Application developers often deploy their apps into a single area, generally represented by a handful of AZs in a single region. No matter where their …
500
How rqlite is tested
rqlite is a lightweight, open-source, distributed relational database written in Go, and built on SQLite and Raft. With its origins dating back to 2014, its design has always prioritized reliability, and quality. The robustness of rqlite is also a testament to its disciplined testing strategy: after more than 10 years…
501
Reverse-Engineering Xkcd's 'Frequency'
About a week ago, Randall Munroe published Frequency , the 1331st entry in his xkcd webcomic series. Unlike most xkcd comics, it's animated — actually, 5-by-10 grid of animated GIFs. Each describes a scenario (e.g., "one birth," or "someone locks their keys in their care") and blinks at the scenario's purported…
502
Migrating 1200 databases from MySQL to Postgres
Once Upon a Time there was a company called Base7Booking that by the beginning of 2016 it was supporting around 1200 databases on MySQL 5...
503
Transformer – Spreadsheet
Make your own AI by hand ✍️ exercises
504
How I slashed a SQL query's runtime with two Unix commands
I was trying to run a simple join query on MariaDB (MySQL) and its performance was horrendous. Here’s how I cut down the query’s run time from over 380 hours to under 12 hours by executing part of it with two simple Unix commands.
505
PostgreSQL query performance bottlenecks
PostgreSQL query planner is full of surprises, so a common-sense approach to writing performant queries is sometimes misleading. In this blog post, I'll describe examples of optimizing seemingly obvious queries with the help of EXPLAIN ANALYZE and Postgres metadata analysis.
506
Detecting LLM-Generated Texts with “Classical” Machine Learning
.admonition { margin: .75em 0; padding: .6rem; overflow: hidden; font-size: 12px; page-break-inside: avoid; border-left: .3rem solid #42b983; border-radius: .3rem; box-shadow: 0 0.1re
507
Investigating how the New York Times A/B tests their headlines
Part 1 of a series on the New York Times, in which I take a close look at how (and when) the New York Times tests multiple headlines for a single article.
508
Efficient pagination of a SQL table with 100M records
Compilation of my favorite SQL queries after a decade of dealing with relational databases. Today we talk about scanning a table with 100 million records.
509
Gzip beats BERT? Part 2: dataset issues, improved speed, and results
Summary : Update to my last post analyzing the gzip+knn paper. Some of the datasets have heavily contaminated train/test tests. I show some ways to significantly speed-up the gzip method. I present more results of my "fair" reimplementation .
510
Hierarchical Structures in PostgreSQL (2020)
Modelling hierarchical/team/categorical/tag data with arbitrary depths.