1021
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 1,888 posts.
1022
Bayesian ranking of items with up and downvotes or 5 star ratings
Evan Miller wrote a great article How Not To Sort By Average Rating . He shows that simple formulas like score = upvotes - downvotes or score = upvotes / (upvotes + downvotes) don’t work. He proposes to use the lower bound of the Wilson score confidence interval. The formula is complicated and somewhat of a hack…
1023
1024
Mapping the off-target effects of every FDA-approved drug in existence
6.2k words, 29 minutes reading time
1025
SQLite 3.44: Interactive release notes
Order by in aggregates, better string concatenation, and thorough date formatting.
1026
Examining PostgreSQL 9.4 – A first look
PostgreSQL is currently entering its final commit fest. While its still going, which means there could still be more great features to come, we can start to take a look at what you can expect from it now. This release seems to bring a lot of minor increments versus some bigger highlights of previous ones. At the same…
1027
Relational shell programming
Unix is a bestiary of ad hoc databases: comma-, colon-, tab- and space-separated tables. Think of /etc/* or /var/log/* , or of columnar commands.
1028
Connecting to Google Sheets with Python
I wrote up my experience using Python to process data from a shared Google Sheets spreadsheet
1029
Postgres indexes under the hood
Many software engineers use database indexes every day, but few of us really understand how they work. In this post I’ll explain: How indexing works in Postgres using B-Trees What B-Trees are Why they are a good fit for this problem Indexes in Postgres Postgres actually offers 4 different kinds of indexes for…
1030
Groundhog: Addressing the Threat That R Poses to Reproducible Research
R, the free and open source program for statistical computing, poses a substantial threat to the reproducibility of published research. This post explains the problem and introduces a solution. The…
1031
Using command line to process CSV files (2022)
Process CSV files from the command line with awk, sort, and grep: pull out fields, order rows, and filter data without a spreadsheet or a script.
1032
Generalizing 'jq' and Traversal Systems using optics and standard monads
The Personal blog and musings of Chris Penner, a designer, developer and future opsimath.
1033
Building a MongoDB Clone in Postgres
I had a crazy thought. Don't all good ideas start with that phrase? Well, this one was suitably crazy: why not build my own version of MongoDB right on top of Postgres? It sounds a little far-fetched, but in all honesty it's pretty simple.
1034
Implementing system-versioned tables in Postgres
There's no official support for system-versioned tables in Postgres yet, and the temporal extensions aren't supported on Azure/AWS/GCP. Let's implement it ourselves with 3 triggers and an index.
1035
1036
Evidence of fraud in an influential field experiment about dishonesty
This post is co-authored with a team of researchers who have chosen to remain anonymous. They uncovered most of the evidence reported in this post. These researchers are not connected in any way to…
1037
Call me maybe: Redis redux
In a recent blog post , antirez detailed a new operation in Redis: WAIT . WAIT is proposed as an enhancement to Redis’ replication protocol to reduce the window of data loss in replicated Redis systems; clients can block awaiting acknowledgement of a write to a given number of nodes (or time out if the given threshold…
1038
I Want Decentralized Version Control for Structured Data
How do I synchronize application data between devices and people? That's the question I was unable to answer when thinking about ideas for new applications I could develop. I wanted the synchronization to be
1039
1040
Most Bitcoin Inscriptions belong to a single person
TLDR We have analyzed all Ordinal Inscriptions created since Bitcoin block 779832, which marks the introduction of the BRC-20 standard.
1041
Gender and verbs across 100K stories: a tidy analysis
Previously in this series
1042
Good data scientist, bad data scientist
What separates a good Data Scientist from a bad one?
1043
Hacking on PostgreSQL is hard
Hacking on PostgreSQL is really hard. I think a lot of people would agree with this statement, not all for the same reasons. Some might poin...
1044
1045
Supercharge SQLite with Ruby Functions
An interesting twist in my recent usage of SQLite was the fact that I noticed my research scripts and the database intertwine more. SQLite is unique in that it really lives in-process, unlike standalone database servers. There is a feature to that which does not get used very frequently, but can be indispensable in…
1046
Why Can't Database Tables Index Themselves? (2006)
Here’s a thought question for today: why can’t database tables index themselves? Obviously, indexes are a central concept to databases and database performance. But horror tales still abound of naïve developers who “forget” to index their tables, and encounter massive performance and scalability problems down the road…
1047
Parsing SQL
SQL is a language to handle data in databases. However, it has limitations and you might need to parse SQL to get around them.
1048
The Simpsons by the Data
Analysis of Simpsons characters, dialogue, TV ratings, and more
1049
1050
1,600 days of a failed hobby data science project
I dedicated over 1,600 days to a data science project, only to see it fail. In this article, I share a new checklist I developed to help prevent similar setbacks in the future.