31
Forecasts need to have error bars
Richard Rosenfeld in the most recent Criminologist published a piece about forecasting national level crime rates. People complain about the FBI releasing crime stats a year late, academics are wor…
New
The latest blog posts to take off on Hacker News. 371 posts.
32
A new approach to domain ranking
This is a very brief post announcing a fascinating discovery. It appears to be possible to use the cosine similarity approach powering explore2.marginalia.nu as a substitute for the link graph in an eigenvector-based ranking algorithm (i.e. PageRank). The original PageRank algorithm can be conceptualized as a…
33
Pipeworld: Spreadsheet Dataflow Computing (2021)
Now for something completely different. In the spiritual vein of One Night in Rio: Vacation photos from Plan9 and AWK for multimedia, here is a tool that is the link that ties almost all the projec…
34
Accidental database programming
Join me as we take a look at common application data patterns, and how they relate to the inner-workings of databases. In this post, we discuss data caching, indexing, optimistic mutations, and recursive cache invalidation. We will see how life might be easier if we could just use a frontend optimized database like…
35
36
S3 Express Is All You Need
The future of modern data infrastructure is object storage.
37
Antirez' tiny JSON selector library
Salvatore Sanfilippo, aka antirez released a telegram bot library in C recently. While I'm not at all interested in a telegram bot library, I always like to read Salvatore's code, which is uniformly superb.
38
My favorite database shirts
Andy reviews his favorite NoSQL and NewSQL database shirts.
39
River: A fast, robust job queue for Go and Postgres
Introducing River, a Postgres-based job queue designed for resilience and correctness through strong transactional guarantees.
40
41
An automatic indexing system for Postgres
The pganalyze Indexing Engine simulates hundreds of possible combinations to recommend missing Postgres indexes using its 'What if?' analysis.
42
I analyzed Stack Overflow for secrets
I analyzed stackoverflow for secrets and leaks.
43
Operating on a minimal two-core Postgres instance: Query optimization insights
In an era where the costs for cloud services continue to soar, I want to outline how our data team has successfully managed to run the infrastructure of our energy trading company within the first year using only two PostgreSQL database instances - each equipped with just two CPU cores and four GB of RAM. Despite this…
44
The Architecture of Serverless Data Systems
I recently blogged about why I believe the future of cloud data services is large-scale and multi-tenant, citing, among others, S3. “ Top tier SaaS services like S3 are able to deliver amazing simplicity, reliability, durability, scalability, and low price because their technologies are s
45
Summing columns in remote Parquet files using DuckDB
vivym/midjourney-messages on Hugging Face is a large (~8GB) dataset consisting of 55,082,563 Midjourney images - each one with the prompt and a URL to the image hosted on Discord.
46
Backblaze Drive Stats for Q3 2023
Explore Backblaze's Q3 2023 hard drive stats, revealing insights on 259,533 HDDs, failure rates, and data center performance for enhanced reliability.
47
48
Loro: Reimagine state management with CRDTs
Loro, our high-performance CRDTs library, is now open source. In this article, we share our vision for the local-first software development paradigm, why we're excited about it, and the current status of Loro.
49
Maxar's Open Satellite Feed
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
50
Open, rigorous and reproducible research: A practitioner’s handbook (2021)
Even though many scientists are expected to abide by the principles of open, rigorous and reproducible research, few scientists actually receive formal training in this. Indeed, there is no collection written by working scientists for working scientists about the principles of transparent, rigorous and reproducible…
51
Debunking the Myth of "Anonymous" Data
Today, almost everything about our lives is digitally recorded and stored somewhere. Each credit card purchase, personal medical diagnosis, and preference about music and books is recorded and then used to predict what we like and dislike, and—ultimately—who we are. This often happens without our...
52
SciPy builds for Python 3.12 on Windows are a minor miracle
Moving SciPy to Meson meant finding a different Fortran compiler on Windows, which was particularly tricky to pull off for conda-forge. This blog tells the story about how things looked pretty grim for the Python 3.12 release, and how things ended up working out just in the nick of time.
53
Kurdish Parentheses on OpenStreetMap, Three Ways
While making a proposal to update Noto fonts for OpenStreetMap tiles, which fixes an issue with Kurdish labels using ڵا , I noticed a street with mismatched parentheses and was anticipating a deep dive into the tile renderer. But the actual problem was somewhere else:
54
Vitess 18
Vitess 18 is now generally available, with a number of new enhancements designed to improve usability, performance, and MySQL compatibility.
55
PostgreSQL Encryption: The Available Options
On an episode of Postgres FM , the hosts had a (very brief) discussion of data encryption in PostgreSQL. While Postgres FM is a podcast well worth a subscribe, the hosts aren’t data security experts, and so as someone who builds a queryable database encryption system , I found the coverage to be somewhat… lacking. I…
56
Building a Streaming Platform in Go for Postgres
At PeerDB, our mission is to create a Postgres-first data-movement platform that makes it fast and simple to stream data from Postgres to Data Warehouses…
57
Updates to the H2O.ai db-benchmark
The H2O.ai db-benchmark has been updated with new results. In addition, the AWS EC2 instance used for benchmarking has been changed to a c6id.metal for improved repeatability and fairness across libraries. DuckDB is the fastest library for both join and group by queries at almost every data size.
58
Tracking SQLite Database Changes in Git
Note : This post stirred up some interesting discussions on HackerNews and Lobste.rs .
59
SQLite 3.44: Interactive release notes
Order by in aggregates, better string concatenation, and thorough date formatting.
60
Rules of schema growth (2017)
The free database designed for immutable values and a time-aware information model.