2131
Absurd Workflows: Durable Execution with Just Postgres
Durable execution with just postgres.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 4,224 posts.
2132
Realtime Postgres Row Level Security
Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security.
2133
Migrating bajillions of database records at Stripe
There are roughly ten thousand bajillion (to the nearest bajillion) merchants registered with Stripe. We recently finished a Very Big Migration of Large Amounts Of Data between several database tables for every single one of them, without losing any of it, without any downtime, mis-reads or mis-writes, whilst running…
2134
2135
2136
Book Review: Designing Data-Intensive Applications
What a great book Designing Data-Intensive Applications is! It covers databases and distributed systems in clear language, great detail and without any fluff. I particularly like that the author Ma…
2137
On Uber’s Choice of Databases
Uber provides bad advice about PostreSQL vs. MySQL
2138
Introduction to Hacking PostgreSQL (2007)
Gavin Sherry and I have presented several versions of a tutorial introduction on how to modify the PostgreSQL source code. The original version was given at the PostgreSQL Anniversary Summit in June of 2006. Gavin has since given a revised version of the talk (at LCA 2007), as have I (at PgCon 2007).
2139
MapReduce Patterns, Algorithms, and Use Cases
In this article I digested a number of MapReduce patterns and algorithms to give a systematic view of the different techniques that can be found on the web or scientific articles. Several practical…
2140
2141
Do we need a third Apache project for columnar data representation?
Apache Parquet and Apache ORC have become a popular file formats for storing data in the Hadoop ecosystem. Their primary value proposition ...
2142
Debunking Misleading Benchmarks Of Redshift vs BigQuery
Publishing misleading performance benchmarks is a classic old guard marketing tactic. It’s not surprising to see old guard companies (like Oracle) doing this, but we were kind of surprised to see Google take this approach, too. So, when Google presented their BigQuery vs. Amazon Redshift benchmark results at a private…
2143
PostgreSQL 14 on Kubernetes
Run PostgreSQL 14 on Kubernetes, and see how convenient it is to use distributed features like parallel remote Postgres queries!
2144
2145
2146
NumPy 1.20
This NumPy release is the largest so made to date, some 684 PRs contributed by 184 people have been merged. See the list of highlights below for more details. The Python versions supported for this release are 3.7-3.9, support for Python 3.6 has been dropped. Highlights are
2147
Memory efficiency of parallel IO operations in Python
Python allows for several different approaches to parallel processing. The main issue with parallelism is knowing its limitations. We either want to parallelise IO operations or …
2148
2149
APFS – A Backup Software Developer’s Perspective
A backup-software developer's deep dive into Apple's APFS file system — snapshots, cloning, space sharing, and what it all means for reliable Mac backups.
2150
2151
2152
SQLite Begin Concurrent
Usually, SQLite allows at most one writer to proceed concurrently. The BEGIN CONCURRENT enhancement allows multiple writers to process write transactions simultanously if the database is in "wal" or "wal2" mode, although the system still serializes COMMIT commands.
2153
NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
We've achieved 10x data efficiency with NanoGPT Slowrun within a few weeks. An ensemble of 1.8B parameter models (18B total params) trained on 100M tokens matches what would normally require 1B tokens with a standard LM baseline. Data efficiency matters because compute grows much faster than data . Since our current…
2154
2155
Parsing the Infamous Japanese Postal CSV
Late last year I released posuto, a package presenting Japanese postal code data in an easy-to-use format. It's based on data released by Japan Post, which is infamous for being widely used but hard to parse.
2156
Understanding Recursive Queries in PostgreSQL
This article gives a simple introduction with examples to recursive queries in PostgreSQL. It also includes a comparison to Oracle recursive queries.
2157
Nvidia Scientists Take Top Spots in 2021 Brain Tumor Segmentation Challenge
NVIDIA data scientists took three of the top spots in a brain tumor segmentation challenge validation phase at the prestigious MICCAI 2021.
2158
PostgreSQL Streaming Replication (WAL); What It Is and How to Configure One
What is Streaming replication in PostgreSQL?Streaming replication in PostgreSQL is an efficient method for maintaining a near real-time replica of a primary database on one or more standby servers.The primary server continuously sends data to replica
2159
Introducing the Open Images Dataset
Posted by Ivan Krasin and Tom Duerig, Software EngineersIn the last few years, advances in machine learning have enabled Computer Vision to progres...
2160
Waiting for Postgres 16: Cumulative I/O statistics with pg_stat_io
We talk about querying system wide I/O statistics in Postgres, show several use cases for pg_stat_io, and talk about the future of I/O observability in Postgres.