2131
131M American Buildings
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
Top posts
Every blog post that took off on Hacker News, ranked by the votes of its best submission. 4,225 posts.
2132
Absurd Workflows: Durable Execution with Just Postgres
Durable execution with just postgres.
2133
Realtime Postgres Row Level Security
Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security.
2134
Migrating bajillions of database records at Stripe
There are roughly ten thousand bajillion (to the nearest bajillion) merchants registered with Stripe. We recently finished a Very Big Migration of Large Amounts Of Data between several database tables for every single one of them, without losing any of it, without any downtime, mis-reads or mis-writes, whilst running…
2135
2136
2137
Book Review: Designing Data-Intensive Applications
What a great book Designing Data-Intensive Applications is! It covers databases and distributed systems in clear language, great detail and without any fluff. I particularly like that the author Ma…
2138
On Uber’s Choice of Databases
Uber provides bad advice about PostreSQL vs. MySQL
2139
Introduction to Hacking PostgreSQL (2007)
Gavin Sherry and I have presented several versions of a tutorial introduction on how to modify the PostgreSQL source code. The original version was given at the PostgreSQL Anniversary Summit in June of 2006. Gavin has since given a revised version of the talk (at LCA 2007), as have I (at PgCon 2007).
2140
MapReduce Patterns, Algorithms, and Use Cases
In this article I digested a number of MapReduce patterns and algorithms to give a systematic view of the different techniques that can be found on the web or scientific articles. Several practical…
2141
2142
Do we need a third Apache project for columnar data representation?
Apache Parquet and Apache ORC have become a popular file formats for storing data in the Hadoop ecosystem. Their primary value proposition ...
2143
Debunking Misleading Benchmarks Of Redshift vs BigQuery
Publishing misleading performance benchmarks is a classic old guard marketing tactic. It’s not surprising to see old guard companies (like Oracle) doing this, but we were kind of surprised to see Google take this approach, too. So, when Google presented their BigQuery vs. Amazon Redshift benchmark results at a private…
2144
PostgreSQL 14 on Kubernetes
Run PostgreSQL 14 on Kubernetes, and see how convenient it is to use distributed features like parallel remote Postgres queries!
2145
2146
2147
NumPy 1.20
This NumPy release is the largest so made to date, some 684 PRs contributed by 184 people have been merged. See the list of highlights below for more details. The Python versions supported for this release are 3.7-3.9, support for Python 3.6 has been dropped. Highlights are
2148
Memory efficiency of parallel IO operations in Python
Python allows for several different approaches to parallel processing. The main issue with parallelism is knowing its limitations. We either want to parallelise IO operations or …
2149
2150
APFS – A Backup Software Developer’s Perspective
A backup-software developer's deep dive into Apple's APFS file system — snapshots, cloning, space sharing, and what it all means for reliable Mac backups.
2151
2152
2153
SQLite Begin Concurrent
Usually, SQLite allows at most one writer to proceed concurrently. The BEGIN CONCURRENT enhancement allows multiple writers to process write transactions simultanously if the database is in "wal" or "wal2" mode, although the system still serializes COMMIT commands.
2154
NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
We've achieved 10x data efficiency with NanoGPT Slowrun within a few weeks. An ensemble of 1.8B parameter models (18B total params) trained on 100M tokens matches what would normally require 1B tokens with a standard LM baseline. Data efficiency matters because compute grows much faster than data . Since our current…
2155
2156
Parsing the Infamous Japanese Postal CSV
Late last year I released posuto, a package presenting Japanese postal code data in an easy-to-use format. It's based on data released by Japan Post, which is infamous for being widely used but hard to parse.
2157
Understanding Recursive Queries in PostgreSQL
This article gives a simple introduction with examples to recursive queries in PostgreSQL. It also includes a comparison to Oracle recursive queries.
2158
Nvidia Scientists Take Top Spots in 2021 Brain Tumor Segmentation Challenge
NVIDIA data scientists took three of the top spots in a brain tumor segmentation challenge validation phase at the prestigious MICCAI 2021.
2159
PostgreSQL Streaming Replication (WAL); What It Is and How to Configure One
What is Streaming replication in PostgreSQL?Streaming replication in PostgreSQL is an efficient method for maintaining a near real-time replica of a primary database on one or more standby servers.The primary server continuously sends data to replica
2160
Introducing the Open Images Dataset
Posted by Ivan Krasin and Tom Duerig, Software EngineersIn the last few years, advances in machine learning have enabled Computer Vision to progres...