91
How percentile approximation works and why it's more useful than averages
What percentile approximations are, how percentiles work in PostgreSQL, and calculating them efficiently on time-series data.
New
The latest blog posts to take off on Hacker News. 314 posts.
92
Trademark Actions Against the PostgreSQL Community
The PostgreSQL Core Team and the PostgreSQL Community Association of Canada (PGCAC) wish to provide clarification on the ownership and …
93
94
How We Proved the Eth2 Deposit Contract Is Free of Runtime Errors
Consensys is a blockchain software company building the tools, infrastructure and research that power the new Era of Decentralized Finance, for a more secure and open economy
95
Index bloat reduced in PostgreSQL v14
This article describes how PostgreSQL v14 reduces B-tree index bloat with "bottom-up index tuple deletion". A test case shows the difference.
96
97
How We Went All In on sqlc/pgx for Postgres and Go
Touring the ORM and Postgres landscape in Go, and why sqlc is today’s top pick.
98
Spice.ai – Open-source, time series AI for developers
Read blog posts about Spice.ai OSS from the team and community
99
Product Quantization: Compressing high-dimensional vectors
Product quantization (PQ) is a popular method for dramatically compressing high-dimensional vectors to use 97% less memory, and for making nearest-neighbor search speeds 5.5x faster in our tests.
100
Lessons Learned from Two Years as a Data Scientist
I finally escaped from (grad) school in 2019, spent two months interning as an assistant trader at FTX, and have since spent the last two years working as a data scientist, with the bulk of that time at Microsoft and the last two months at Anthropic. Not gonna lie — my time as a data scientist has been pretty awesome…
101
PostgreSQL Subtransactions Considered Harmful
PostgreSQL subtransactions (nested transactions) may cause multiple performance and scalability issues: higher rates of XID growth and higher risks of transaction ID wraparound, performance degradation when more than PGPROC_MAX_CACHED_SUBXIDS (64) are used in a session, drastic performance slowdowns when…
102
Fastest table sort in the West – Redesigning DuckDB's sort
DuckDB, a free and open-source analytical data management system, has a new highly efficient parallel sorting implementation that can sort much more data than fits in main memory.
103
The 74,000 numbers of Barclays Bank
The UK faces an epidemic of telephone scams. Fraudsters are constantly calling people up pretending to be their bank. But how can you be sure the number displayed on your screen in genuine? You can't. The telecom system is hopelessly insecure and shouldn't be trusted for anything more complicated than dialling the…
104
An Introduction to JQ
In this article, I'm going to go over the basics building blocks of jq in enough depth that you will be able to understand how jq works.
105
SQL vs. NoSQL Is the Wrong Distinction
A misconception that I held for a really long time was that systems that use SQL (Structured Query Language) are by definition relational databases, and systems that don’t (e.g., MongoDB, Cassandra) are non-relational. It’s easy to understand whether a database uses SQL or not, but what does it mean when we say a…
106
How Discord Stores Billions of Messages (2017)
Discover the latest news and app updates, stories, company insights, and plenty more about Discord, all in one place.
107
Write an SQL query builder in 150 lines of Python
This is the fourth article in a series about writing my own SQL query builder. Today, we'll rewrite it from scratch, explore API design, learn when to be lazy, and look at worse and better ways of doing things – all in 150 lines of Python!
108
Wikidata or Scraping Wikipedia
Yesterday I was pointed to a blog post describing how to answer an interesting project: how many generations from Alfred the Great to Elizabeth II ? Alfred the Great was a king in England at the end of the 9th century, and Elizabeth II is the current Queen of England (and a bit more).
109
Project Jupyter Celebrates 20 Years, Fernando Pérez Reflects on How It Started
UC Berkeley Division of Computing, Data Science and Society2021118 points34 commentsDatabases & Data details
Twenty years ago, UC Berkeley Associate Statistics Professor Fernando Pérez started one of the foundational tools for analyzing large amounts of data in a transparent and collaborative way. We spoke with Pérez about why he started this project, what challenges he’s faced and what to expect from him and Project Jupyter…
110
Strict Tables – Column type constraints in SQLite - Draft
SQLite strives to be flexible regarding the datatype of the content that it stores. For example, if a table column has a type of "INTEGER", then SQLite tries to convert anything inserted into that column into an integer. So an attempt to insert the string '123' results in an integer 123 being inserted. But if the…
111
Data-Mining Wikipedia for Fun and Profit
The engineering blog of Bill P. Godfrey.
112
How is efficacy vs. severe disease strong if 60% of hospitalized are vaccinated?
A surge involving the rapidly-transmitting Delta variant in heavily vaccinated countries has led to much hand-wringing that the vaccines are not effective against Delta, or vaccine effectivenss wanes after 4-6 months. This has fueled anti-vaccine sentiment suggesting the vaccines are not working, and causing much…
113
114
Evidence of fraud in an influential field experiment about dishonesty
This post is co-authored with a team of researchers who have chosen to remain anonymous. They uncovered most of the evidence reported in this post. These researchers are not connected in any way to…
115
116
Why query planning for streaming systems is hard
Many groups are working on running sql queries in incremental/streaming systems . Query planning in this context is not a well understood problem. This post is a quick pointer towards the many open problems. It's not intended to be complete or authoritative, because I haven't engaged with the problem much beyond…
117
MeiliSearch: A Minimalist Full-Text Search Engine
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
118
119
Event Sourcing (2005)
Capture all changes to an application state as a sequence of events.
120
Nearest neighbor indexes for similarity search
Vector similarity search is a game-changer in the world of search. It allows us to efficiently search a huge range of media, from GIFs to articles — with incredible accuracy in sub-second timescales for billion+ size datasets.