The latest blog posts to take off on Hacker News. 4,224 posts.
In this post we’ll discuss how Spotify optimized and sped up elements from our largest Dataflow job, , for using a technique called Sort Merge Bucket (SMB) join. We’ll present the design and implementation of SMB and how we incorporated it into our data pipelines.
Despite an exponential increase in computing power, our applications require more machines than ever because of architectural decisions made 25 years ago. You can eliminate much of your complexity and cost by using SQLite & Litestream for your production applications.
Mark Huberty & Clare Corthell, Lyft Mapping
Data labeling is often the biggest bottleneck in machine learning. Active learning lets you train machine learning models with much less labeled data. The best AI-driven companies, like Tesla, use active learning.
Which means, that to test it, we need some data for recursive queries. This time I took directory listing of my /etc, and loaded it to db:
The Apache Arrow team is pleased to announce the 3.0.0 release. This covers over 3 months of development work and includes 666 resolved issues from 106 distinct contributors. See the Install Page to learn how to get the libraries for your platform. The release notes below are not exhaustive and only expose selected…
Memory allocator is an important part of the system, so choosing the right allocator for a workload can give huge benefits. Here is a story of how we decreased service memory usage by almost three times.
Databricks combines the best of data lakes and data warehouses.
How to free space without dropping indexes or deleting data
This NumPy release is the largest so made to date, some 684 PRs contributed by 184 people have been merged. See the list of highlights below for more details. The Python versions supported for this release are 3.7-3.9, support for Python 3.6 has been dropped. Highlights are
Blog article by Jeroen Janssens. Jul 7, 2024. 93 min read. Hey there, you’re reading an updated version of a post I published way back in 2019. Because of the 2024 Plotnine contest and the recent release of Polars 1.0, I decided to freshen things up. Changes include: Upgrade to Plotnine version 0.13.6 (from…
Analyzing and breaking external encrypted HDD has been a “hobby” of mine for quite some time. With my colleagues Joffrey Czarny and Julien Lenoir we looked at several models in the past :
The AFR for 2020 dropped below 1% down to 0.93%. In 2019, it stood at 1.89%. That’s over a 50% drop year over year.
Planning to scale PostgreSQL? Learn when a single node is enough, how to prepare for sharding, and which scaling steps to consider first.
With the second gen of Graviton2 instances announced, we decided to take a look at the price/performance from the standpoint of running PostgreSQL.
Differential dataflow is a library that lets you write simple dataflow programs and a) then runs them in parallel and b) efficiently updates the outputs when new inputs arrive. Compared to competition like spark and kafka streams , it can handle more complex computations and provides dramatically better throughput and…
Last week, Elastic announced they will change their software licensing strategy, and will not release new versions of Elasticsearch and Kibana under the Apache License, Version 2.0 (ALv2). Instead, new versions of the software will be offered under the Elastic License (which limits how it can be used) or the Server…
Let’s Encrypt helps to protect a huge portion of the Web by providing TLS certificates to more than 235 million websites. A database is at the heart of how Let’s Encrypt manages certificate issuance. If this database isn’t performing well enough, it can cause API errors and timeouts for our subscribers. Database…
Now in its third year, the 2021 Cloud Report from Cockroach Labs compares AWS, Azure, and Google (GCP) to tell realistic performance stories on behalf of mission-critical OLTP applications.
NoSQL was the next big thing in system architecture in 2011, but overall interest in it has plateaued recently. What is NoSQL, what does it have to do with modern development, and is it worth implementing in your project?
