91
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 272 posts.
92
Processing 40TB of code from 10M projects with a dedicated server and Go
The command line tool I created Sloc Cloc and Code (scc) (which is now modified and maintained by many other excellent people) counts lines of code, comments and makes a complexity estimate for files …
93
Technical Challenges Developing a Distributed SQL Database
Here is an outline of some of architectural issues addressed as we build YugabyteDB, our high-performance distributed SQL database.
94
PostgreSQL 12 Beta 1 Released
The PostgreSQL Global Development Group announces that the first beta release of PostgreSQL 12 is now available for download. This …
95
96
Open-sourcing F14 for memory-efficient hash tables
Hash tables provide a fast way to maintain a set of keys or map keys to values, even if the keys are objects, like strings. They are such a ubiquitous tool in computer science that even incremental…
97
How much faster is Redis at storing a blob of JSON compared to Postgres?
tl;dr; Redis is 16 times faster at reading these JSON blobs.*
98
Reducing the Performance Gap of Intel's MKL on AMD Threadripper
In this post I'm going to show you a simple way to significantly speedup Python numpy compute performance on AMD CPU's when using Anaconda Python.
99
100
101
Keeping CALM: when distributed consistency is easy
Keeping CALM: when distributed consistency is easy Hellerstein & Alvaro, arXiv 2019 The CALM conjecture (and later theorem) was first introduced to the world in a 2010 keynote talk at PODS. Beh…
102
Harvesting LinkedIn data for fun and profit
A few years ago, before the GDPR era, I was involved in a stealth marketing startup. We went out of
103
104
105
A Tale of Query Optimization
Last December, I got an interesting bug report from the VWO support team. It seemed like the loading time of a particular analytics report was ridiculously slow for a big Enterprise customer. Since I am part of the Data platform, a flag was immediately raised and I got involved in debugging the issue.
106
107
108
Amazon Aurora: Avoiding Distributed Consensus for I/Os, Commits, Membership
Amazon Aurora: on avoiding distributed consensus for I/Os, commits, and membership changes, Verbitski et al., SIGMOD’18 This is a follow-up to the paper we looked at earlier this week on the design…
109
Comprehensive Guide on Data Visualization with Pandas
Visualization has always been challenging task but with the advent of dataframe plot() function it is quite easy to create decent looking plots with your dataframe, The **plot** method on Series and DataFrame is just a simple wrapper around Matplotlib plt.plot() and you really don’t have to write those long matplotlib…
110
ScyllaDB Open Source 3.0
With the release of ScyllaDB Open Source 3.0, we’ve introduced a rich set of new production-ready features that surpass the capabilities of Apache Cassandra.
111
PostgreSQL example of self-contained stored procedures
Derek Sivers official site. Thoughts on philosophy, culture, self-improvement. Author of Useful Not True, How to Live, Hell Yeah or No, Anything You Want.
112
How We Built a Vectorized SQL Engine
Using vectorized processing in an execution engine makes more efficient use of modern CPUs by changing the data orientation (from rows to columns) to get more out of the CPU cache and deep instruction pipelines by operating on batches of data at a time.
113
File systems unfit as distributed storage back ends: 10 years of Ceph
This paper is by Abutalib Aghayev (Carnegie Mellon University), Sage Weil (Red Hat Inc.), Michael Kuchnik (Carnegie Mellon University), Mar...
114
On Sharding
If you need to handle really a lot of traffic, there’s only one way to do it: sharding. Which is to say, splitting up the incoming requests among as many hosts (or Lambda functions, or message brokers, or data streams) as you need. Once you get this working you can handle an essentially unlimited request volume. Of…
115
An introduction to data processing on the Linux command line
This article will provide the reader with a brief overview for a number of different Linux commands. A special emphasis will be placed on explaining how each command can be used in the context of performing data science tasks. The goal will be to convince the reader that each of these commands can be extremely useful…
116
Introduction to Datalog
In contrast to the imperative programming paradigm, declarative programming expresses the logic of a computation without explicitly specifying its control flow. That is, declarative programming focuses on the what , not the how . If you have tentatively researched declarative programming before, you have likely read…
117
118
119
MySQL Challenge: 100k Connections
It's not beyond possibility to set up MySQL to achieve 100,000 concurrent connections if you balance network config and use thread pools. By Vadim Tkachenko
120
Natural Language Processing: The Age of Transformers
This article is the first installment of a two-post series on Building a machine reading comprehension system using the latest advances in deep learning for NLP.