61
Solving a linear optimization problem on incentive allocation
At Lyft, scientists solve all kinds of optimization problems. While solvers can come in handy, there are times when complicated or…
New
The latest blog posts to take off on Hacker News. 272 posts.
62
63
Microsoft’s visual data explorer SandDance open sourced
SandDance, the beloved data visualization tool from Microsoft Research, has been re-released as an open source project on GitHub.
64
Books: an immutable double-entry accounting database service
Tracking financial transactions at scale
65
Apache Arrow Flight: A Framework for Fast Data Transport
This post introduces Arrow Flight, a framework for building high performance data services. We have been building Flight over the last 18 months and are looking for developers and users to get involved.
66
Amazon’s Consumer Business Turned Off Final Oracle Database
Over my 17 years at Amazon, I have seen that my colleagues on the engineering team are never content to leave good-enough alone. They routinely re-evaluate every internal system to make sure that it is as scalable, efficient, performant, and secure as possible. When they find an avenue for improvement, they will use…
67
Shareable Jupyter Notebooks That Run on Free Cloud GPUs
Gradient Community Notebooks provide free GPUs with pre-configured environments for machine learning and deep learning projects.
68
AWS Elasticsearch: a fundamentally-flawed offering
I'm currently working on a logging project that was initially implemented using AWS Elasticsearch. Having worked with large-scale mainline Elasticsearch clusters for several years, I'm absolutely stunned at how poor Amazon's implementation is and I can't fathom why they're unable to fix or at least improve it. I…
69
PyPy's New JSON Parser
Introduction In the last year or two I have worked on and off on making PyPy's JSON faster, particularly when parsing large JSON files. I...
70
71
Scribe: Transporting petabytes per hour via a distributed, buffered queueing
Our hardware infrastructure comprises millions of machines, all of which generate logs that we need to process, store, and serve. The total size of these logs is several petabytes every hour. The o…
72
Sqlite 3.30.0
This page provides a high-level summary of changes to SQLite. For more detail, see the Fossil checkin logs at https://sqlite.org/src/timeline and https://sqlite.org/src/timeline?t=release . See the chronology a succinct listing of releases.
73
74
How much faster is Redis at storing a blob of JSON compared to Postgres?
tl;dr; Redis is 16 times faster at reading these JSON blobs.*
75
New In Postgres 12: Generated Columns
pgDash is an in-depth monitoring solution designed specifically for PostgreSQL deployments. pgDash shows you information and metrics about every aspect of your PostgreSQL database server, collected using the open-source tool pgmetrics.
76
Processing 40TB of code from 10M projects with a dedicated server and Go
The command line tool I created Sloc Cloc and Code (scc) (which is now modified and maintained by many other excellent people) counts lines of code, comments and makes a complexity estimate for files …
77
On Sharding
If you need to handle really a lot of traffic, there’s only one way to do it: sharding. Which is to say, splitting up the incoming requests among as many hosts (or Lambda functions, or message brokers, or data streams) as you need. Once you get this working you can handle an essentially unlimited request volume. Of…
78
Designing far memory data structures: think outside the box
Designing far memory data structures: think outside the box Aguilera et al., HotOS’19 Last time out we looked at some of the trade-offs between RInKs and LInKs, and the advantages of local in…
79
Efficient GraphQL Queries in Ruby on Rails and Postgres
We discuss what N+1 queries are, how they are easily produced in GraphQL, and how to solve them using the graphql-batch gem along with a few custom batch loaders.
80
Comparing Database Types
Many types of databases exist, each with their own benefits. In this guide, we compare the different types of databases and what each of them offer.
81
Building a Relational Database Using Kafka
In a previous post, I showed how Kafka can be used as the persistent storage for an embedded key-value store, called KCache. Once you have a key-value store, it can be used as the basis for other m…
82
Weld: Accelerating numpy, scikit and pandas as much as 100x with Rust and LLVM
Lambdaclass's blog about distributed systems, machine learning, compilers, operating systems, security and cryptography.
83
84
85
Challenges in Implementing a Full-Text-Search Engine
While Full-text-search is famous for search through documents at a faster rate, there are other interesting ways in which these systems handle scale, synonyms, abbreviations, misspellings etc., we will
86
87
New Query Language for Graph Databases to Become International Standard
Read the latest Neo4j press releases and company news, including product launches, partnerships, and milestones in graph database and analytics.
88
Announcing Yugabyte DB 2.0 GA: High-Performance Distributed SQL
YugabyteDB 2.0 released. Read to learn about production-ready YSQL, performance benchmarks, Jepsen results, and Beta feature previews
89
Comprehensive Guide on Data Visualization with Pandas
Visualization has always been challenging task but with the advent of dataframe plot() function it is quite easy to create decent looking plots with your dataframe, The **plot** method on Series and DataFrame is just a simple wrapper around Matplotlib plt.plot() and you really don’t have to write those long matplotlib…
90
Harvesting LinkedIn data for fun and profit
A few years ago, before the GDPR era, I was involved in a stealth marketing startup. We went out of