961
Doing a database join with CSV files
How to combine data spread over two CSV files, like separate tables in a normalized relational database.
New
The latest blog posts to take off on Hacker News. 1,888 posts.
962
963
Stock Option Pricing Inference
I’m a stock market enthusiast 📈, so lately I have spent my spare time on the topic of stock option pricing, and I was curious to find out whether or not I could build a script that accurately reflected real market option prices. It turns out I came close, and learned a lot along the way, which I share in this article.
964
Xor Filters: Faster and Smaller Than Bloom Filters
In software, you frequently need to check whether some objects is in a set. For example, you might have a list of forbidden Web addresses. As someone enters a new Web address, you may want to check whether it is part of your black list. Or maybe you have a large list of already used … Continue reading Xor Filters…
965
Dynamo vs. Cassandra: Systems Design of NoSQL Databases (2018)
A comparison of ideas behind Dynamo and Cassandra
966
Why databases use ordered indexes but programming uses hash tables
I think it is safe to state that hash tables (e.g. maps in Go, dicts in Python, HashMap in Java, etc.) are far more common than ordered data structures such as trees for in-memory data structures. One data point is that in a talk about C++ hash table optimization at Google ( available in Abseil ), the speaker noted…
967
Using machine learning to choose compression algorithms
30% less disk required compared to using the single best strategy
968
File systems unfit as distributed storage back ends: 10 years of Ceph
This paper is by Abutalib Aghayev (Carnegie Mellon University), Sage Weil (Red Hat Inc.), Michael Kuchnik (Carnegie Mellon University), Mar...
969
PostgreSQL at low level: stay curious
Yes, I know, it’s a long text, and it was my conscious decision to write it in this way. But fear not! Imagine that you read a book, take a look at the introduction and first few interesting sections, think about it and then find time to read further. I hope I’ve left enough references, so if you don’t get some ideas…
970
What’s Wrong with Bayes
My problem is not just with the methods—although I do have problems with the method—but also with the ideology.
971
Mergeable replicated data types – Part I
Mergeable replicated data types Kaki et al., OOPSLA’19 This paper was published at OOPSLA, but perhaps it’s amongst the distributed systems community that I expect there to be the greatest in…
972
An introduction to data processing on the Linux command line
This article will provide the reader with a brief overview for a number of different Linux commands. A special emphasis will be placed on explaining how each command can be used in the context of performing data science tasks. The goal will be to convince the reader that each of these commands can be extremely useful…
973
Common Mistakes and Missed Optimization Opportunities in SQL
Made by Developers and Non-Developers
974
When your data doesn’t fit in memory: the basic techniques
You can process data that doesn’t fit in memory by using four basic techniques: spending money, compression, chunking, and indexing.
975
The Next 50 Years of Databases (2015)
Andy attempts to revive his blog by predicting what databases will look like in the year 2065.
976
Building a CRUD App with Datomic Cloud Ions
Software engineer, etc etc
977
Modern Data Practice and the SQL Tradition
This made it to the Hacker News front-page too: some very interesting comments there
978
File systems unfit as distributed storage back ends: lessons from Ceph evolution
File systems unfit as distributed storage backends: lessons from 10 years of Ceph evolution Aghayev et al., SOSP’19 Ten years of hard-won lessons packed into just 17 pages (13 if you don’t co…
979
Layers in Software: From Data to Value
Then Back in the 2000s, we wrote applications in layers. Presentation layer, client, data transfer, API, business logic, data access, database. We maintained strict separation bet ween these layers…
980
FastSpark: A New Fast Native Implementation of Spark from Scratch
TLDR: Here is the code to explore.
981
982
983
The Most Gender-Switched Names in the U.S.
We use some names mostly for boys and some mostly for girls, but then there is a small percentage that, over time, switched from one gender to another. Which names made the biggest switch?
984
SQLite is easy to compile
SQLite is really easy to compile
985
PostgreSQL example of self-contained stored procedures
Derek Sivers official site. Thoughts on philosophy, culture, self-improvement. Author of Useful Not True, How to Live, Hell Yeah or No, Anything You Want.
986
AWS Elasticsearch: a fundamentally-flawed offering
I'm currently working on a logging project that was initially implemented using AWS Elasticsearch. Having worked with large-scale mainline Elasticsearch clusters for several years, I'm absolutely stunned at how poor Amazon's implementation is and I can't fathom why they're unable to fix or at least improve it. I…
987
988
How much faster is Redis at storing a blob of JSON compared to Postgres?
tl;dr; Redis is 16 times faster at reading these JSON blobs.*
989
Processing 40TB of code from 10M projects with a dedicated server and Go
The command line tool I created Sloc Cloc and Code (scc) (which is now modified and maintained by many other excellent people) counts lines of code, comments and makes a complexity estimate for files …
990
On Sharding
If you need to handle really a lot of traffic, there’s only one way to do it: sharding. Which is to say, splitting up the incoming requests among as many hosts (or Lambda functions, or message brokers, or data streams) as you need. Once you get this working you can handle an essentially unlimited request volume. Of…