1441
Realtime Data Processing at Facebook
Recently there has been a lot of development in realtime data processing systems, including Twitter's Storm and Heron, Google's Millwheel, a...
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 1,888 posts.
1442
Machine Learning: A Love Story
The video from my keynote at Strange Loop 2010 is up! You can watch the video here: Machine Learning: A Love Story The original abstract: Machine learning has come a long way in recent years — from…
1443
Sorting in Japanese – An Unsolved Problem (2011)
Sorting Japanese is not only difficult—it’s an unsolved problem. This seems hard to believe if you are not familiar with the complexities of processing Japanese digitally. But what is trivially easy in English is impossible in Japanese, even with the amount of computer power we have available today.
1444
Important database news from the last few months
Google and Amazon acknowledge benefits of SQL, Elasticsearch introduces SQL interface, and MySQL finally becomes modern…after 20 years. These are exciting times in the world of databases.
1445
A Blackstone real estate company exposed SSN digits, DOBs, addresses and more
Update: This post received some attention on Hacker News — see the discussion thread.
1446
1447
Analyzing Spotify Stream History
I recently learned Spotify provides downloads for users’ streaming history. For me, this is over 10 years worth of data, so at the very least it seemed like a good nostalgia trip. This post covers some of my analysis of my personal export, and hopefully is a good starting place if you’re interested in exploring your…
1448
The time keepers: pg_cron and pg_timetable
Need to schedule tasks in PostgreSQL? Compare pg_cron for simple in-database scheduling with pg_timetable for complex workflows and multi-cluster orchestration.
1449
Why I Wrote Data Science for Crime Analysis with Python (2023)
To help crime analysts get started with python, I am early releasing Data Science for Crime Analysis with Python, an introductory book for analysts.
1450
High Frequency Trading Strategies
Discusses performance characteristics of high frequency trading strategies and the requirements for implementation
1451
1452
Breaking Bell's Inequality with Monte Carlo Simulations in Python
The article explains the Bell inequality using Monte Carlo simulations in Python, and shows how non-local action-at-a-distance can be used to break it with entangled qubits.
1453
Imagining a personal data pipeline
I've been thinking a lot about personal data lately: where it's stored, how to extract it, and what to do with it. Here's where I landed.
1454
Fundamentals of Object Storage in Elixir
I did a livestream where I talked about Object Storage. The how and why. The Bad Old Days. And also the neat and interesting stuff just beyond the basics. I figured I'd cover that in text as well.
1455
A Closer Look at Transit
Disclaimer: I am an employee of Cognitect and I worked on the transit-js implementation. The opinions found herein are my own and do not necessarily reflect the recommendations or views of Cognitect.
1456
1457
My Thoughts on the Datomic License
I have a greenfield project where the choice of database is open. I have been very interested in Datomic for a couple of years now and figured that I'd try Datomic out for the project. Then I tried to actually download and install it and decided against it... mostly because the value of Datomic does not clearly…
1458
ChatLZMA – text generation from data compression
I came across a random tweet, found a Wikipedia page and bumped into some smart people and long story short, apparently compression is equivalent to general intelligence.
1459
1460
1461
1462
It's not Ruby that's slow, it's your database
Many people keep repeating that Ruby is slow. It is. But that doesn't matter, because your database is so much slower that it is the bottleneck. So, an alternative title would be "Ruby is slow, but that doesn't matter for you."
1463
The principles of database design, or, the Truth is out there
Every software project needs to represent the reality of the business he is embedded in. The way we can represent reality as limited rational beings is through propositions, i.e, declarative statements that affirm or deny something about reality. When a collection of such propositions is stored in a computer system…
1464
Analysis of the data job market using HN job posts
I parse HackerNews (HN) “Ask HN: Who is hiring?” posts from 2013 to time of writing and analyse them to better understand the trends in the data job market with a focus on the fate of data science. Here are my main conclusions:
1465
Domain Logic and SQL (2003)
A long-form article entitled: "Domain Logic and SQL"
1466
MySQL transactions per second vs. fsyncs per second (2020)
Just wondering how many transactions or writes per second MySQL can handle? While it depends on many factors, fundamentally, about as many transactions as MySQL can commit to disk per second. A modern disk can do ~1000 fsyncs per second , but MySQL will group multiple writes with each fsync. An okay rule-of-thumb…
1467
Database Isolation Is Broken and You Should Care
The database zeitgeist has decided that ANSI SQL '92 isn't enough. We need something better.
1468
1469
1470
Building a Relational Database Using Kafka
In a previous post, I showed how Kafka can be used as the persistent storage for an embedded key-value store, called KCache. Once you have a key-value store, it can be used as the basis for other m…