481
ChatLZMA – text generation from data compression
I came across a random tweet, found a Wikipedia page and bumped into some smart people and long story short, apparently compression is equivalent to general intelligence.
New
The latest blog posts to take off on Hacker News. 1,888 posts.
482
Pgvector 0.5.0 Feature Highlights and how tos
An overview of new features in pgvector 0.5.0, including HNSW, parallel index builds, and more!
483
The broad set of computer science problems faced at cloud database companies
A couple of weeks ago, I was in an internal meeting where our CTO (Adam Prout) was giving a talk to our summer interns. The talk was about the current state of database companies, and how different providers are competing on different facets of their products. As the talk was
484
All of the vehicle license plates available in America
States now offer a vast menu of personalized plate options for a dizzying array of organizations, professions, sports teams, causes and other groups.
485
Most companies do not need Snowflake or Databricks
The cost for something that can be replicated free and open source is absurd. The Fortune 100 have a use case for these companies, the rest…
486
487
Retrieving 1TB of data from a faulty drive with the help of woodworking tools
So, a while ago I built a gaming PC with the following specification: Case Dan A4 v4.1 Motherboard Z690I Strix Gaming Pro...
488
489
Analysis of the data job market using HN job posts
I parse HackerNews (HN) “Ask HN: Who is hiring?” posts from 2013 to time of writing and analyse them to better understand the trends in the data job market with a focus on the fate of data science. Here are my main conclusions:
490
Python: Just Write SQL
I have been writing a lot more Go this past year. For those not familiar, Go favours a non-ORM, non-query-builder approach to interacting with databases. This comes naturally due to the sql package : A common interface to be used alongside database drivers. It’s very common to see actual SQL in Go, even in large…
491
492
Which vector database should I use? A comparison cheatsheet
Semantic search and retrieval-augmented generation (RAG) applications require systems to be able to save lots of embedding vectors…
493
So, you want to deploy on the edge?
Application developers often deploy their apps into a single area, generally represented by a handful of AZs in a single region. No matter where their …
494
Gzip beats BERT? Part 2: dataset issues, improved speed, and results
Summary : Update to my last post analyzing the gzip+knn paper. Some of the datasets have heavily contaminated train/test tests. I show some ways to significantly speed-up the gzip method. I present more results of my "fair" reimplementation .
495
Ways to shoot yourself in the foot with Redis
Personal website of Phil Booth
496
Data diffs: Algorithms for explaining what changed in a dataset (2022)
tl;dr: part 1 explains what an explanation algorithm is, and part 2 describes an open source SQL data differ.
497
Visualizing every job in the world
A visualization of every single job in the world (or at least in the EU)
498
Statistical Arbitrage – An Easy Walkthrough
Statistical arbitrage (stat arb) is a pillar of quantitative trading that relies on mean reversion to predict the future returns of an asset. Mean reversion believes that if a stock has risen higher it’s more likely to revert in the short term which is the opposite of a momentum strategy that believes if a stock has…
499
Bad numbers in the “gzip beats BERT” paper?
The recent paper, “Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors by Jiang et al. [ link ] recently received a lot of attention on twitter.
500
There is no data engineering roadmap
Data Engineering is booming and many are looking for a cheat sheet to get started. Do you really need to learn 200 tools before you get your first job?
501
Understanding Kafka with Factorio (2019)
While playing Factorio the other day, I was struck by the many similarities with Apache Kafka.
502
UCLA professor refuses to cover for Dan Ariely in issue of data provenance
In one of the more bizarre twists of the Dan Ariely saga, there is a huge issue of provenance in one of Dan Ariely’s studies. The study in question is Dan Ariely’s Ten Commandments study at JMR. The hugely impactful paper has over 3,000 citations but unfortunately failed to replicate in a large-scale systematic…
503
Demystifying Text Data with the Unstructured Python Library
In the world of data, textual data stands out as being particularly complex. It doesn’t fall into neat rows and columns like numerical data does. As a side project, I’m in the process of developing my own personal AI assistant. The objective is to use the data within my notes and documents to answer my questions. The…
504
Things that surprised me while running SQLite in production
I've been using SQLite in several production projects for six months, and I've encountered some surprising aspects.
505
Joins 13 Ways
Relational (inner) joins are really common in the world of databases, and one weird thing about them is that it seems like everyone has a different idea of what they are. In this post I’ve aggregated a bunch of different definitions, ways of thinking about them, and ways of implementing them that will hopefully be…
506
Using command line to process CSV files (2022)
Process CSV files from the command line with awk, sort, and grep: pull out fields, order rows, and filter data without a spreadsheet or a script.
507
What are embeddings?
A deep-dive into machine learning embeddings.
508
[109] Data Falsificada (Part 1): “Clusterfake”
This is the introduction to a four-part series of posts detailing evidence of fraud in four academic papers co-authored by Harvard Business School Professor Francesca Gino. In 2021, we and a team o…
509
I don't need your query language
Seriously, I don't. I'd prefer SQL.
510
The PostgreSQL documentation and the limitations of community
In my opinion, the PostgreSQL documentation is simultaneously excellent and fairly poor, and both its excellence and its shortcomings are di...