91
Building a semantic search engine in Rust
If you're interested in building a semantic search engine in Rust, this article will show you how! We'll cover building a basic engine using sentence transformers and a kd-tree.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 125 posts.
92
ULIDs and Primary Keys
Not all non-sequential primary keys are created equal, the ULID format is a fantastic substitute for the typical UUIDv4 choice.
93
94
95
MySQL now shows its thread names at OS level for better troubleshooting
Starting from v8.0.27, the MySQL daemon shows its thread names at the OS level too, for better troubleshooting. The MySQL 8.0.27 release notes say this: To assist monitoring and troubleshooting, the Performance Schema instrumentation is now used to export names of instrumented threads to the operating system. This…
96
I disagree with Turing and Kahneman regarding statistical evidence (2014)
It’s funny. I’m the statistician, but I’m more skeptical about statistics, compared to these renowned scientists.
97
Some thoughts on machine learning with small data
I feel techniques for making machine learning work with small data are not talked about enough. This makes sense because many ML applications are only possible by collecting a huge amount of data. …
98
Revisiting the principles of data-oriented programming
Principles of Data-Oriented Programming. Benefits and costs of Data-Oriented Programming. DOP vs OOP. DOP vs FP.
99
Push-based outbox pattern with Postgres logical replication
There are only a few patterns that I’m comfortable saying: “You should use it always if you want to build mature system”. One of them is the Outbox Pattern. Why? Because…
100
Where is every IP Address?
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
101
Let's Talk SkipList
the forgotten data structure (not really)
102
How to Find and Stop Running Queries on PostgreSQL
Your PostgreSQL server is seizing up, with some naughty queries consuming too many resources or blocking others. Don’t panic! You can stop those problem queries and stabilize your system.
103
Maintaining sufficient free space with ZFS
TLDR: ZFS free-space reporting is a lagging indicator. Background I use Proxmox VM server backed by a ZFS array of hard drives for various personal infrastructure. I also have security cameras that upload motion-triggered videos to my server (via FTP!). Problem Description I would like to use 90% of my available space…
104
105
It's not Ruby that's slow, it's your database
Many people keep repeating that Ruby is slow. It is. But that doesn't matter, because your database is so much slower that it is the bottleneck. So, an alternative title would be "Ruby is slow, but that doesn't matter for you."
106
Dicom File Format Basics
Explore the fundamentals of the DICOM file format! This quick introduction covers the basics of DICOM's structure, its essential uses, and tips for easily navigating its complex and abstract components.
107
SQL + M4 = Composable SQL
I often work with clients who have large “data lakes” or big star schema style enterprise databases with fact and dimension tables as far as the eye can see. Invariably said clients end up with a substantial SQL codebase composed of hundreds of independent queries with lots of overlap between them. I want to be able…
108
A sequel to SQL? An intro to Malloy
Homepage of Carlin Eng
109
Filesystems can experience at least three different sorts of errors
You're probably reading this page because you've attempted to access some part of my blog (Wandering Thoughts) or CSpace , the wiki thing it's part of. Unfortunately you're using a browser version that my anti-crawler precautions consider suspicious, most often because it's too old (most often this applies to versions…
110
Overview of Consistency Levels in Database Systems
Database systems typically give users the ability to trade off correctness for performance. We have spent the previous two posts in this s...
111
A/B test improved your website's conversion rate? Not so fast
Many internet companies use A/B testing to evaluate the performance of website changes they want to make. A typical metric in such scenarios is the conversion rate (CVR) of a user, i.e. the probability that a user converts (e.g. makes a purchase) on the website. A typical A/B test setup in such a situation would…
112
Partitioning in Postgres, 2022 Edition
Tracking improvements to Postgres partitioning since 2017. In just five years it’s gone from a non-feature to a very good one.
113
Do data-driven companies win?
Some gut-based judgements on the effectiveness of data.
114
Simple declarative schema migration for SQLite
At my company we use a small SQLite database. We define the schema in a single file with SQL CREATE TABLE statements. If we add tables, columns, or indexes, our application will create them automatically the next time it starts up.
115
Squeezing Performance from SQLite: EXPLAINing the Virtual Machine
The virtual machine? Yep, but not the Java Virtual Machine.
116
The Economics of Data Businesses
How data businesses start, and how they keep going, and growing, and growing.
117
118
The Plausibly Deniable DataBase (PDDB)
The problem with building a device that is good at keeping secrets is that it turns users into the weakest link.
119
Surprises in the Bitcoin blockchain and how they are stored (2014)
Every Bitcoin transaction is stored in the distributed database known as the Bitcoin blockchain. However, people have found ways to hack ...
120