D1: Improvements to performance and scalability
We've been heads down iterating on D1, and we've just shipped a major new version that's substantially faster, more reliable, and introduces Time Travel: the ability to restore a D1 database to any point in time.
The Architecture of Serverless Data Systems
I recently blogged about why I believe the future of cloud data services is large-scale and multi-tenant, citing, among others, S3. “ Top tier SaaS services like S3 are able to deliver amazing simplicity, reliability, durability, scalability, and low price because their technologies are s
Summing columns in remote Parquet files using DuckDB
vivym/midjourney-messages on Hugging Face is a large (~8GB) dataset consisting of 55,082,563 Midjourney images - each one with the prompt and a URL to the image hosted on Discord.
Approximate Nearest Neighbor Oh Yeah (Annoy)
Discover the capabilities of Annoy, an innovative algorithm revolutionizing approximate nearest neighbor searches for enhanced efficiency and precision.
Running Databases on Kubernetes
Things to consider when running a database on Kubernetes.
AlloyDB Omni – run AlloyDB anywhere
Today, we’re excited to announce the technology preview of AlloyDB Omni, a downloadable edition of AlloyDB designed to run on premises, at the edge, across clouds, or even on developer laptops.
The new local data stack: Integrating Dagster, dbt and DuckDB
Lean and efficient MDS experience: Delivers better software engineering practices to the data ecosystem with the new local MDS stack comprised of Dagster, dbt and DuckDB which offers better developer productivity by enhancing testability of the E2E pipeline.
WAL Mode in LiteFS
LiteFS v0.3.0 now supports SQLite's write-ahead log journal mode.
Short, friendly base32 slugs from timestamps
~4 lines of Go to produce succinct, stable, URL-friendly slugs from timestamps.
Demystifying Text Data with the Unstructured Python Library
In the world of data, textual data stands out as being particularly complex. It doesn’t fall into neat rows and columns like numerical data does. As a side project, I’m in the process of developing my own personal AI assistant. The objective is to use the data within my notes and documents to answer my questions. The…
Upsert in SQL
Inserting and updating data with a single query.
SQLite 2022 Recap
There have been many improvements to SQLite during 2022. Listed below or just a few of the bigger changes. See the change logs or the trunk timeline for a more complete list of enhancements and improvements.
Optimizing LLMs from a Dataset Perspective
Practical guide to improving LLM finetuning with better instruction datasets, covering data curation, prompt-output pairs, synthetic data, and experiment ideas.
MongoDB’s new query engine
SBE Article logo Discussion on HackerNews and Lobsters. MongoDB has recently released a new query engine coming in version 7.0. I was one of the people work...
Apache Kafka Beyond the Basics: Windowing
Learn what windowing is, the difference between the four types of windows (hopping and tumbling, or session and sliding), and how to create them.
WordPress Core to start using SQLite
This post is an update to the proposal for officially support SQLite in WordPress. The initial implementation was included in the Performance Lab plugin and then released as a stand-alone plugin. T…
Fixing Hallucination with Knowledge Bases
Large Language Models (LLMs) have a data freshness problem. Even some of the most powerful models, like GPT-4, have no idea about recent events. Let's learn how to fix this.
Type constraints in 65 lines of SQL
Creating validated data types in Postgres
Modern Polars: A comparison of the Polars and Pandas dataframe libraries
A side-by-side comparison of the Polars and Pandas libraries.
Visualizing every job in the world
A visualization of every single job in the world (or at least in the EU)
UK Address Oddities (2018)
A few of my previous blog posts have been about quirks in the UK postal address system. As a result of these posts being moderately popular, people have contacted me to tell me about other such quirks. This pleases me greatly – I like quirks.
Which vector database should I use? A comparison cheatsheet
Semantic search and retrieval-augmented generation (RAG) applications require systems to be able to save lots of embedding vectors…
PlanetScale Scaler Pro
See how Scaler Pro combines the best of our current plans for companies of all sizes while enabling you to grow with the best database for serverless.