181
A LLM+OLAP Solution
The exploration of a LLM+OLAP solution is a bumpy journey, but phew, it now works well for the Tencent case, and they're writing down every lesson learned to share with you.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 212 posts.
182
We Built a Streaming SQL Engine
A streaming SQL engine keeps queries’ results up to date without ever having to recalculate them, even as the underlying data changes. To explain this, imagine a simple query, such as SELECT count(*) FROM humans
183
Indexing a billion pages
It’s two years since we launched Mwmbl, the open source, non-profit search engine, on Boxing Day 2021. A good time to take stock of where we are and where we’re going. We’ve indexed over 100 million pages Thanks to our volunteers, who crawl the web using the Firefox extension and command line script, we’re crawling up…
184
Daft: A High-Performance Distributed Dataframe Library for Multimodal Data
Discover how Daft accelerates multimodal data processing with a high-performance distributed dataframe engine built for modern AI and analytics tasks.
185
The challenges of supporting foreign key constraints
Today, PlanetScale launched support for foreign key constraints. This article covers some of the behind-the-scenes technical challenges we had to overcome to support them.
186
Rerun 0.9 – a framework for visualizing streams of multimodal data
We're releasing Rerun 0.9.0 with a completely revamped API that gives more control over the underlying data model, a hierarchical and time aware Entity Component System (ECS).
187
Modern Transactional Stack
Transactional databases have long been the most critical component of application design. Why? Because a steadfast database is generally the ultimate enforcement point for correctness in a messy, distributed world. Without them we’d overpay and undercharge. We’d lose riders trying to get home from the airport, and…
188
Open source LLM with 32k Context Length
Following up on the work from our previous blog post, we are releasing today an arXiv paper titled “Giraffe: Adventures in Expanding Context Lengths in LLMs”. Giraffe is a new family of models that are...
189
Open-Source Data Collection Platform for LLM Fine-Tuning and RLHF
We are thrilled to introduce Argilla Feedback, completely open-source and designed to enhance the performance, safety and reliability of Large Language Models.
190
Strong Consistency with Raft and SQLite
In the recent past, a typical client/server environment consisted of multiple clients connected to a central database server. A central database architecture is a system where all data is stored in a single centralized location. When client A from Si...
191
What's Postgres got to do with AI?
Chris is cooking up some examples for us on how to use AI inside Postgres. We have a simple demo of using pgvector and Open AI to grab relationships and query for other similar items.
192
Representing Enums in PostgreSQL
Recently, we had a discussion about whether we should use native enums in PostgreSQL, or rely on regular string columns with CHECK constraints. In the end, we decided that we wanted to go with the latter.
193
Around the World with SQLite3 and Rsync
Run sqlite3 dbs in multiple regions with rsync
194
SQLite builds for WASI since 3.41.0
A short recap of how SQLite now has a WASI build
195
Full-text search engine with PostgreSQL (part 2): Postgres vs. Elasticsearch
Using PostgreSQL as a full-text search engine is tempting because it requires less infrastructure. But is its set of search-related features enough to compete with the Lucene based alternatives?
196
I stopped worrying and learned to love denormalized tables
Glean is the Enterprise AI platform connected to your enterprise's data. Find, create, and automate anything. Explore what Glean can do for you!
197
ScyllaDB is Moving to a New Replication Algorithm: Tablets
How moving from Vnode-based replication to tablets helps dynamically distribute data across the cluster, ultimately increasing ScyllaDB’s elasticity.
198
DuckDB 0.7.0
The DuckDB team is happy to announce the latest DuckDB version (0.7.0) has been released. This release of DuckDB is named “Labradorius” after the Labrador Duck (Camptorhynchus labradorius) that was native to North America. To install the new version, please visit the installation guide. The full release notes can be…
199
Apache Hudi vs. Delta Lake vs. Apache Iceberg Lakehouse Feature Comparison
A thorough comparison of the Apache Hudi™, Delta Lake, and Apache Iceberg™ data lakehouse projects across features, community, and performance benchmarks. This includes a focus on common use cases such as change data capture (CDC) and data ingestion.
200
HTTP vs. WebSockets: Which one is the fastest for Postgres queries at the edge?
Faster is always better, especially when executing SQL queries. We recently introduced SQL-over-HTTP to our driver, which previously only supported WebSockets. Why did we do that? And which approach is faster? Our journey developing the WebSockets serverless driver We first developed the Neon driver to help developers…
201
SVG Images from Postgres
We are excited to announce a new set of functions to generate svgs from Postgres and PostGIS! This gives you really easy maps, images, or charts directly from your database. Once you start reading through these samples, you'll want to start playing with images from your database.
202
203
Getting 10TB of GitHub logs and extracting details of all users and repositories
Trickest workflow parses 15TB of GitHub Archive logs and extracts public details for every user and repository back to 2015.
204
Apache Pinot 1.0
Introducing Apache Pinot 1.0 Release
205
Vitess 18
Vitess 18 is now generally available, with a number of new enhancements designed to improve usability, performance, and MySQL compatibility.
206
207
WebAssembly is eating the database?
User-Defined Functions: powered by WebAssembly
208
20x faster than pgvector: HNSW index in Postgres with pg_embedding
We’re excited to announce the release of our pg_embedding extension for Postgres and LangChain! The new pg_embedding extension brings 20x the speed for 99% accuracy to graph-based approximate nearest neighbor search to your Postgres databases. While the pgvector extension with IVFFlat indexing has been a popular…
209
When should you use the IN instead of the OR operator in Postgres?
Query performance often varies depending on how you write queries. For PostgreSQL, what provides better performance? The IN or OR operator?
210
Pipeworld: Spreadsheet Dataflow Computing (2021)
Now for something completely different. In the spiritual vein of One Night in Rio: Vacation photos from Plan9 and AWK for multimedia, here is a tool that is the link that ties almost all the projec…