931
An automatic indexing system for Postgres
The pganalyze Indexing Engine simulates hundreds of possible combinations to recommend missing Postgres indexes using its 'What if?' analysis.
New
The latest blog posts to take off on Hacker News. 4,224 posts.
932
I analyzed Stack Overflow for secrets
I analyzed stackoverflow for secrets and leaks.
933
Operating on a minimal two-core Postgres instance: Query optimization insights
In an era where the costs for cloud services continue to soar, I want to outline how our data team has successfully managed to run the infrastructure of our energy trading company within the first year using only two PostgreSQL database instances - each equipped with just two CPU cores and four GB of RAM. Despite this…
934
The Architecture of Serverless Data Systems
I recently blogged about why I believe the future of cloud data services is large-scale and multi-tenant, citing, among others, S3. “ Top tier SaaS services like S3 are able to deliver amazing simplicity, reliability, durability, scalability, and low price because their technologies are s
935
Summing columns in remote Parquet files using DuckDB
vivym/midjourney-messages on Hugging Face is a large (~8GB) dataset consisting of 55,082,563 Midjourney images - each one with the prompt and a URL to the image hosted on Discord.
936
Backblaze Drive Stats for Q3 2023
Explore Backblaze's Q3 2023 hard drive stats, revealing insights on 259,533 HDDs, failure rates, and data center performance for enhanced reliability.
937
938
Loro: Reimagine state management with CRDTs
Loro, our high-performance CRDTs library, is now open source. In this article, we share our vision for the local-first software development paradigm, why we're excited about it, and the current status of Loro.
939
Maxar's Open Satellite Feed
Benchmarks & Tips for Big Data, Hadoop, AWS, Google Cloud, PostgreSQL, Spark, Python & More...
940
Open, rigorous and reproducible research: A practitioner’s handbook (2021)
Even though many scientists are expected to abide by the principles of open, rigorous and reproducible research, few scientists actually receive formal training in this. Indeed, there is no collection written by working scientists for working scientists about the principles of transparent, rigorous and reproducible…
941
Debunking the Myth of "Anonymous" Data
Today, almost everything about our lives is digitally recorded and stored somewhere. Each credit card purchase, personal medical diagnosis, and preference about music and books is recorded and then used to predict what we like and dislike, and—ultimately—who we are. This often happens without our...
942
SciPy builds for Python 3.12 on Windows are a minor miracle
Moving SciPy to Meson meant finding a different Fortran compiler on Windows, which was particularly tricky to pull off for conda-forge. This blog tells the story about how things looked pretty grim for the Python 3.12 release, and how things ended up working out just in the nick of time.
943
Kurdish Parentheses on OpenStreetMap, Three Ways
While making a proposal to update Noto fonts for OpenStreetMap tiles, which fixes an issue with Kurdish labels using ڵا , I noticed a street with mismatched parentheses and was anticipating a deep dive into the tile renderer. But the actual problem was somewhere else:
944
Vitess 18
Vitess 18 is now generally available, with a number of new enhancements designed to improve usability, performance, and MySQL compatibility.
945
PostgreSQL Encryption: The Available Options
On an episode of Postgres FM , the hosts had a (very brief) discussion of data encryption in PostgreSQL. While Postgres FM is a podcast well worth a subscribe, the hosts aren’t data security experts, and so as someone who builds a queryable database encryption system , I found the coverage to be somewhat… lacking. I…
946
Building a Streaming Platform in Go for Postgres
At PeerDB, our mission is to create a Postgres-first data-movement platform that makes it fast and simple to stream data from Postgres to Data Warehouses…
947
Updates to the H2O.ai db-benchmark
The H2O.ai db-benchmark has been updated with new results. In addition, the AWS EC2 instance used for benchmarking has been changed to a c6id.metal for improved repeatability and fairness across libraries. DuckDB is the fastest library for both join and group by queries at almost every data size.
948
Tracking SQLite Database Changes in Git
Note : This post stirred up some interesting discussions on HackerNews and Lobste.rs .
949
SQLite 3.44: Interactive release notes
Order by in aggregates, better string concatenation, and thorough date formatting.
950
Rules of schema growth (2017)
The free database designed for immutable values and a time-aware information model.
951
RedPajama v2 Open Dataset with 30T Tokens for Training LLMs
RedPajama-Data-v2 is a 30 trillion token open dataset from 84 CommonCrawl dumps across five languages, with over 40 precomputed quality annotations.
952
Approximate Nearest Neighbor Oh Yeah (Annoy)
Discover the capabilities of Annoy, an innovative algorithm revolutionizing approximate nearest neighbor searches for enhanced efficiency and precision.
953
Why you should probably be using SQLite
Where you store your application data has enormous impacts on your entire application. There are implications on the entire stack based on what you decide to...
954
Making PostgreSQL tick: New features in pg_cron
Since pg_cron 1.5, you can easily schedule jobs that run every 1-59 seconds. And pg_cron 1.6 added support for Postgres 16. This blog post walks you through the benefits of second granularity jobs with the pg_cron open source extension to Postgres.
955
The Case of a Curious SQL Query
Languages that suffer success often have to do so by selling out and adding features that go against some of the original purposes of their design. SQL is a...
956
Nile: Serverless Postgres for modern SaaS
I am thrilled to announce Nile, a serverless Postgres database designed for modern SaaS applications.
957
Embeddings: What they are and why they matter
Embeddings are a really neat trick that often come wrapped in a pile of intimidating jargon. If you can make it through that jargon, they unlock powerful and exciting techniques …
958
Excel will allow certain auto data conversions to be turned off
Based on your feedback, we've improved the Automatic Data Conversion settings, and made them also available in Excel for Mac.
959
OpenTelemetry at Scale: Using Kafka to handle bursty traffic
Learn how to use Apache Kafka as a buffer between OpenTelemetry Collectors for a scalable multi-collector pipeline, with YAML configuration examples.
960
Transactions Are Not Locks (2022)
One thing I wish I had understood better earlier on in my experience with PostgreSQL is how transactions and locks can be used together to provide serializable logic.