1171
Improved fraud prevention with Radar 2.0
Radar 2.0 improves machine learning performance and reduces fraud by up to an additional 25%.
Topics /
Databases & Data
The most popular blog posts about databases & data on Hacker News. 2,337 posts.
1172
RethinkDB 1.12: simplified map/reduce, ARM port, new caching infrastructure
Today, we’re happy to announce RethinkDB 1.12 ( The Wizard of Oz ). Download it now!
1173
Lower Cost S3 Storage Option and Glacier Price Reduction
Like all AWS services, the Amazon S3 team is always listening to customers in order to better understand their needs. After studying a lot of feedback and doing some analysis on access patterns over time, the team saw an opportunity to provide a new storage option that would be well-suited to data that is accessed […]
1174
Best practices for migrating an Oracle database to Amazon PostgreSQL
An Oracle to PostgreSQL migration in the AWS Cloud can be a complex multistage process with different technologies and skills involved, starting from the assessment stage to the cutover stage. This blog series covers the environment and configuration setups for your source Oracle database, the AWS Database Migration…
1175
Apache Arrow: A new open source in-memory columnar data format
Open source Big Data in-memory columnar layer accelerates analytical processing and interchange by more than 100x. Forest Hill, MD –17 Feb 2016– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache…
1176
Using logistic regression to predict parking difficulty
Posted by James Cook, Yechen Li, Software Engineers and Ravi Kumar, Research Scientist"When Solomon said there was a time and a place for everythin...
1177
Open-Source Babelfish for Postgres
Babelfish for PostgreSQL is an Apache-2.0 and PostgreSQL open source project that adds a Microsoft SQL Server-compatible end-point to PostgreSQL to enable your PostgreSQL database to understand the SQL Server wire protocol and commonly used SQL Server commands.
1178
How to Read MySQL EXPLAINs
Learn how to read the output in MySQL EXPLAIN plans so you can utilize them to improve query performance.
1179
Footguns with Postgres “at time zone 'UTC'”
And why you may need to repeat AT TIME ZONE 'UTC' twice. SUMMARY * AT TIME ZONE 'UTC' converts the data type from timestamptz to timestamp (example). * Now you are inadvertently using the timestamp without time zone (aka timestamp) data type, which is markedly discouraged. * There are tons of footguns with timestamp…
1180
LevelDB: A Fast Persistent Key-Value Store
The latest news from Google on open source releases, major projects, events, and outreach programs for early career developers.
1181
Microsoft, UW demonstrate first fully automated DNA data storage
Researchers encode the word “hello” in fabricated DNA, convert it back to digital using fully automated DNA storage system.
1182
Hard Drive Stats for Q3 2017
We take a look at the performance of our 86,529 hard drives and break down the failure rates by manufacturer and model size.
1183
Realtime Postgres Row Level Security
Realtime database changes are now broadcast to authenticated users, respecting the same PostgreSQL policies that you use for Row Level Security.
1184
1185
Debunking Misleading Benchmarks Of Redshift vs BigQuery
Publishing misleading performance benchmarks is a classic old guard marketing tactic. It’s not surprising to see old guard companies (like Oracle) doing this, but we were kind of surprised to see Google take this approach, too. So, when Google presented their BigQuery vs. Amazon Redshift benchmark results at a private…
1186
PostgreSQL 14 on Kubernetes
Run PostgreSQL 14 on Kubernetes, and see how convenient it is to use distributed features like parallel remote Postgres queries!
1187
1188
1189
NumPy 1.20
This NumPy release is the largest so made to date, some 684 PRs contributed by 184 people have been merged. See the list of highlights below for more details. The Python versions supported for this release are 3.7-3.9, support for Python 3.6 has been dropped. Highlights are
1190
Memory efficiency of parallel IO operations in Python
Python allows for several different approaches to parallel processing. The main issue with parallelism is knowing its limitations. We either want to parallelise IO operations or …
1191
SQLite Begin Concurrent
Usually, SQLite allows at most one writer to proceed concurrently. The BEGIN CONCURRENT enhancement allows multiple writers to process write transactions simultanously if the database is in "wal" or "wal2" mode, although the system still serializes COMMIT commands.
1192
NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute
We've achieved 10x data efficiency with NanoGPT Slowrun within a few weeks. An ensemble of 1.8B parameter models (18B total params) trained on 100M tokens matches what would normally require 1B tokens with a standard LM baseline. Data efficiency matters because compute grows much faster than data . Since our current…
1193
Understanding Recursive Queries in PostgreSQL
This article gives a simple introduction with examples to recursive queries in PostgreSQL. It also includes a comparison to Oracle recursive queries.
1194
Nvidia Scientists Take Top Spots in 2021 Brain Tumor Segmentation Challenge
NVIDIA data scientists took three of the top spots in a brain tumor segmentation challenge validation phase at the prestigious MICCAI 2021.
1195
PostgreSQL Streaming Replication (WAL); What It Is and How to Configure One
What is Streaming replication in PostgreSQL?Streaming replication in PostgreSQL is an efficient method for maintaining a near real-time replica of a primary database on one or more standby servers.The primary server continuously sends data to replica
1196
Introducing the Open Images Dataset
Posted by Ivan Krasin and Tom Duerig, Software EngineersIn the last few years, advances in machine learning have enabled Computer Vision to progres...
1197
Waiting for Postgres 16: Cumulative I/O statistics with pg_stat_io
We talk about querying system wide I/O statistics in Postgres, show several use cases for pg_stat_io, and talk about the future of I/O observability in Postgres.
1198
Anatomy of a SQL Engine
Blog for DoltHub, a website hosting databases made with Dolt, an open-source version-controlled SQL database with Git-like semantics.
1199
The evolution of the data engineer role
The Evolution of The Data Engineer: A Look at The Past, Present & Future
1200
Meta quickly detects silent data corruptions at scale
Silent data corruptions (SDCs), data errors that go undetected by the larger system, are a widespread problem for large-scale infrastructure systems. Left undetected, these types of corruptions can…