Databricks Blog

DBRX: A new open LLM

Explore DBRX, the advanced open-source LLM from Databricks redefining model efficiency and quality, leading in AI benchmarks.

Hello Dolly: Democratizing the magic of ChatGPT with open models

Introducing 'Hello Dolly,' a project to democratize AI by integrating ChatGPT and open models, making advanced AI accessible to everyone.

Databricks acquires Neon

Databricks agrees to acquire Neon, a serverless Postgres database company, to bring speed and flexibility to production workloads.

Managing AI Coding Costs at Scale

AI coding tools deliver immense value: at Databricks, agentic coding has measurably improved every velocity metric we track and, in some teams, driven an order-of-magnitude gains in output. But nearly every company deploying AI tools at scale has hit the same wall: exponentially growing costs.

Spark 2.0 Technical Preview

Scala at scale at Databricks

Explore how Databricks leverages Scala for large-scale data processing, enhancing performance and developer productivity.

Spark as a Compiler: Joining a Billion Rows per Second on a Laptop

Discover how Apache Spark can join a billion rows per second on a laptop, showcasing its powerful data processing capabilities.

Deep Learning with Spark and TensorFlow

Databricks acquires serverless Postgres vendor bit.io

bit.io is joining Databricks, where co-founders Adam Fletcher and Jonathan Mortensen and the team will apply their expertise to deliver groundbreaking new developer experiences for data and AI.

Databricks response to Snowflake's accusation of lacking integrity

Analyze the claims made by Snowflake regarding price/performance compared to Databricks and the counterarguments presented.

Databricks is raising a Series K Investment at >$100B valuation

SAN FRANCISCO, CA — August 19, 2025 — Databricks, the Data and AI company, today announced it has signed a term sheet for its Series K round, which it expects to close soon with backing from existing investors. This funding values the company at >$100 billion.

Postgres data stored in Parquet on S3: LTAP architecture explained

Almost every traditional database keeps its write-ahead log and data files on one machine's disk, which is the root cause of data loss risk, expensive read replicas and high-availability clones, and analytics queries that drag down transactions.

Free Dolly: First truly open instruction-tuned LLM

Introducing Dolly, the first open-source, commercially viable instruction-tuned LLM, enabling accessible and cost-effective AI solutions.

Benchmarking coding agents on Databricks' multi-million line codebase

Databricks shares results from its internal coding benchmark, evaluating coding agents on a multi-million line codebase to optimize engineering cost and performance.

Announcing Apache Spark 1.4

Apache Spark 1.4 introduces SparkR, new DataFrame features, ML pipelines, and enhanced visualization and monitoring tools for better production debugging.

Spark Breaks Previous Large-Scale Sort Record

Read how Apache Spark sorted 100 TB of data (1 trillion records) 3X faster using 10X fewer machines in 2014.

Introducing DataFrames in Spark for Large Scale Data Science

Introducing Apache Spark's DataFrame API for easier, efficient big data processing and data science applications.

MLflow: An Open Source Machine Learning Platform

MLflow is an open-source platform to manage the ML lifecycle, offering tools for data preparation, model training, and deployment at scale.

Intelligent Kubernetes Load Balancing at Databricks

How Databricks uses client-side load balancing to improve gRPC traffic, reduce tail latency, and handle ingress with real-time service discovery.

Lakebase architecture delivers faster Postgres writes

How Lakebase's storage architecture pushes down full page write generation to cut WAL traffic by 94% and boost Postgres write throughput by up to 5x.

English as the new programming language for Apache spark

Discover the English SDK for Apache Spark, making data analytics more accessible with simple English instructions powered by Generative AI.

Announcing Spark 1.6

Announcing Spark 1.3

Apache Spark 1.3 introduces the DataFrame API, enhancing data processing with new features across streaming, ML, and SQL.