OpenAI Blog

Scaling Git at OpenAI

How OpenAI built Pando, a Git-compatible backend designed for the scale of development that Codex makes possible.

OpenAI Releases Largest GPT-2 Text Generation Model

As the final model release of GPT-2’s staged release, we’re releasing the largest version (1.5B parameters) of GPT-2 along with code and model weights to facilitate detection of outputs of GPT-2 models. While there have been larger language models released since August, we’ve continued with our original staged release…

CriticGPT: Finding GPT-4's mistakes with GPT-4

CriticGPT, a model based on GPT-4, writes critiques of ChatGPT responses to help human trainers spot mistakes during RLHF

Expanding on what we missed with sycophancy

A deeper dive on our findings, what went wrong, and future changes we’re making.

Addendum to GPT-5 system card: GPT-5-Codex

This addendum to the GPT-5 system card shares a new model: GPT-5-Codex, a version of GPT-5 further optimized for agentic coding in Codex. GPT-5-Codex adjusts its thinking effort more dynamically based on task complexity, responding quickly to simple conversational queries or small tasks, while independently working…

Standardizing OpenAI’s deep learning framework on PyTorch

We are standardizing OpenAI’s deep learning framework on PyTorch.

Scaling Kubernetes to 7,500 Nodes

We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.

Separating signal from noise in coding evaluations

A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.

OpenAI releases larger GPT-2 model

We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training.

New embedding models and API updates

Fine-tuning now available for GPT-4o

Fine-tune custom versions of GPT‑4o to increase performance and accuracy for your applications.

New Capabilities for GPT-3: Edit and Insert

We’ve released new versions of GPT-3 and Codex which can edit or insert content into existing text, rather than just completing existing text.

Training agents to invent a language

In this post we’ll outline new OpenAI research in which agents develop their own language.

Multimodal Neurons in Artificial Neural Networks

We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar…

Customizing GPT-3 for Your Application

Fine-tune with a single command.

Techniques for Training Large Neural Networks

Large neural networks are at the core of many recent advances in AI, but training them is a difficult engineering and research challenge which requires orchestrating a cluster of GPUs to perform a single synchronized calculation.

Clip: Connecting Text and Images

We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the “zero-shot” capabilities of GPT-2 and GPT-3.

New and Improved Embedding Model for OpenAI

We are excited to announce a new embedding model which is significantly more capable, cost effective, and simpler to use.

GPT-5.1 for Developers

GPT-5.1 is now available in the API, bringing faster adaptive reasoning, extended prompt caching, improved coding performance, and new apply_patch and shell tools.

Advanced coding in low-resource programming languages

Learn how OpenAI is improving ChatGPT across low-resource programming languages, with new evaluations for software engineering, SQLite, and COBOL migration.

PaperBench

We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research.

Aligning Language Models to Follow Instructions

We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models…