Claude 3 beats GPT-4 on Aider's code editing benchmark
Claude 3 Opus outperforms all of OpenAI’s models on Aider’s code editing benchmark, making it the best available model for pair programming with AI.
Aider lets you pair program with LLMs to start a new project or build on your existing codebase.
Visit aider.chat → On Hacker News Mar 2024 – Oct 2024
Claude 3 Opus outperforms all of OpenAI’s models on Aider’s code editing benchmark, making it the best available model for pair programming with AI.
OpenAI’s GPT-4 Turbo with Vision model scores worse on aider’s code editing benchmarks than all the previous GPT-4 models. In particular, it seems much more prone to “lazy coding” than the existing GPT-4 Turbo “preview” models.
An Architect model describes how to solve the coding problem, and an Editor model translates that into file edits. This Architect/Editor approach produces SOTA benchmark results.