METR Blog

METR is a nonprofit that studies the most advanced AI agents to understand whether their developers are on track to lose control of them.

Popular posts: 6
Total points: 1,511
HN comments: 998
Rank: #848 company

Visit metr.org → RSS feedOn Hacker News Jul 2025 – Sep 2026

AI & Machine Learning 5 Society, Policy & Culture 1 Security & Privacy 1 Software Engineering 1 Open Source 1 Career & Management 1

Most points Most discussed Most cited Newest

Many SWE-bench-Passing PRs would not be merged

2026278 points153 comments Software Engineering details

We find that roughly half of test-passing SWE-bench Verified PRs written by recent AI agents would not be merged into main by repo maintainers. A naive interpretation of benchmark scores may lead one to overestimate how useful agents are without more elicitation or human feedback.