Are LLM merge rates not getting better?
I was reading the metr article on how llm code passes test much more often than it is of mergeable quality . They look at the performance of llm s doing programming when the success criterion is “passes all tests” and compare it to when the success criterion is “would get approved by the maintainer”. Unsurprisingly…