Mythos’ system card contains the following graph to support its argument that Mythos performs better on SWE-bench: Anthropic and others are worried LLMs are memorizing SWE-bench, so they asked an LLM to estimate the probability that a solution is memorized. Next, they calculated the pass rate if they only included…
Visit philosophicalhacker.com → RSS feedOn Hacker News Aug 2019 – May 2020
Tools & Productivity 1Software Engineering 1Life, Learning & Writing 1Career & Management 1
In the original Unix tradition, command-line options are single letters preceded by a single hyphen…The original Unix style evolved on slow ASR-33 teletypes that made terseness a virtue; thus the single-letter options.