SWE-bench asks the harder question: can models fix real GitHub issues?

Princeton's benchmark uses 2,294 real issues from real repos; the best model of the day (Claude 2) resolves 1.96%. That number becomes the era's most-watched scoreboard — from under 2% to over 80% in roughly two years.