评测集定义与方法论见 swe-bench-verified。本页只放分数:各模型修真实仓库 issue 的能力强度。
Verified 总榜(各家用自己的 harness,前 12,共 180 条,抓取于 2026-08-13)
| # | 模型 | Agent/Harness | % Resolved | 日期 |
|---|
| 1 | Claude 4.5 Opus | live-SWE-agent | 79.2% | 2025-12 |
| 2 | Claude 4.5 Opus | Sonar Foundation Agent | 79.2% | 2025-12 |
| 3 | Doubao-Seed-Code | TRAE | 78.8% | 2025-09 |
| 4 | Gemini 3 Pro Preview | live-SWE-agent | 77.4% | 2025-11 |
| 5 | Multiple (Anthropic) | Atlassian Rovo Dev | 76.8% | 2025-09 |
| 6 | Claude 4 Sonnet | EPAM AI/Run Developer Agent | 76.8% | 2025-08 |
| 7 | Claude 4.5 Opus | mini-SWE-agent | 76.8% | 2026-02 |
| 8 | Multiple (Anthropic) | ACoder | 76.4% | 2025-08 |
| 9 | Gemini 3 Flash / MiniMax M2.5 | mini-SWE-agent | 75.8% | 2026-02 |
| 10 | Multiple (OpenAI) | Warp | 75.6% | 2025-09 |
| 11 | Claude 4.6 Opus | mini-SWE-agent | 75.6% | 2026-02 |
| 12 | Claude 4.5 Sonnet | Sonar Foundation Agent | 74.8% | 2025-11 |
Bash-Only 视图(官方默认视图:统一 mini-SWE-agent,前 12,共 47 条)
| # | 模型 | % Resolved | $/实例 | 日期 |
|---|
| 1 | Claude 4.5 Opus | 76.8% | $0.754 | 2026-02 |
| 2 | Gemini 3 Flash | 75.8% | $0.356 | 2026-02 |
| 3 | MiniMax M2.5(开源权重) | 75.8% | $0.073 | 2026-02 |
| 4 | Claude 4.6 Opus | 75.6% | $0.552 | 2026-02 |
| 5 | Claude 4.5 Opus | 74.4% | $0.721 | 2025-11 |
| 6 | Gemini 3 Pro Preview | 74.2% | $0.460 | 2025-11 |
| 7 | GPT 5.2 Codex | 72.8% | $0.449 | 2026-02 |
| 8 | GLM 5(开源权重) | 72.8% | $0.534 | 2026-02 |
| 9 | GPT 5.2 | 72.8% | $0.474 | 2026-02 |
| 10 | Claude 4.5 Sonnet | 71.4% | $0.658 | 2026-02 |
| 11 | Kimi K2.5(开源权重) | 70.8% | $0.147 | 2026-02 |
| 12 | DeepSeek V3.2(开源权重) | 70.0% | $0.448 | 2026-02 |
分数观察
- harness 差异巨大:同一模型(Claude 4.5 Opus)总榜 79.2%(live-SWE-agent)vs bash-only 76.8%——总榜混入了 harness 工程能力,这是官方把 Bash-Only 设为默认视图的原因
- 成本差 10x:同为 75.8%,MiniMax M2.5 每实例 0.073,Claude4.5Opus0.754;开源权重(MiniMax/GLM/Kimi/DeepSeek)已占 bash-only 榜 1/3 且逼近闭源头部
- 进展速度:2024-08 发布时 GPT-4o + RAG scaffolding 约 33%,不到两年到 79%,但距人类专家基线 ~92% 仍有差距
- 污染风险:数据集公开且被当标准用了两年,榜单分数与真实泛化能力的差距是社区持续质疑点