SWE-Bench Verified Leaderboard

131 submissions ยท 500 issues

SpecReportsExplorer โ†’
Showing 131 of 131 submissions
#
Name
Resolvedโ†“
%
Model
Org
Open
Date
Links
1TRAE + Doubao-Seed-Code39478.8%DoubaoByteDance
S
9/28/2025
๐ŸŒ๐Ÿ“„
2live-SWE-agent + Gemini 3 Pro Preview (2025-11-18)38777.4%GeminiUIUC
S
11/20/2025
๐ŸŒ๐Ÿ“„
3Atlassian Rovo Dev (2025-09-02)38476.8%ClaudeAtlassian
9/2/2025
๐ŸŒ๐Ÿ“„
4EPAM AI/Run Developer Agent v20250719 + Claude 4 Sonnet38476.8%ClaudeEPAM Systems, Inc.
8/4/2025
๐ŸŒ๐Ÿ“„
5ACoder38276.4%ClaudeACoder
8/19/2025
๐ŸŒ๐Ÿ“„
6Warp37875.6%GPTWarp
9/1/2025
๐ŸŒ๐Ÿ“„
7TRAE37675.2%ClaudeTRAE
S
6/12/2025
๐ŸŒ๐Ÿ“„
8Sonar Foundation Agent + Claude 4.5 Sonnet37474.8%ClaudeSonar
11/3/2025
๐ŸŒ๐Ÿ“„
9Harness AI37474.8%ClaudeHarness
7/31/2025
๐ŸŒ๐Ÿ“„
10JoyCode37374.6%ClaudeJoyCode
S
9/15/2025
๐ŸŒ๐Ÿ“„
11Lingxi-v1.5_claude-4-sonnet-2025051437374.6%ClaudeLingxi
S
7/20/2025
๐ŸŒ๐Ÿ“„
12Prometheus-v1.2.1 + GPT-537274.4%GPTEuniAI, Delysium
S
10/15/2025
๐ŸŒ๐Ÿ“„
13Refact.ai Agent37274.4%ClaudeRefact.ai
S
6/3/2025
๐ŸŒ๐Ÿ“„
14Salesforce AI Research SAGE (OpenHands)36973.8%ClaudeSalesforce AI Research
11/3/2025
๐ŸŒ๐Ÿ“„
15Tools + Claude 4 Opus (2025-05-22)36673.2%Claudeโ€”
5/22/2025
๐ŸŒ๐Ÿ“„
16Salesforce AI Research SAGE (bash-only)36573.0%ClaudeSalesforce AI Research
10/21/2025
๐ŸŒ๐Ÿ“„
17Tools + Claude 4 Sonnet (2025-05-22)36272.4%Claudeโ€”
5/22/2025
๐ŸŒ๐Ÿ“„
18OpenHands + GPT-535971.8%GPTOpenHands
Sโœ“
8/7/2025
๐ŸŒ๐Ÿ“„
19Lingxi v1.5 x Kimi K235671.2%KimiHuawei
WS
10/14/2025
๐ŸŒ๐Ÿ“„
20Prometheus-v1.2 + GPT-535671.2%GPTEuniAI, Delysium
S
9/29/2025
๐ŸŒ๐Ÿ“„
21Qodo Command35671.2%ClaudeQodo
7/15/2025
๐ŸŒ๐Ÿ“„
22Bloop35671.2%ClaudeBloop AI
7/10/2025
๐ŸŒ๐Ÿ“„
23Warp35571.0%ClaudeWarp
6/23/2025
๐ŸŒ๐Ÿ“„
24Moatless Tools + Claude 4 Sonnet35470.8%ClaudeMoatless AI
Sโœ“
6/11/2025
๐ŸŒ๐Ÿ“„
25TRAE35370.6%Otherโ€”
5/19/2025
๐ŸŒ๐Ÿ“„
26Augment Agent v135270.4%Otherโ€”
S
6/10/2025
๐ŸŒ๐Ÿ“„
27OpenHands + Claude 4 Sonnet35270.4%ClaudeOpenHands
Sโœ“
5/24/2025
๐ŸŒ๐Ÿ“„
28Refact.ai Agent35270.4%ClaudeRefact.ai
S
5/15/2025
๐ŸŒ๐Ÿ“„
29devlo35170.2%Claudedevlo
5/19/2025
๐ŸŒ๐Ÿ“„
30Zencoder (2025-04-30)35070.0%Claudeโ€”
4/30/2025
๐ŸŒ๐Ÿ“„
31OpenHands + Qwen3-Coder-480B-A35B-Instruct34869.6%QwenQwen, OpenHands
WS
8/5/2025
๐ŸŒ๐Ÿ“„
32GLM-4.634168.2%GLMZ.ai
WS
9/30/2025
๐ŸŒ๐Ÿ“„
33Nemotron-CORTEXA34168.2%OtherNVIDIA
S
5/16/2025
๐ŸŒ๐Ÿ“„
34SWE-agent + Claude 4 Sonnet33366.6%ClaudeSWE-agent
Sโœ“
5/22/2025
๐ŸŒ๐Ÿ“„
35Aime-coder v1 + Anthopic Claude 3.7 Sonnet33266.4%ClaudeByteDance DevInfra
5/14/2025
๐ŸŒ๐Ÿ“„
36OpenHands32965.8%OtherOpenHands
Sโœ“
4/15/2025
๐ŸŒ๐Ÿ“„
37OpenHands + Kimi K232765.4%KimiOpenHands
WSโœ“
7/16/2025
๐ŸŒ๐Ÿ“„
38Amazon Q Developer Agent (v20250405-dev)32765.4%OtherAWS
4/5/2025
๐ŸŒ๐Ÿ“„
39Augment Agent v032765.4%Otherโ€”
S
3/16/2025
๐ŸŒ๐Ÿ“„
40PatchPilot-v1.132364.6%GPTโ€”
S
5/3/2025
๐Ÿ“„
41W&B Programmer O1 crosscheck532364.6%GPTโ€”
1/17/2025
๐ŸŒ๐Ÿ“„
42GLM-4.532164.2%GLMZ.ai
WS
7/28/2025
๐ŸŒ๐Ÿ“„
43AgentScope31763.4%ClaudeAlibaba
2/6/2025
๐ŸŒ๐Ÿ“„
44Tools + Claude 3.7 Sonnet (2025-02-24)31663.2%ClaudeAnthropic
2/24/2025
๐ŸŒ๐Ÿ“„
45EPAM AI/Run Developer Agent v20250219 + Anthopic Claude 3.5 Sonnet31462.8%Claudeโ€”
2/28/2025
๐ŸŒ๐Ÿ“„
46Blackbox AI Agent31462.8%Otherโ€”
1/10/2025
๐ŸŒ๐Ÿ“„
47SWE-agent + Claude 3.7 Sonnet w/ Review Heavy31262.4%ClaudeSWE-agent
Sโœ“
2/25/2025
๐ŸŒ๐Ÿ“„
48CodeStory Midwit Agent + swe-search31162.2%Claudeโ€”
12/21/2024
๐ŸŒ๐Ÿ“„
49OpenHands + 4x Scaled (2024-02-03)30460.8%OtherOpenHands
Sโœ“
2/3/2025
๐ŸŒ๐Ÿ“„
50EntroPO + R2E + Qwen3-Coder-30B-A3B-Instruct30260.4%Qwen42-b3yond-6ug
WS
9/1/2025
๐ŸŒ๐Ÿ“„
51Learn-by-interact30160.2%ClaudeGoogle
1/10/2025
๐ŸŒ๐Ÿ“„
52DeepSWE-Preview + TTS(Bo16)29458.8%OtherAgentica
WS
6/29/2025
๐ŸŒ๐Ÿ“„
53Nemotron-CORTEXA29158.2%OtherNVIDIA
โœ“
4/10/2025
๐ŸŒ๐Ÿ“„
54devlo29158.2%Otherโ€”
12/13/2024
๐ŸŒ๐Ÿ“„
55Emergent E1 (v2024-12-23)28657.2%Claudeโ€”
12/23/2024
๐ŸŒ๐Ÿ“„
56Artemis Agent v2 (2025-09-24)28557.0%ClaudeTurintech
9/24/2025
๐ŸŒ๐Ÿ“„
57Gru(2024-12-08)28557.0%Otherโ€”
12/8/2024
๐ŸŒ๐Ÿ“„
58SWE-Rizzo28356.6%Claudeโ€”
S
4/5/2025
๐ŸŒ๐Ÿ“„
59EPAM AI/Run Developer Agent v20241212 + Anthopic Claude 3.5 Sonnet27755.4%Claudeโ€”
12/12/2024
๐ŸŒ๐Ÿ“„
60Amazon Q Developer Agent (v20241202-dev)27555.0%OtherAWS
12/2/2024
๐ŸŒ๐Ÿ“„
61devlo27154.2%Claudeโ€”
11/8/2024
๐ŸŒ๐Ÿ“„
62FrogBoss-32B-251026853.6%OtherMicrosoft
S
11/10/2025
๐ŸŒ๐Ÿ“„
63CodeSweep - SWE-agent - Kimi K2 Instruct26753.4%KimiCodeSweep Inc.
WS
8/4/2025
๐ŸŒ๐Ÿ“„
64Bracket.sh26653.2%Otherโ€”
1/20/2025
๐ŸŒ๐Ÿ“„
65OpenHands + CodeAct v2.1 (claude-3-5-sonnet-20241022)26553.0%ClaudeOpenHands
Sโœ“
10/29/2024
๐ŸŒ๐Ÿ“„
66EntroPO + R2E + Qwen3-Coder-30B-A3B-Instruct26152.2%Qwen42-b3yond-6ug
WS
9/1/2025
๐ŸŒ๐Ÿ“„
67Google Jules + Gemini 2.0 Flash (v20241212-experimental)26152.2%GeminiGoogle
12/12/2024
๐ŸŒ๐Ÿ“„
68Engine Labs (2024-11-25)25951.8%ClaudeEngine Labs
11/25/2024
๐ŸŒ๐Ÿ“„
69OpenHands + Qwen3-Coder-30B-A3B-Instruct25851.6%QwenQwen, OpenHands
WS
8/5/2025
๐ŸŒ๐Ÿ“„
70AutoCodeRover-v2.1 (Claude-3.5-Sonnet-20241022)25851.6%Claudeโ€”
1/22/2025
๐ŸŒ๐Ÿ“„
71Agentless-1.5 + Claude-3.5 Sonnet (20241022)25450.8%ClaudeAgentless
S
12/2/2024
๐ŸŒ๐Ÿ“„
72Bytedance MarsCode Agent25050.0%OtherBytedance
11/25/2024
๐ŸŒ๐Ÿ“„
73Solver (2024-10-28)25050.0%Otherโ€”
10/28/2024
๐ŸŒ๐Ÿ“„
74nFactorial (2024-11-05)24649.2%Claudeโ€”
11/5/2024
๐ŸŒ๐Ÿ“„
75Tools + Claude 3.5 Sonnet (2024-10-22)24549.0%ClaudeAnthropic
10/22/2024
๐ŸŒ๐Ÿ“„
76Composio SWE-Kit (2024-10-25)24348.6%Claudeโ€”
Sโœ“
10/25/2024
๐ŸŒ๐Ÿ“„
77AppMap Navie v223647.2%Claudeโ€”
Sโœ“
11/6/2024
๐ŸŒ๐Ÿ“„
78Skywork-SWE-32B + TTS(Bo8)23547.0%QwenSkywork AI
WSโœ“
6/16/2025
๐ŸŒ๐Ÿ“„
79OpenHands + DevStral Small 250523446.8%MistralOpenHands, Mistral
WSโœ“
5/20/2025
๐ŸŒ๐Ÿ“„
80Emergent E1 (v2024-10-12)23346.6%Claudeโ€”
10/23/2024
๐ŸŒ๐Ÿ“„
81AutoCodeRover-v2.0 (Claude-3.5-Sonnet-20241022)23146.2%Claudeโ€”
S
11/8/2024
๐ŸŒ๐Ÿ“„
82PatchPilot + Co-PatcheR23046.0%Otherโ€”
WS
5/28/2025
๐ŸŒ๐Ÿ“„
83Solver (2024-09-12)22745.4%Otherโ€”
9/24/2024
๐ŸŒ๐Ÿ“„
84Gru(2024-08-24)22645.2%Otherโ€”
8/24/2024
๐ŸŒ๐Ÿ“„
85FrogMini-14B-251022545.0%OtherMicrosoft
S
11/10/2025
๐ŸŒ๐Ÿ“„
86CodeShellAgent + Gemini 2.0 Flash (Experimental)22144.2%Geminiโ€”
1/18/2025
๐ŸŒ๐Ÿ“„
87Solver (2024-09-12)21843.6%Otherโ€”
9/20/2024
๐ŸŒ๐Ÿ“„
88Amazon Nova Premier 1.0 (2025-04-30)21242.4%NovaAmazon Nova
5/27/2025
๐ŸŒ๐Ÿ“„
89Agentless Lite + O3 Mini (20250214)21242.4%GPTAgentless
Sโœ“
2/14/2025
๐ŸŒ๐Ÿ“„
90DeepSWE-Preview21142.2%OtherAgentica
WS
6/29/2025
๐ŸŒ๐Ÿ“„
91SWE-Exp21042.0%DeepSeekSWE-Exp
WS
8/6/2025
๐ŸŒ๐Ÿ“„
92ugaiforge20841.6%Otherโ€”
1/12/2025
๐Ÿ“„
93nFactorial (2024-10-30)20841.6%Claudeโ€”
10/30/2024
๐ŸŒ๐Ÿ“„
94SWE-RL (Llama3-SWE-RL-70B + Agentless Mini) (20250226)20641.2%LlamaAgentless
S
2/26/2025
๐ŸŒ๐Ÿ“„
95Nebius AI Qwen 2.5 72B Generator + LLama 3.1 70B Critic20340.6%Llamaโ€”
11/13/2024
๐ŸŒ๐Ÿ“„
96Tools + Claude 3.5 Haiku20340.6%ClaudeAnthropic
10/22/2024
๐ŸŒ๐Ÿ“„
97Composio SWEkit + Claude 3.5 Sonnet (2024-10-16)20340.6%Claudeโ€”
S
10/16/2024
๐ŸŒ๐Ÿ“„
98Honeycomb20340.6%Otherโ€”
8/20/2024
๐ŸŒ๐Ÿ“„
99SWE-agent + SWE-agent-LM-32B20140.2%QwenSWE-agent
WSโœ“
5/11/2025
๐ŸŒ๐Ÿ“„
100EPAM AI/Run Developer Agent v20241029 + Anthopic Claude 3.5 Sonnet19839.6%Claudeโ€”
10/29/2024
๐ŸŒ๐Ÿ“„
101Agentless-1.5 + GPT 4o (2024-05-13)19438.8%GPTAgentless
S
10/28/2024
๐ŸŒ๐Ÿ“„
102Amazon Q Developer Agent (v20240719-dev)19438.8%OtherAWS
7/21/2024
๐ŸŒ๐Ÿ“„
103AutoCodeRover (v20240620) + GPT 4o (2024-05-13)19238.4%GPTโ€”
6/28/2024
๐ŸŒ๐Ÿ“„
104SWE-agent + DevStral Small 250719038.0%MistralSWE-agent
WSโœ“
7/25/2025
๐ŸŒ๐Ÿ“„
105Skywork-SWE-32B19038.0%QwenSkywork AI
WSโœ“
6/16/2025
๐ŸŒ๐Ÿ“„
106Factory Code Droid18537.0%Otherโ€”
6/17/2024
๐ŸŒ๐Ÿ“„
107SWE-agent + Claude 3.5 Sonnet16833.6%ClaudeSWE-agent
Sโœ“
6/20/2024
๐ŸŒ๐Ÿ“„
108SWE-Fixer (Qwen2.5-7b retriever + Qwen2.5-72b editor)16432.8%Qwenโ€”
WSโœ“
3/6/2025
๐ŸŒ๐Ÿ“„
109MASAI + GPT 4o (2024-06-12)16332.6%GPTโ€”
6/12/2024
๐ŸŒ๐Ÿ“„
110Artemis Agent v1 (2024-11-20)16032.0%Otherโ€”
11/20/2024
๐ŸŒ๐Ÿ“„
111nFactorial (2024-10-07)15831.6%Otherโ€”
10/7/2024
๐ŸŒ๐Ÿ“„
112SWE-Fixer (Qwen2.5-7b retriever + Qwen2.5-72b editor) 2024112815130.2%Qwenโ€”
S
11/28/2024
๐ŸŒ๐Ÿ“„
113Lingma Agent + Lingma SWE-GPT 72b (v0925)14428.8%GPTAlibaba
S
10/2/2024
๐ŸŒ๐Ÿ“„
114EPAM AI/Run Developer Agent + GPT4o13527.0%GPTโ€”
10/16/2024
๐ŸŒ๐Ÿ“„
115AppMap Navie + GPT 4o (2024-05-13)13126.2%GPTโ€”
Sโœ“
6/15/2024
๐ŸŒ๐Ÿ“„
116nFactorial (2024-10-01)12925.8%Otherโ€”
10/1/2024
๐ŸŒ๐Ÿ“„
117Amazon Q Developer Agent (v20240430-dev)12825.6%OtherAWS
5/9/2024
๐ŸŒ๐Ÿ“„
118Lingma Agent + Lingma SWE-GPT 72b (v0918)12525.0%GPTAlibaba
S
9/18/2024
๐ŸŒ๐Ÿ“„
119EPAM AI/Run Developer Agent + GPT4o12024.0%GPTโ€”
8/20/2024
๐ŸŒ๐Ÿ“„
120MCTS-Refine-7B11623.2%Otherโ€”
WS
6/27/2025
๐ŸŒ๐Ÿ“„
121SWE-agent + GPT 4o (2024-05-13)11623.2%GPTSWE-agent
Sโœ“
7/28/2024
๐ŸŒ๐Ÿ“„
122SWE-agent + GPT 4 (1106)11222.4%GPTSWE-agent
Sโœ“
4/2/2024
๐ŸŒ๐Ÿ“„
123Lingma Agent + Lingma SWE-GPT 7b (v0925)9118.2%GPTAlibaba
S
10/2/2024
๐ŸŒ๐Ÿ“„
124SWE-agent + Claude 3 Opus7915.8%ClaudeSWE-agent
Sโœ“
4/2/2024
๐ŸŒ๐Ÿ“„
125Lingma Agent + Lingma SWE-GPT 7b (v0918)5110.2%GPTAlibaba
S
9/18/2024
๐ŸŒ๐Ÿ“„
126RAG + Claude 3 Opus357.0%Claudeโ€”
Sโœ“
4/2/2024
๐ŸŒ๐Ÿ“„
127RAG + Claude 2224.4%Claudeโ€”
Sโœ“
10/10/2023
๐Ÿ“„
128RAG + GPT 4 (1106)142.8%GPTโ€”
Sโœ“
4/2/2024
๐Ÿ“„
129RAG + SWE-Llama 7B71.4%LlamaSWE-agent
Sโœ“
10/10/2023
๐Ÿ“„
130RAG + SWE-Llama 13B61.2%LlamaSWE-agent
Sโœ“
10/10/2023
๐Ÿ“„
131RAG + ChatGPT 3.520.4%GPTSWE-agent
Sโœ“
10/10/2023
๐Ÿ“„