Recently added benchmarks

Dyno Therapeutics AAV capsid packaging prediction evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 0.8
2. #2 Claude Opus 4.8 0.8

AgentQuest: Evaluates autonomous agent performance on multi-step tasks requiring planning, state tracking, tool use, and recovery.

1. #1 Modified LangChain chat agent + GPT-4 on ALFWorld 93%
2. #2 LangChain chat agent + GPT-4 on ALFWorld 86%
3. #3 Modified LangChain chat agent + GPT-4 on Mastermind 60%

Audio hallucination benchmark for large audio-language models across semantic, acoustic, and confusion hallucination types.

1. #1 Gemini-2.5-Pro 60%
2. #2 GPT-Audio 28.75%
3. #3 Kimi-Audio 23.94%

MathArena ArxivMath final-answer research-math benchmark slice from the March and April 2026 releases, as reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 71.8%
2. #2 GPT-5.5 71.5%
3. #3 Gemini 3.1 Pro Preview 64.8%

Interspeech 2026 challenge benchmark for audio reasoning systems, with public Single Model and Agent track leaderboards scored by rubric and accuracy.

1. #1 TalTech (Agent Track) 69.83 rubrics / 76.9% accuracy
2. #2 AISpeech (Agent Track) 66.23 rubrics / 77.4% accuracy
3. #3 AI^2 (Agent Track) 66.09 rubrics / 75.1% accuracy

Verified BioPipelineBench slice for bioinformatics pipeline tasks, reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 88.1%
2. #2 Claude Opus 4.8 87.7%
3. #3 Claude Opus 4.7 83.6%

Dyno Therapeutics black-box RNA sequence modeling and design evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 11.2
2. #2 Claude Opus 4.8 10.1

Spatial-reasoning benchmark measuring how accurately models convert apartment photos into 2D floor plans.

1. #1 Human* 0.809
2. #2 GPT 5.5 0.706 +/- 0.008
3. #3 Gemini 3.5 Flash 0.694 +/- 0.006

Browser Agent Red teaming Toolkit benchmark for evaluating whether browser agents pursue harmful web tasks despite refusal training in their underlying chat models.

1. #1 OpenHands + Opus-3 40%
2. #2 OpenHands + o1-preview 63%
3. #3 OpenHands + Gemini-1.5 65%

CanItEdit: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

1. #1 NextCoder-32B 62.4%
2. #2 QwenCoder-2.5-32B 61%
3. #3 NextCoder-14B 60.2%

Clue-grounded long-video question-answering benchmark evaluating MCQ accuracy, clue-grounding credibility metrics, and open-ended answer accuracy.

1. #1 GPT-4o-08-06 39.2% open-ended acc. / 44.9% MCQ long acc.
2. #2 Claude3.5-Sonnet 35.6% open-ended acc. / 40.3% MCQ long acc.
3. #3 InternVL2.5 34.2% open-ended acc. / 44.2% MCQ long acc.

Subseasonal-to-seasonal climate prediction benchmark with deterministic and probabilistic model leaderboards across T-850, Z-500, and Q-700 variables.

1. #1 climatology vs. ERA5 (control) 3.3882 T-850 RMSE at day 44
2. #2 ecmwf vs. ERA5 (ensemble) 3.4618 T-850 RMSE at day 44
3. #3 ncep vs. ERA5 (ensemble) 3.7616 T-850 RMSE at day 44

Chart-understanding benchmark reported in Anthropic system cards, with no-tools and Python-tools variants.

1. #1 Claude Opus 4.8 89.7%
2. #2 Claude Opus 4.7 85.9%

Harder chart-understanding evaluation for professional and technical visual-question-answering tasks.

1. #1 Claude Opus 4.8 72.3%
2. #2 Claude Opus 4.7 69.8%

Codeforces: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

1. #1 DeepSeek-V4-Flash-Max 1
2. #1 DeepSeek-V4-Pro-Max 1
3. #3 DeepSeek-V3.2-Speciale 0.9

Long story generation benchmark measuring cross-scene consistency bugs using Consistency Error Density over 2,000 generated stories.

1. #1 GPT-5-Reasoning CED 0.113
2. #2 Gemini-2.5-Pro CED 0.302
3. #3 Gemini-2.5-Flash CED 0.305

A private benchmark evaluating understanding of long-context credit agreements

1. #1 Grok 4.3 68.532%
2. #2 GPT 5.5 68.415%
3. #3 Kimi K2.5 Thinking 68.259%

Qwen internal cowork productivity benchmark spanning long-horizon tasks in computer science, finance, law, medical, and other productivity domains.

1. #1 Claude Opus 4.6 Max 68.2%
2. #2 Qwen3.7 Max 67.2%
3. #3 DeepSeek V4 Pro Max 66.3%

CTFBench: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 SavantChat Dec 2025 1.000
2. #2 gpt_5.5 1.000
3. #3 gemini_3.1_pro 0.984

Cursor's coding-agent benchmark for ambiguous, multi-file tasks sourced from real Cursor sessions, with CursorBench 3.1 adding codebase understanding, bugfinding, planning, and code review problems.

1. #1 Opus 4.8 Max 64.8%
2. #2 GPT-5.5 Extra High 64.3%
3. #3 Composer 2.5 63.2%

CyberBench: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 GPT-4 69.6%
2. #2 GPT-3.5-Turbo 62.6%
3. #3 Mistral-7B-v0.1 58.1%

Data-agent benchmark spanning the full data intelligence lifecycle, with data-engineering architecture, implementation, evolution, and data-analysis tasks in English and Chinese.

1. #1 GPT-5 (DA-Agent, Data Analysis, EN) 50.84 Data Analysis
2. #1 GPT-5 (DA-Agent, Data Analysis, ZH) 49.49 Data Analysis
3. #1 GPT-5 (DE-Agent, DE Architecture, EN) 63.93 DE Architecture

Benchmark for evaluating how well language models write production-grade DAX for Power BI and Analysis Services business-intelligence scenarios.

1. #1 Gemini 3.1 Flash Lite Preview HIGH Google 97.4%
2. #2 GPT-5.3 Chat OpenAI 96.2%
3. #3 GLM 5 Z.AI 96.2%

SecureBio DNA synthesis screening evasion evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 0.8
2. #2 Claude Opus 4.8 0.7

Perplexity DRACO deep-research agent evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 80.4%

Interactive Factorio automation benchmark for LLM agents, tracking production score, milestones, automation milestones, and lab-task success rate.

1. #1 Claude 3.5-Sonnet 293206 production score
2. #2 Gemini-2-Flash 115782 production score
3. #3 GPT4o 87599 production score

Financial analytics benchmark over SEC filings covering detail/reasoning QA, enterprise comparison QA, and longitudinal tracking QA.

1. #1 GPT-5-websearch 43.37%
2. #2 GPT-4.1 33.24%
3. #4 Qwen3-235B 24.39%

Evaluating agents on core financial analyst tasks using the FAB v2 harness

1. #1 Claude Opus 4.8 53.9%
2. #1 Gemini 3.5 Flash 57.861%
3. #2 GPT-5.5 51.8%

Financial chain-of-thought benchmark reporting ChainEval and lexical/semantic similarity metrics for general and finance-specific models.

1. #1 Gemini 2.5 Pro 58.65 ChainEval
2. #2 Claude Sonnet 4.5 58.22 ChainEval
3. #3 Claude Sonnet 4 58.18 ChainEval

Financial data-search benchmark evaluating model+search-tool agents on time-sensitive, simple historical, and complex historical data retrieval for global and Greater China markets.

1. #1 DouBao (web) (Greater China Data Performance) 54.2% average accuracy
2. #1 Grok 4 (web) (Global Data Performance) 68.9% average accuracy
3. #2 GPT-5-Thinking (web) (Global Data Performance) 63.9% average accuracy

Anthropic internal Firefox 147 SpiderMonkey exploitation evaluation reported in the Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 70.8%
2. #2 Claude Opus 4.8 8.8%
3. #3 Claude Opus 4.7 1.2%

Interactive 3D vision-reasoning benchmark where models plan physical actions in puzzle and stacking environments.

1. #1 GPT-5.2 22.9%
2. #2 Gemini-3-Pro 19.3%
3. #3 Claude-Opus-4.5 15.6%

Software-engineering agent benchmark targeting frontier-level implementation, performance optimization, and research tasks.

1. #1 Claude Opus 4.8 2.7 avg rank
2. #1 GPT-5.5 (Codex) 2.53 avg rank / 83% dominance
3. #2 Claude Opus 4.7 4.2 avg rank

Leaderboard for large language models on geospatial code generation using AutoGEEval and AutoGEEval++ pass@k metrics on Google Earth Engine tasks.

1. #1 Gemini2.5-Flash-0520 71.87% pass@1
2. #2 DeepSeek-V3-1226 71.67% pass@1
3. #3 GPT-4.1 70.93% pass@1

Global MMLU multilingual knowledge-and-reasoning evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Gemini 3.1 Pro Preview 92.2%
2. #2 GPT-5.4 90.6%
3. #3 Claude Opus 4.8 90.4%

Graduate-level Google-Proof Q&A benchmark evaluating models on questions that require deep reasoning.

1. #1 Gemini 3.1 Pro Preview 95.454%
2. #2 GPT 5.5 93.182%
3. #3 Gemini 3.5 Flash 92.676%

Multitask multilingual hallucination detection benchmark spanning QA and dialogue summarization in English, Arabic, Hindi, and Turkish.

1. #1 GPT-4o 80.30%
2. #2 GPT-4.1 78.66%
3. #3 Claude-3.5 76.98%

Open-source long-horizon legal agent benchmark where agents work from client-matter instructions, closed-universe matter files, and expert rubrics to produce reviewable legal work product.

1. #1 Claude Opus 4.7 7.1%
2. #1 Claude Opus 4.8 9.6%
3. #2 Claude Sonnet 4.6 5.4%

HealthBench professional subset for medically challenging, expert-oriented healthcare question answering.

1. #1 Claude Opus 4.8 55.8%
2. #2 Claude Opus 4.7 51.9%
3. #3 Claude Sonnet 4.6 41.7%

Holistic Evaluation of Audio Representations benchmark comparing general-purpose audio embeddings across diverse audio and speech tasks.

1. #1 RedRice ced_base 14.0610
2. #2 GURA Fuse Cat H+w+C 13.9713
3. #3 RedRice ced_small 13.7305

HELM AIR-Bench: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 Claude 4.5 Haiku (20251001) 0.931507
2. #2 Claude 3.5 Sonnet (20241022) 0.908325
3. #3 Claude 4.5 Sonnet (20250929) 0.898402

HELM Instruct: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 GPT-3.5 Turbo (0613) 0.688889
2. #2 Anthropic Claude v1.3 0.611111
3. #3 GPT-4 (0314) 0.611111

HELM Lite: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 GPT-4o (2024-05-13) 0.959457
2. #2 Claude 3.5 Sonnet (20240620) 0.912171
3. #3 GPT-4 (0613) 0.908908

HELM MedQA: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 GPT-5 (2025-08-07) 0.968191
2. #2 GPT-5 mini (2025-08-07) 0.956262
3. #3 o4-mini (2025-04-16) 0.948310

HELM Safety: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 o3 (2025-04-16) 0.981606
2. #2 Claude 4 Sonnet (20250514, extended thinking) 0.980722
3. #3 Claude 3.5 Sonnet (20240620) 0.976697

Humanity's Last Exam results reported by Qwen with tool use enabled.

1. #1 Kimi K2.6 Thinking 54%
2. #2 Qwen3.7 Max 53.5%
3. #3 Claude Opus 4.6 Max 53%

INCLUDE multilingual evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.6 Max 87.4%
2. #1 Claude Opus 4.8 87.6%
3. #2 Qwen3.7 Max 86.2%

Head-to-head LLM game-arena ladder for chess, using InfiniteBM's per-game Bradley-Terry Elo ratings across model and human matches.

1. #1 Claude Opus 4.7 1997.52 Elo / 16 games
2. #2 GPT-OSS 120B 1660.89 Elo / 6 games
3. #3 Claude Sonnet 4.6 1190.33 Elo / 11 games

Head-to-head LLM game-arena ladder for Coup, using InfiniteBM's per-game Bradley-Terry Elo ratings across bluffing and imperfect-information matches.

1. #1 GPT-5.4 1690.86 Elo / 21 games
2. #2 Claude Sonnet 4.6 1549.3 Elo / 34 games
3. #3 Claude Haiku 4.5 1488.43 Elo / 29 games

Head-to-head LLM game-arena ladder for heads-up no-limit hold'em, using InfiniteBM's per-game Bradley-Terry Elo ratings across poker decision matches.

1. #1 Gemini 2.5 Flash Lite (high) 1684.29 Elo / 13 games
2. #2 GPT-5.5 (high) 1620.63 Elo / 19 games
3. #3 Claude Sonnet 4.6 (high) 1485.1 Elo / 20 games

Head-to-head LLM game-arena ladder for Liar's Dice, using InfiniteBM's per-game Bradley-Terry Elo ratings across hidden-information bidding and challenge timing matches.

1. #1 Gemini 3 Flash (high) 1566.83 Elo / 27 games
2. #2 Gemini 3.1 Pro (high) 1566.69 Elo / 27 games
3. #4 Gemini 2.5 Flash Lite (high) 1380.31 Elo / 26 games

Head-to-head LLM game-arena ladder for Settlers of Catan, using InfiniteBM's per-game Bradley-Terry Elo ratings across negotiation and planning matches.

1. #1 GPT-OSS 120B 1958.76 Elo / 5 games
2. #2 Claude Sonnet 4.6 1805.89 Elo / 24 games
3. #3 Claude Opus 4.7 1740.25 Elo / 24 games

Head-to-head LLM game-arena ladder for Werewolf, using InfiniteBM's per-game Bradley-Terry Elo ratings across social-deduction matches.

1. #1 GPT-5.4 (high) 2241.79 Elo / 7 games
2. #2 GPT-5.4 Mini 1385.83 Elo / 10 games
3. #3 Claude Opus 4.7 1255.77 Elo / 22 games

Artificial Analysis implementation of IBM's ITBench SRE benchmark, testing AI agents on Kubernetes incident root-cause analysis from offline incident snapshots.

1. #1 Claude Opus 4.7 (Adaptive Reasoning, Max Effort) 46.7%
2. #2 GPT-5.5 (xhigh) 45.8%
3. #3 Qwen3.7 Max 42.5%

NIST JARVIS AI model contribution index across materials ML tasks including force fields, property prediction, spectra, atom generation, and materials text tasks.

1. #1 mlearn_analysis_Ge_orb-v2 on mlearnall_Ge_stresses 0.668917 MULTIMAE
2. #2 mlearn_analysis_Ge_eqV2_31M_omat_mp_salex on mlearnall_Ge_forces 0.0282175 MULTIMAE
3. #3 mlearn_analysis_Si_eqV2_153M_omat on mlearnall_Si_forces 0.0123348 MULTIMAE

Structured-output benchmark measuring schema compliance and JSON validity for language models across easy, medium, and hard schema-constrained generation datasets.

1. #1 openAI/gpt-4o 96.9% schema compliance
2. #2 openAI/gpt-4o-mini 95.8% schema compliance
3. #3 Qwen/Qwen2.5-72B-Instruct 95.5% schema compliance

Kernel optimization benchmark reporting median speedup over a PyTorch eager reference and the fraction of problems faster than torch.compile.

1. #1 Claude Opus 4.6 Max 2.63/98%
2. #2 GLM-5.1 Thinking 2.00/78%
3. #3 Qwen3.7 Max 1.98/96%

LABBench2 clinical-trials subset reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 86.3%
2. #2 Claude Opus 4.8 85.3%
3. #3 Claude Opus 4.7 70.8%

LABBench2 patent-question subset reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 68.8%
2. #2 Claude Mythos Preview 64.3%
3. #3 Claude Opus 4.7 48.3%

LABBench2 table-reading subset reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 77.2%
2. #2 Claude Opus 4.7 66.4%

LABBench2 supplementary-materials subset reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 58.9%
2. #2 Claude Opus 4.7 47.8%

Evaluating language models on a wide range of open source legal reasoning tasks.

1. #1 Gemini 3.1 Pro Preview 87.398%
2. #2 Gemini 3 Pro Preview 87.025%
3. #3 Gemini 3 Flash Preview 86.858%

Legal reasoning benchmark derived from 340 law exams across 116 law-school courses, covering long-form open questions and multiple-choice questions in English and German.

1. #1 GPT-5 70.20% open / 62.65% MCQ
2. #2 Gemini-2.5-Pro 67.40% open / 55.72% MCQ
3. #3 Claude-3.7-Sonnet 62.86% open / 57.23% MCQ

Our Implementation of the LiveCodeBench benchmark

1. #1 Gemini 3.1 Pro Preview 88.485%
2. #2 GPT 5.2 Codex 87.993%
3. #3 Claude Opus 4.8 87.819%

Live benchmark for research-level theorem comprehension, with monthly multiple-choice questions derived from newly published arXiv mathematics papers.

1. #1 Gemini 3.1 Pro Preview (high) 43.5%
2. #2 GPT-5.4 (high) 41.8%
3. #4 Qwen3.5-397B-A17B (enabled) 35.6%

Automated biological-risk evaluation for long-form virology task completion, reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 0.8
2. #2 Claude Opus 4.8 0.8

Automated biological-risk evaluation for long-form virology task completion, reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 0.9
2. #2 Claude Opus 4.8 0.9

Finance benchmark measuring lookahead bias in LLM trading workflows by comparing in-sample and out-of-sample alpha decay.

1. #1 Pitinf-Large +1.30pp
2. #2 Pitinf-Medium +0.85pp
3. #3 Pitinf-Small +0.31pp

Magic: The Gathering benchmark leaderboard for LLMs, reporting Season 1 model ratings, blunder index, games played, win rate, and average API cost across the combined format leaderboard.

1. #1 Claude Opus 4.6 (medium) 1747 rating / 16 games
2. #2 GPT-5.2 (medium) 1737 rating / 11 games
3. #3 Gemini 3 Pro (medium) (retired) 1722 rating / 11 games

Materials-discovery benchmark evaluating machine-learning energy models for crystal stability prediction, geometry optimization, and related high-throughput discovery tasks.

1. #1 EquiformerV3+DeNS-OAM 0.931 unique-prototypes F1
2. #2 eSEN-30M-OAM 0.925 unique-prototypes F1
3. #3 PET-OAM-XL 0.924 unique-prototypes F1

Can models support the medical billing process?

1. #1 Gemini 3.1 Pro Preview 59.062%
2. #2 Gemini 3 Flash Preview 55.92%
3. #3 Gemini 3.5 Flash 55.825%

Can models support doctors with their administrative work?

1. #1 GPT 5.1 2025-11-13 88.09%
2. #2 GPT 5.5 86.868%
3. #3 Claude Opus 4.6 86.738%

Multilingual knowledge-and-reasoning evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 90.3%

Academic multiple-choice benchmark covering 14 subjects including STEM, humanities, and social sciences.

1. #1 Gemini 3.1 Pro Preview 90.987%
2. #2 Gemini 3 Pro Preview 90.102%
3. #3 Claude Opus 4.7 89.871%

Multimodal Multi-task Benchmark

1. #1 Gemini 3.5 Flash 88.266%
2. #2 GPT 5.5 88.266%
3. #3 Gemini 3.1 Pro Preview 88.208%

Multi-image spatial intelligence VQA benchmark with 1,000 human-designed questions across real-world 3D scene understanding, robotics, driving, and motion reasoning.

1. #1 Human Level 97.2%
2. #2 Gemini-3-pro 49.2%
3. #3 SenseNova-SI-1.2-InternVL3-8B 42.6%

Evaluating reading and understanding tax certificates as images

1. #1 Claude Opus 4.7 70.27%
2. #2 Claude Opus 4.8 69.912%
3. #3 Gemini 3.1 Pro Preview 69.396%

MRCR-v2 long-context retrieval subset using a 128K context window with 8 needles, as reported in Qwen's Qwen3.7-Max launch post.

1. #1 Qwen3.7 Max 90.4%
2. #2 Qwen3.6 Plus 85.9%
3. #3 Claude Opus 4.6 Max 84%

Professional office question-answering evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 77.6%
2. #2 Claude Opus 4.7 76.3%

Harder professional office question-answering evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Opus 4.8 66.2%
2. #2 Claude Opus 4.7 65%

OC20 leaderboard for catalyst and adsorbate relaxation/energy prediction, spanning S2EF, IS2RS, and/or IS2RE task splits.

1. #1 EquiformerV2 - 31M - LBFGS Fix (IS2RE, OOD Both) 14.59 IS2RE OOD Both primary score
2. #1 EquiformerV2 (λᴇ=2) -- All+MD (IS2RS, Average) 55.45 IS2RS Average primary score
3. #1 EquiformerV2 (λᴇ=2) -- All+MD (IS2RS, OOD Ads) 55.28 IS2RS OOD Ads primary score

OC22 leaderboard for catalyst and adsorbate relaxation/energy prediction, spanning S2EF, IS2RS, and/or IS2RE task splits.

1. #1 EquiformerV2 + DeNS (127M, λᴇ=4, λF=100, OC22-only, fmax=0.1) (IS2RE, Average) 3.93 IS2RE Average primary score
2. #1 EquiformerV2 + DeNS (127M, λᴇ=4, λF=100, OC22-only, fmax=0.1) (IS2RE, ID) 4.57 IS2RE ID primary score
3. #1 EquiformerV2 + DeNS (127M, λᴇ=4, λF=100, OC22-only, fmax=0.1) (IS2RE, OOD) 3.17 IS2RE OOD primary score

Anthropic internal organic-chemistry evaluation reported in the Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 86.5%
2. #2 Claude Opus 4.8 86.2%
3. #3 Claude Opus 4.7 77.2%

Synthetic insider-threat detection benchmark built from OrgForge organizational simulation telemetry with triage, verdict, and false-positive scoring.

1. #1 Claude Opus 4.6 1.000
2. #2 Devstral 2 123B 1.000
3. #3 Claude Haiku 4.5 0.800

Anthropic system-card ProgramBench evaluation on the 166-task golden subset, reported as hidden behavioral test pass rate across 1-5 episodes.

1. #1 Claude Opus 4.8 88%
2. #2 Claude Opus 4.7 84%

Automated theorem proving benchmark

1. #1 Aristotle 71%
2. #2 Claude Opus 4.8 69%
3. #3 GPT 5.4 2026-03-05 56%

Protein fitness prediction benchmark comparing zero-shot models on DMS substitution assays with aggregate Spearman correlations and drill-downs by function, MSA depth, taxa, and mutation depth.

1. #1 AIDO Protein-RAG (16B) 0.518 average Spearman
2. #2 VenusREM 0.518 average Spearman
3. #3 ProSST (K=2048) 0.507 average Spearman

Hard ProteinGym subset reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 43.1%
2. #2 Claude Opus 4.8 39.6%
3. #3 Claude Opus 4.7 37.7%

Anthropic internal biological-protocol troubleshooting evaluation reported in the Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 69.6%
2. #2 Claude Opus 4.8 59.6%
3. #3 Claude Opus 4.7 51.8%

Numerical forecasting benchmark evaluating whether LLMs produce calibrated 90% prediction intervals for 1,000 real-world questions under zero-shot, grounded, and agentic retrieval settings.

1. #1 Gemini 3.1 Pro 0.7910 coverage
2. #2 Grok 4 0.7638 coverage
3. #3 GPT-5.4 0.7533 coverage

Real-user-distribution Claw agent benchmark referenced in Qwen's Qwen3.7-Max launch post.

1. #1 Claude Opus 4.6 Max 65.5%
2. #2 Qwen3.7 Max 64.3%
3. #3 DeepSeek V4 Pro Max 59.2%

Qwen-published SVG generation benchmark reported as a comparison-table score for frontier coding and artifact-generation models.

1. #1 Qwen3.7 Max 1608
2. #2 GLM-5.1 Thinking 1605
3. #3 Claude Opus 4.6 Max 1541

Qwen internal front-end code generation benchmark covering bilingual web artifact tasks with auto-rendering and multimodal judging.

1. #1 Claude Opus 4.6 Max 1617
2. #2 DeepSeek V4 Pro Max 1570
3. #3 Qwen3.7 Max 1568

Qwen internal benchmark evaluating LLMs as world models for simulating agentic environments across Terminal, SWE, MCP, Search, OS, Android, and Web domains.

1. #1 Qwen3.7 Max 57.3%
2. #2 Claude Opus 4.6 Max 56.1%
3. #3 DeepSeek V4 Pro Max 52.3%

Open evaluation framework from Callstack measuring AI model performance on real-world React Native development tasks across navigation, animation, async state, lists, and React Native APIs.

1. #1 Composer 2 96.0806% overall
2. #2 Composer 2 Fast 94.8718% overall
3. #3 Apex 87.8022% overall

Cybersecurity benchmark with 30K multiple-choice and 240 open-ended QA items covering knowledge, offensive skills, and tool expertise.

1. #1 GPT-5 88.68%
2. #2 RedSage-8B-Ins 85.73%
3. #3 Qwen3-32B 85.4%

Vendor-reported 928-question finance-agent benchmark spanning vertical-specific skills, metrics, financial-statement analysis, and forecasting workflows.

1. #1 Claude Opus 4.7 59% rubric / 41% final
2. #2 GPT-5.5 59% rubric / 44% final
3. #3 Claude Sonnet 4.6 59% rubric / 38% final

Student Assessment with Generative Evaluation

1. #1 Claude Opus 4.7 56.103%
2. #2 Gemma 4 31B It 55.034%
3. #3 Claude Opus 4.8 54.788%

Security engineering benchmark evaluating agents on vulnerability discovery, proof-of-concept generation, and vulnerability patching targets.

1. #1 Claude Code (v2.1.81) + Opus 4.6 (high) (SpiderMonkey) 38.8% resolved
2. #1 OpenAI Codex (v0.115.0) + GPT-5.4 (xhigh) (V8) 32.0% resolved
3. #2 Claude Code (v2.1.81) + Opus 4.6 (high) (V8) 21.4% resolved

Security benchmark for AI-generated and AI-repaired code, reporting secure-code repair and generation scores with and without hints.

1. #1 Qwen3.5-Plus(Thinking) 67.08%
2. #2 Claude Opus 4.6 64.9%
3. #3 Qwen3-Coder-Next 62.76%

SEC filing retrieval and question-answering benchmark for financial RAG over public company disclosures.

1. #1 GPT-4o 0.69
2. #2 Llama-3.3-70B-Instruct 0.65
3. #3 GPT-4o-mini 0.64

Synthetic meta-evaluation benchmark for text-to-long-video evaluation systems, covering long videos and ten quality/consistency aspects.

1. #1 Human 91.73%
2. #2 Video-based GPT-5 71.66%
3. #3 Text-based GPT-5-mini 61.35%

Structured-output benchmark evaluating text and visual structured generation and conversion across 18 formats and 2,035 examples.

1. #1 GPT-4o 76.02%
2. #2 GPT-4.1-mini 75.64%
3. #3 o1-mini 75.58%

Open-ended structural-biology evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 81.6%
2. #2 Claude Opus 4.8 79%
3. #3 Claude Opus 4.7 74%

Solving production software engineering tasks

1. #1 Claude Opus 4.8 88.6%
2. #2 GPT 5.5 82.6%
3. #3 Claude Opus 4.7 82%

A Vals-created set of questions and responses to tax questions

1. #1 Muse Spark 77.678%
2. #2 Claude Sonnet 4.6 77.106%
3. #3 Claude Opus 4.6 Thinking 75.961%

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.AMES, reporting AUROC on submitted molecular models.

1. #1 ZairaChem 0.871 AUROC
2. #2 ADMETrix 0.87 AUROC
3. #3 MapLight + GNN 0.869 AUROC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.BBB_Martins, reporting AUROC on submitted molecular models.

1. #1 MiniMol 0.924 AUROC
2. #2 CFA 0.92 AUROC
3. #3 MapLight 0.916 AUROC

Therapeutics Data Commons drug-target interaction domain-generalization benchmark on BindingDB Patent.

1. #1 Otter-Knowledge-Ensemble 0.588 pcc
2. #2 OrangeBalls-ProtBertMorgan 0.538 pcc
3. #3 MMD 0.433 pcc

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Bioavailability_Ma, reporting AUROC on submitted molecular models.

1. #1 MiniMol 0.942 AUROC
2. #2 MapLight + GNN 0.938 AUROC
3. #3 ZairaChem 0.935 AUROC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Caco2_Wang, reporting MAE on submitted molecular models.

1. #1 CaliciBoost 0.256 MAE
2. #2 XG Boost 0.274 MAE
3. #3 MapLight 0.276 MAE

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Clearance_Hepatocyte_AZ, reporting Spearman on submitted molecular models.

1. #1 CFA 0.536 Spearman
2. #2 MapLight + GNN 0.498 Spearman
3. #3 MapLight 0.466 Spearman

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Clearance_Microsome_AZ, reporting Spearman on submitted molecular models.

1. #1 MapLight + GNN 0.63 Spearman
2. #2 MiniMol 0.628 Spearman
3. #3 MapLight 0.626 Spearman

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.CYP2C9_Substrate_CarbonMangels, reporting AUPRC on submitted molecular models.

1. #1 MiniMol 0.474 AUPRC
2. #2 ZairaChem 0.441 AUPRC
3. #3 MapLight + GNN 0.437 AUPRC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.CYP2C9_Veith, reporting AUPRC on submitted molecular models.

1. #1 MapLight + GNN 0.859 AUPRC
2. #2 ContextPred 0.839 AUPRC
3. #3 AttrMasking 0.829 AUPRC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.CYP2D6_Substrate_CarbonMangels, reporting AUPRC on submitted molecular models.

1. #1 ContextPred 0.736 AUPRC
2. #2 DeepMol (AutoML) 0.731 AUPRC
3. #3 MapLight + GNN 0.72 AUPRC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.CYP2D6_Veith, reporting AUPRC on submitted molecular models.

1. #1 MapLight + GNN 0.79 AUPRC
2. #2 ContextPred 0.739 AUPRC
3. #3 MapLight 0.723 AUPRC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.CYP3A4_Substrate_CarbonMangels, reporting AUROC on submitted molecular models.

1. #1 CFA 0.667 AUROC
2. #2 MiniMol 0.663 AUROC
3. #3 CNN (DeepPurpose) 0.662 AUROC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.CYP3A4_Veith, reporting AUPRC on submitted molecular models.

1. #1 MapLight + GNN 0.916 AUPRC
2. #2 ContextPred 0.904 AUPRC
3. #3 AttrMasking 0.902 AUPRC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.DILI, reporting AUROC on submitted molecular models.

1. #1 MiniMol 0.956 AUROC
2. #2 ZairaChem 0.925 AUROC
3. #3 AttrMasking 0.919 AUROC

Therapeutics Data Commons docking benchmark for DRD3 molecule generation with oracle-call-budget leaderboard tables.

1. #1 Graph-GA (Leaderboard: Models Make at Most 5000 Oracle Calls) -14.811 top100
2. #2 SMILES-LSTM (Leaderboard: Models Make at Most 5000 Oracle Calls) -13.017 top100
3. #3 GCPN (Leaderboard: Models Make at Most 5000 Oracle Calls) -10.045 top100

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Half_Life_Obach, reporting Spearman on submitted molecular models.

1. #1 CFA 0.576 Spearman
2. #2 MapLight 0.562 Spearman
3. #3 MapLight + GNN 0.557 Spearman

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.hERG, reporting AUROC on submitted molecular models.

1. #1 MapLight + GNN 0.88 AUROC
2. #2 CFA 0.875 AUROC
3. #3 SimGCN 0.874 AUROC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.HIA_Hou, reporting AUROC on submitted molecular models.

1. #1 MiniMol 0.993 AUROC
2. #2 DeepMol (AutoML) 0.99 AUROC
3. #3 MapLight + GNN 0.989 AUROC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.LD50_Zhu, reporting MAE on submitted molecular models.

1. #1 BaseBoosting KyQVZ6b2 0.552 MAE
2. #2 ADMETrix 0.573 MAE
3. #3 MiniMol 0.585 MAE

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Lipophilicity_AstraZeneca, reporting MAE on submitted molecular models.

1. #1 MiniMol 0.456 MAE
2. #2 Chemprop-RDKit 0.467 MAE
3. #3 Chemprop 0.47 MAE

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Pgp_Broccatelli, reporting AUROC on submitted molecular models.

1. #1 MapLight + GNN 0.938 AUROC
2. #2 ZairaChem 0.935 AUROC
3. #3 MapLight 0.93 AUROC

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.PPBR_AZ, reporting MAE on submitted molecular models.

1. #1 Gradient Boost 7.44 MAE
2. #2 MapLight + GNN 7.526 MAE
3. #3 MapLight 7.66 MAE

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.Solubility_AqSolDB, reporting MAE on submitted molecular models.

1. #1 MiniMol 0.741 MAE
2. #2 Chemprop-RDKit 0.761 MAE
3. #3 DeepMol (AutoML) 0.775 MAE

Therapeutics Data Commons TCR-epitope binding interaction benchmark with NA and RN sampling-method leaderboards.

1. #1 MIX-TPI (Leaderboard 1 : NA Sampling Method) 0.995 auprc
2. #2 NET-TCR2 (Leaderboard 1 : NA Sampling Method) 0.985 auprc
3. #3 TEINet (Leaderboard 1 : NA Sampling Method) 0.981 auprc

Therapeutics Data Commons ADMET benchmark leaderboard for TDC.VDss_Lombardo, reporting Spearman on submitted molecular models.

1. #1 MapLight + GNN 0.713 Spearman
2. #2 MapLight 0.707 Spearman
3. #3 CFA 0.628 Spearman

State-of-the-art set of difficult terminal-based tasks

1. #1 Claude Mythos Preview 82%
2. #1 GPT 5.5 73.202%
3. #1 Qwen3.7 Max 69.7%

State-of-the-art set of difficult terminal-based tasks

1. #1 GPT 5.5 76.404%
2. #2 Claude Opus 4.8 74.6%
3. #2 Gemini 3.5 Flash 74.157%

Simulator-based bilateral price negotiation benchmark for LLM agents, measuring surplus extraction, feasible agreement calibration, belief error, and procedural compliance without an LLM judge.

1. #1 Claude Opus 4.6 69.4% SE+
2. #2 GLM 5.1 68.6% SE+
3. #3 Claude Opus 4.7 66.0% SE+

Thai language and Thai cultural-context safety benchmark reporting attack success rate across harmful-content categories.

1. #1 GPT-5 4.43% overall ASR
2. #2 Claude 4.5 Sonnet 9.75% overall ASR
3. #3 SeaLLMs-v3-7B 9.83% overall ASR

2026 USA Mathematical Olympiad proof-based evaluation reported in Anthropic's Claude Opus 4.8 system card using MathArena-style model-judge grading.

1. #1 Claude Opus 4.8 96.7%
2. #2 Claude Opus 4.7 69.3%

Benchmark consisting of a weighted performance across finance and coding tasks. Showing the potential impact that LLM's can have on the economy.

1. #1 Claude Opus 4.8 70.166%
2. #2 GPT 5.5 67.622%
3. #3 Claude Opus 4.7 66.099%

Benchmark consisting of a weighted performance across finance, coding, and education tasks. Showing the potential impact that LLM's can have on the economy.

1. #1 Claude Opus 4.8 70.712%
2. #2 GPT 5.5 67.768%
3. #3 Claude Opus 4.7 67.361%

Multi-modal structured extraction benchmark over 1,777 government forms with per-document JSON schemas and image/text/spatial modalities.

1. #1 Gemini 2.5 Pro 98.0% EM
2. #2 Gemini 2.5 Flash 97.3% EM
3. #3 Qwen3-VL 96.6% EM

SecureBio Virology Capabilities Test multimodal virology evaluation reported in Anthropic's Claude Opus 4.8 system card.

1. #1 Claude Mythos Preview 0.6
2. #2 Claude Opus 4.8 0.5

Long-horizon autonomous agent benchmark measuring how well models operate a simulated vending-machine business over an extended period.

1. #1 Claude Opus 4.7 10936.76
2. #2 Claude Opus 4.6 8017.59
3. #3 Claude Opus 4.8 (high effort) 5787.4 USD

Can models build web applications from scratch?

1. #1 Claude Opus 4.8 82.725%
2. #2 Claude Opus 4.7 71.003%
3. #3 GPT 5.5 69.847%

Vision-language model throughput benchmark reporting peak tokens/sec, TTFT, TPOT, and worker count on an RTX PRO 6000 Blackwell vLLM setup.

1. #1 lightonai/LightOnOCR-2-1B 2439.8 tok/s
2. #2 Qwen/Qwen3-VL-2B-Instruct 2409.3 tok/s
3. #3 PaddlePaddle/PaddleOCR-VL 2341.9 tok/s

Reproducible browser-agent benchmark of 532 tedious web tasks extending WebArena with massive-memory, calculation, and long-term-memory chores.

1. #1 BrowserGym + Gemini 2.5 Pro (preview-03-25) 44.9%
2. #2 AgentOccam + Gemini 2.5 Pro (preview-03-25) 37.8%
3. #3 AgentOccam + Claude 3.7 Sonnet (claude-3-7-sonnet-20250219) 23.5%

Agent API-use benchmark measuring robustness to real-world API complexity across multi-turn conversations, API functions, and injected complexity scenarios.

1. #1 Claude-4.0-Sonnet (Think) 67.5%
2. #2 Claude-4.0-Sonnet 63.6%
3. #3 Qwen3-235B-Thinking 62.6%

Interactive game-agent leaderboard where LLMs play games in LLM-generated contexts with LLM-enforced rules and LLM-scored outcomes.

1. #1 gemini-2.5-pro 65.86 overall
2. #2 grok-4-0709 63.22 overall
3. #3 gpt-5 62.77 overall

Agentic backend coding benchmark evaluating whether coding agents can explore real repositories, edit code, configure environments, deploy containerized services, and pass external HTTP integration tests.

1. #1 Claude Sonnet 4.5 63.2% +/- 1.9
2. #2 DeepSeek-V3.2 50.1% +/- 1.9
3. #3 GPT-5 49.4% +/- 1.9

ActivityNet-QA: Evaluates temporal, video, speech, or audio understanding beyond static text and image inputs.

1. #1 E-SA 31.8%
2. #2 E-MN 27.1%
3. #3 E-VQA 25.1%

AdvBench: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 meta-llama/Llama-2-7b-chat-hf 0.999099
2. #2 meta-llama/Llama-2-13b-chat-hf 0.99903
3. #3 meta-llama/Llama-2-70b-chat-hf 0.998804

AgentBoard: Evaluates autonomous agent performance on multi-step tasks requiring planning, state tracking, tool use, and recovery.

1. #1 GPT-4 70%
2. #2 Claude2 48.9%
3. #3 GPT-3.5-Turbo 41.4%

Clinical simulation benchmark for sequential diagnostic decision-making with multimodal patient interactions and compliance checks.

1. #1 Claude 3.5 62.1%
2. #2 GPT-4 51.6%
3. #3 Mixtral-8x7B 37.1%

AgentDojo: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 claude-3-5-sonnet-20241022 (None, important_instructions) 85.69%
2. #2 claude-3-7-sonnet-20250219 (None, important_instructions) 84.98%
3. #3 gpt-4o-2024-05-13 (None, direct) 81.8%

AgentHarm: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 Llama-3.1 8B (None) 3.1%
2. #2 Llama-3.1 405B (None) 4.3%
3. #3 Llama-3.1 405B (Template) 4.3%

Universal benchmark for evaluating sequential decision agents across RL, LLM, VLM, modality, harness, task, and category settings.

1. #1 Oracle Agent 0.895
2. #2 Qwen3.5-4B (SFT-250k) 0.447
3. #4 Qwen3.5-4B (SFT-120k) 0.354

Agent instruction-following benchmark measuring constraint and instruction success across vanilla, condition, example, formatting, semantic, and tool constraints.

1. #1 o1-mini 59.8
2. #2 GPT-4o 58.5
3. #3 Qwen3-32B 58.4

Spreadsheet-assistant benchmark with realistic spreadsheet prompts, dynamic-output checks, and latency measurements.

1. #1 Rows AI Analyst 89%
2. #2 Microsoft Excel Copilot 53%
3. #3 Google Sheets + Gemini 57%

AI2D: Measures visual question answering, OCR, document understanding, chart comprehension, or layout-aware reasoning.

1. #1 Qwen3.5-9B 90.2
2. #2 InternVL3-78B 89.7
3. #3 Qwen3.5-4B 89.6

Aider polyglot coding-agent leaderboard over 225 Exercism tasks across C++, Go, Java, JavaScript, Python, and Rust.

1. #1 gpt-5 (high) (diff, 2025-08-23) 88
2. #2 gpt-5 (medium) (diff, 2025-08-25) 86.7
3. #3 o3-pro (high) (diff, 2025-06-28) 84.9

AIME 2024: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 Grok 4 94.0
2. #2 Magistral Medium 1.2 91.8
3. #3 DeepSeek R1 0528 91.4

Audio instruction benchmark evaluating large audio-language models on generative comprehension across chat and foundation audio tasks.

1. #1 Qwen-Audio-Turbo (foundation) 58.285
2. #2 Qwen-Audio (foundation) 54.595
3. #3 Whisper+GPT 4 (foundation) 53.5889

Embodied instruction-following benchmark in AI2-THOR for mapping natural language goals and instructions to household actions.

1. #1 Human Performance 91%
2. #2 GRL 68.52%
3. #3 EPO 62.35%

ALFWorld: Measures embodied-agent, navigation, manipulation, or simulated robotics task success.

1. #1 claude-opus-4.5 (high) 1.0
2. #2 claude-opus-4.6 (high) 1.0
3. #3 claude-sonnet-4.6 (high) 1.0

Audio-text conflict benchmark measuring whether audio-language models follow the audio signal instead of conflicting text.

1. #1 Ultravox v0.6 48.8%

Automatic instruction-following evaluator comparing model responses against a reference using GPT-4 judgments and length-controlled win rates.

1. #1 xwinlm-70b-v0.1 95.56803995
2. #2 Mistral-7B-ReMax-v0.1 94.39601494396015
3. #3 xwinlm-70b-v0.3 94.01522563893708

Reasoning benchmark for conditioned override, testing whether models fall back to memorized answers when familiar riddles are deliberately modified with constraints, context swaps, meaning shifts, or bias probes.

1. #1 xiaomi/mimo-v2-pro, high reasoning 0.2873
2. #2 openai/gpt-oss-20b (MXFP4), high reasoning 0.2896
3. #3 openai/gpt-5.4-mini, high reasoning 0.3058

AMC 12: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

Android development benchmark measuring how well LLM agents resolve real Android tasks, with success rate, latency, token use, and cost.

1. #1 GPT 5.5 74.0
2. #2 GPT 5.4 72.4
3. #3 Gemini 3.1 Pro Preview 72.4

AndroidWorld: Measures browser, desktop, mobile, or GUI agents operating in interactive environments.

1. #1 AGI-0 97.4%
2. #2 Gemini 3 flash, Gemini 3 flash lite 97.4%
3. #3 Seed1.8-GUI 97.4%

APPS: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

1. #1 GPT-Neo 2.7B 10.15
2. #2 GPT-2 1.5B 7.96
3. #3 GPT-2 0.1B 6.16

ARC Challenge: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 DGEM 27.11
2. #2 TableILP 26.97
3. #3 BiDAF 26.54

ARC Easy: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 PMI (dataset definition) 77.82
2. #2 IR (dataset definition) 74.48
3. #3 IR (using ARC Corpus) 62.55

Arena-Hard: Evaluates conversational quality, human preference, helpfulness, and pairwise response judgments.

1. #1 o3-2025-04-16 85.9%
2. #2 o4-mini-2025-04-16-high 79.1%
3. #3 gemini-2.5 79.0%

Multi-turn voice-agent benchmark with appointment, assistant, conversation, event, grocery, and product workflows scored across multiple dimensions.

1. #1 Ultravox V0.7 77.0199
2. #2 Grok Voice Think Fast 1.0 75.897
3. #3 Gpt Realtime 73.4481

Audio-language benchmark covering speech, sound, music, ASR, QA, translation, and audio reasoning tasks across many task-level metrics.

1. #1 gpt-4o-audio 57.1219
2. #2 gemini-1.5-flash 47.951
3. #3 MERaLiON-AudioLLM-Whisper-SEA-LION 47.8887

Medical AutoResearch benchmark for single-LLM coding agents across segmentation, image enhancement, VQA, report generation, and lesion detection, with workflow and task-quality scoring.

1. #1 Claude Opus 4.6 69.6946
2. #2 GLM-5 63.3196
3. #3 Gemini 3.1 Pro 61.2258

BabyAI: Measures embodied-agent, navigation, manipulation, or simulated robotics task success.

1. #1 Gemini-3.1-Pro (LLM) 100.0 0.0
2. #2 Gemini-3.1-Pro-Thinking (LLM) 98.0 2.0
3. #3 Gemini-3-Pro (LLM) 96.0 2.8

Robotics challenge leaderboard for BEHAVIOR-1K household activity policies, ranked by Q-score and full-task success.

1. #1 Robot Learning Collective 0.2599
2. #2 Comet 0.2514
3. #3 SimpleAI Robot 0.1591

Measures AI models' ability to correctly call and use functions in various contexts

1. #1 Claude-Opus-4-5-20251101 (FC) 77.47%
2. #2 Claude-Sonnet-4-5-20250929 (FC) 73.24%
3. #3 Gemini-3-Pro-Preview (Prompt) 72.51%

BIG-bench: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 PaLM 535b 8.110951766293224
2. #2 PaLM 64b 8.110951766293224
3. #3 BIG-G T=0 128b 6.1313369188035844

BioASQ: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 DMIS_MES_TEST_1 0.7415
2. #2 DMIS_MES_TEST_2 0.7415
3. #3 DMIS_MES_TEST_3 0.7415

BizBench: Evaluates financial analysis, accounting, market reasoning, and quantitative business tasks.

Business-driven financial benchmark covering anomalous event attribution, numerical computation, time reasoning, financial QA, event relation, stock prediction, and entity recognition.

1. #1 ChatGPT-o3 73.86
2. #2 DeepSeek-R1 (671B) 73.05
3. #3 GPT-4o 71.8

BOLD: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 Claude3Opus 0.757401
2. #2 mistralai/Mistral-7B-v0.3 0.742951
3. #3 gemini-1.5-flash-001 0.740392

BoolQ: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 Palmyra X (43B) 89.633333%
2. #2 Llama 2 (70B) 88.6%
3. #3 text-davinci-003 88.133333%

Cybersecurity benchmark measuring AI agent detection, exploitation, and patching on real-world bug bounty tasks, including success rates, bounty value, and token costs.

1. #1 Claude Code 50
2. #2 OpenAI Codex CLI: o3-high 50
3. #3 C-Agent: Claude 3.7 44.1667

Clinical practice text understanding leaderboard for medical LLMs, covering summarization, dialogue, clinical evidence, and EHR-oriented tasks across multiple prompting settings.

1. #1 gemini-1.5-pro-002 (Few-Shot) 55.51
2. #2 gemma-4-31B-it (Few-Shot) 54.88
3. #3 gemini-2.5-flash (Few-Shot) 53.36

BugsInPy: Evaluates software-engineering agents on realistic issue resolution, repository navigation, testing, or maintenance workflows.

BugsJS: Evaluates software-engineering agents on realistic issue resolution, repository navigation, testing, or maintenance workflows.

C-Eval: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 海信星海 92.3%
2. #2 BlueLM 92.2%
3. #3 讯飞星火认知大模型（Spark4.0 Max） 91.8%

Composite CAIS AI Dashboard risk index averaging VCT refusal risk, HLE miscalibration, MASK risk, Machiavelli, and TextQuests Harm for models with all component scores. Lower is better.

1. #1 Opus 4.7 32.9
2. #2 Sonnet 4.5 34.1
3. #3 Opus 4.5 34.7

Composite CAIS AI Dashboard text index averaging Humanity's Last Exam, ARC-AGI-2, TextQuests, and SWE-bench Pro for models with all component scores.

1. #1 GPT-5.5 54.1
2. #2 Gemini 3.1 Pro 52.9
3. #3 GPT-5.4 49.3

Composite CAIS AI Dashboard vision index averaging EnigmaEval, IntPhys2, ERQA, MindCube, ART, and SpatialViz for models with all component scores.

1. #1 Gemini 3.5 Flash 65.7
2. #2 Gemini 3 Flash 64.9
3. #3 Gemini 3.1 Pro 63.1

Language-conditioned robot manipulation benchmark for long-horizon sequences and multitask learning in tabletop environments.

1. #1 FLOWER (Train A, B, C, D -> Test D) 4.67
2. #2 UniVLA (Train A, B, C, D -> Test D) 4.63
3. #3 FLOWER (Train A, B, C -> Test D) 4.53

Autonomous-driving agent leaderboard in CARLA measuring driving score, route completion, and infraction penalties.

1. #1 CarLLaVA (LLM4AD) 6.865108
2. #1 Kyber-E2E 6.975326
3. #1 LRM 42.274775

CaseHOLD: Measures legal reasoning, contract review, statute interpretation, or legal-domain QA.

1. #1 CaseLaw-BERT 75.4
2. #2 Legal-BERT 75.3
3. #3 DeBERTa 72.6

Remote-sensing vision-language benchmark for perception and reasoning over Earth-observation imagery across multiple task dimensions.

1. #1 Qwen2-VL-72B 0.7378
2. #2 InternVL2-40B 0.7263
3. #3 Ovis1.6-Gemma2-9B 0.6998

Class-level code generation benchmark evaluating class and function success under multiple prompting strategies.

1. #1 GPT-4 (H) 37.6
2. #2 GPT-3.5 (H) 29.6
3. #5 GPT-3.5 (I) 25.6

Quarterly refreshed enterprise-workflow benchmark grounded in live ClawHub marketplace signals and scored with deterministic checks plus structured judging.

1. #1 Claude Opus 4.6 66.7
2. #2 GPT-5.4 63.8
3. #3 Claude Sonnet 4.6 61.9

Climate model emulation benchmark over NorESM2 simulation outputs, reporting NRMSE for climate variables under future scenarios.

1. #1 Neural Network 6.1548
2. #2 Gaussian Process 7.9525
3. #3 Random Forest 8.3227

Clotho-AQA: Evaluates temporal, video, speech, or audio understanding beyond static text and image inputs.

1. #1 Binary classifier (Question only) 64.4%
2. #2 Binary classifier (Audio + question) 63.2%
3. #3 Binary classifier (Audio only) 58.2%

CodeContests: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

1. #1 AlphaCode 41B + clustering 29.6%
2. #2 AlphaCode 41B 27.7%
3. #3 AlphaCode 9B 25.8%

Code editing benchmark covering debugging, code translation, requirement switching, and code polishing across primary and plus splits.

1. #1 gpt-4-0613 (plus, Three-shot) 0.882
2. #4 gemini-ultra (primary, Three-shot) 0.855
3. #7 OpenCodeInterpreter-DS-33B (plus, Zero-shot) 0.816

CommonsenseQA: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 BERT-Large 55.9%
2. #2 GPT 45.5%
3. #3 ESIM+ELMo 34.1%

ContractNLI: Measures legal reasoning, contract review, statute interpretation, or legal-domain QA.

1. #1 Span NLI (BERTlarge) 0.875
2. #2 Span NLI (BERTbase) 0.838
3. #3 Doc TF-IDF+SVM 0.733

ConvFinQA: Evaluates financial analysis, accounting, market reasoning, and quantitative business tasks.

1. #1 Human Expert Performance 89.44
2. #2 FinQANet-Gold (RoBERTa-large) 77.32
3. #3 FinQANet (RoBERTa-large) 68.9

COPA: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 BERT tuned with Social IQa 84.4%
2. #2 GPT 78.6%
3. #3 Learning to Rank for Plausible Plausibility 75.4%

HAL's cost-aware agent leaderboard for CORE-Bench Hard scientific programming tasks.

1. #1 Claude Code / Claude Opus 4.5 77.78
2. #2 Claude Code / Claude Sonnet 4.5 (September 2025) 62.22
3. #3 CORE-Agent / Claude Opus 4.1 (August 2025) 51.11

CrowS-Pairs: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 BERT 60.5
2. #2 RoBERTa 64.1
3. #3 ALBERT 67.0

Code reasoning benchmark for input and output prediction, reporting pass@1 and pass@5 across code language models.

1. #1 gpt-4-turbo-2024-04-09+cot (n=3) 78.85
2. #2 claude-3-opus+cot (n=1) 77.70
3. #3 gpt-4-0613+cot 76.30

CUAD: Measures legal reasoning, contract review, statute interpretation, or legal-domain QA.

1. #1 DeBERTa-xlarge 44.0
2. #2 RoBERTa-large 38.1
3. #3 RoBERTa-base + Contracts Pretraining 34.1

Cybersecurity benchmark for autonomous web vulnerability exploitation across 40 critical CVEs in zero-day and one-day settings.

1. #1 Default Agent + Claude Opus 4.6 (oneDay) 40
2. #2 Default Agent + Claude Opus 4.6 (zeroDay) 32.5
3. #3 T-Agent + GPT-4o (2024-11-20) (zeroDay) 8

Cybench: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 openai/gpt-4-turbo-2024-04-09 36%

CyberSecEval: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 codellama-70b-instruct 12.93%
2. #2 gpt-4 19.87%
3. #3 llama 3 70b-instruct 29.27%

Real-world tabular question-answering benchmark over many datasets, used in SemEval 2025 Task 8.

1. #1 TeleAI 95.02%
2. #2 SRPOL AIS 89.66%
3. #3 HITSZ-HLT 86.97%

Multilingual variant of DeepResearch Bench for evaluating deep research agents across translated prompt sets and languages.

Deep information-synthesis benchmark for agents that must gather, browse, extract, and reason over multiple sources to produce structured answers.

1. #1 o3-deep-research 8.97
2. #2 GPT-5.2-Pro 8.70
3. #3 Smolagent (GPT-5) 6.42

Defects4J: Evaluates software-engineering agents on realistic issue resolution, repository navigation, testing, or maintenance workflows.

1. #1 o4-mini-2025-04-16-high 0.538
2. #2 o3-mini-2025-01-31-high 0.488
3. #3 claude-3-7-sonnet-20250219 0.478

Software-development lifecycle benchmark covering environment setup, implementation, acceptance testing, unit testing, and software design.

1. #1 GPT-4-Turbo-0125 56.1636
2. #3 DeepSeek-Coder-33B-Instruct 35.8182
3. #4 DeepSeek-Coder-6.7B-Instruct 27.8545

Document information localization and extraction benchmark over business documents such as invoices, purchase orders, and line items.

1. #1 GraphDoc ( RRC ) 7 placement points
2. #2 YOLOv8 ( RRC ) 3 placement points
3. #3 Union-RoBERTa ( RRC ) 1 placement points

DocVQA: Measures visual question answering, OCR, document understanding, chart comprehension, or layout-aware reasoning.

1. #1 Human Performance 0.9811
2. #2 ORCA 0.9729
3. #3 NEXT-8B 0.9728

DS-1000: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

1. #1 gpt-4-turbo-2024-04-09 0.539
2. #2 gpt-4-0613 0.51
3. #3 gpt-3.5-turbo-0125 0.394

End-to-end document understanding benchmark suite with a public evaluator and task-specific document AI metrics.

1. #1 T5 + 2D + Self-supervised 59.76
2. #2 T5 + Self-Supervised 56.51
3. #3 T5 Baseline 50.71

Dynamic speech-language-model benchmark and leaderboard for speech instruction following across many audio tasks.

1. #1 DeSTA2.5-Audio 46.18
2. #2 Qwen2-Audio-7B-Instruct 39.88
3. #3 Qwen-Audio-Chat 36.59

Education benchmark spanning student and teacher scenarios, educational tasks, and multidimensional educational response quality.

1. #1 DeepSeek R1 (scenario, evaluator: QwQ-Plus) 9.49
2. #2 DeepSeek R1 (dimension, evaluator: QwQ-Plus) 9.37
3. #3 DeepSeek R1 (scenario, evaluator: DeepSeek R1) 9.29

Education-specific safety benchmark for teaching harm, adversarial safety, pedagogical fidelity, refusal, and safe tutoring behavior.

1. #1 Claude-3.7 0.77
2. #2 Deepseek-R1 0.75
3. #3 Qwen3-32B-R 0.75

EgoSchema: Evaluates temporal, video, speech, or audio understanding beyond static text and image inputs.

1. #1 Reality Distortion 80.759292%
2. #2 ft_llava_72B (TarHeels) 80.600278%
3. #3 PCIE (L_PCIE) 78.632479%

ETHICS: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 ALBERT-xxlarge 71%
2. #2 RoBERTa-large 68%
3. #3 BERT-large 56.1%

EvalPlus leaderboard aggregating HumanEval+ and MBPP+ code-generation pass@1 scores.

1. #1 O1 Preview (Sept 2024) 84.60
2. #2 O1 Mini (Sept 2024) 83.90
3. #3 Qwen2.5-Coder-32B-Instruct 82.10

FEVER: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 UNC-NLP 0.6398
2. #2 UCL Machine Reading Group 0.6234
3. #3 Athene UKP TU Darmstadt 0.6132

FinanceQA leaderboard for industry-grade financial analysis tasks across basic tactical, assumption-based tactical, and conceptual categories.

1. #1 o3 54.1
2. #2 Grok 4 49.3
3. #3 o4 mini 48.6

FinBen: Evaluates financial analysis, accounting, market reasoning, and quantitative business tasks.

1. #1 GPT-4 28.19%
2. #2 gemini 14.95%
3. #3 GPT3.5-Turbo 4.48%

Finance and accounting workflow benchmark over spreadsheets, PDFs, and back-office tasks requiring file-grounded outputs.

Chinese financial-domain benchmark covering financial academic knowledge, industry knowledge, security, financial agents, multimodal finance tasks, and rigor testing.

1. #1 Ant Group Finix-CI-72B (fineval 6 0) 86.07
2. #2 DeepSeek DeepSeek-RI (fineval 6 0) 85.25
3. #3 Alibaba Qwen3-32B (think) (fineval 6 0) 84.53

FinQA: Evaluates financial analysis, accounting, market reasoning, and quantitative business tasks.

1. #1 Human Expert Performance 91.16
2. #2 FinQANet-Gold (RoBERTa-large) 70.0
3. #3 FinQANet (RoBERTa-large) 61.24

Financial tool-use benchmark with real tools and APIs, measuring tool invocation, execution success, compliance, and soft-scored task quality.

1. #1 Qwen3-8B 0.4234
2. #2 Doubao-Seed-1.6 0.3958
3. #3 GLM-4.7-Flash 0.2769

FLUE: Evaluates financial analysis, accounting, market reasoning, and quantitative business tasks.

FreshQA: Measures long-context retrieval, needle finding, summarization, factual grounding, or retrieval-augmented generation quality.

1. #1 GPT-4 46.4%
2. #2 ChatGPT 41.4%
3. #3 GPT-3.5 32.4%

FrontierMath: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 GPT-5.2 40.3
2. #2 GPT-5.5 35.4

HAL's standardized, cost-aware agent leaderboard for GAIA web assistance tasks.

1. #1 HAL Generalist Agent / Claude Sonnet 4.5 (September 2025) 74.55
2. #2 HAL Generalist Agent / Claude Sonnet 4.5 High (September 2025) 70.91
3. #3 HAL Generalist Agent / Claude Opus 4.1 High (August 2025) 68.48

Gaokao-Bench: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 GPT-4-0314 72.2%
2. #3 Gemini-Pro 57.9%
3. #4 ERNIE-Bot-0615 56.6%

GeoQA: Evaluates multimodal understanding across image, text, chart, diagram, or cross-modal reasoning tasks.

1. #1 NGS-Auxiliary 60%

Geolocation reasoning benchmark for vision-language models, measuring location inference, F1, and country-level accuracy.

1. #1 Human Expert #3 (Best Expert) 97.33
2. #2 Human Expert #1 96.67
3. #3 Human Expert #2 90

General information-seeking assistant benchmark for structured item, set, list, and table answers from search-capable systems.

1. #1 Claude 4.5 Sonnet (thinking) 19.3
2. #2 Qwen3-Max (thinking) 17.96
3. #3 Claude 4.5 Sonnet (non-thinking) 16.36

Repository-level code-agent benchmark covering real GitHub tasks, reporting task pass rate, execution completion rate, token usage, and cost.

1. #1 RepoMaster + Claude 3.5 62.96
2. #2 OpenHands + Claude 3.7 48.15
3. #3 RepoMaster + DeepSeekV3 44.44

GQA: Evaluates multimodal understanding across image, text, chart, diagram, or cross-modal reasoning tasks.

1. #1 c_q7_m8 (B. Zacharie) 87.85%
2. #2 Molmo2-8B (Allen Institute for AI) 87.07%
3. #3 10B, 448px, finetune (PaliGemma 2) 86.95%

AI Habitat embodied-navigation and rearrangement challenge leaderboards across PointNav, ObjectNav, Rearrange-Easy, and OVMM tracks.

1. #1 Arnold (2020 ObjectNav) 0.1
2. #1 ByteBOT (2022 ObjectNav) 0.35
3. #1 inspir.ai robotics (2021 PointNav) 0.74

HANS: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 esim 49.416667%
2. #2 decomp attn 49.19%
3. #3 bert 48.733333%

Healthcare administration agent benchmark for prior authorization, appeals, durable medical equipment, payer portals, fax, and EHR-adjacent workflows.

1. #1 openai-cua 84.94%
2. #2 anthropic-cua 81.77%
3. #3 gemini-3.1 73.39%

HealthBench: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 o3 0.5990
2. #2 GPT-4.1 0.4778
3. #3 o1 0.4200

HealthBench Hard: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 openai/gpt-oss-120b 0.6
2. #2 google/gemma-3-27b-it 0.59
3. #3 Qwen/Qwen3-30B-A3B-Thinking-2507 0.58

HELM: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 Llama 2 (70B) 94.351981%
2. #2 LLaMA (65B) 90.825175%
3. #3 text-davinci-002 90.502788%

HELM GSM8K: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 Palmyra X (43B) 63.3%
2. #2 code-davinci-002 56.766667%
3. #3 gpt-3.5-turbo-0301 53.1%

HELM LegalBench: Measures legal reasoning, contract review, statute interpretation, or legal-domain QA.

1. #1 Jurassic-2 Jumbo (178B) 63.871847%
2. #2 LLaMA (30B) 63.803681%
3. #3 gpt-3.5-turbo-0301 62.781186%

HELM Long Context: Measures long-context retrieval, needle finding, summarization, factual grounding, or retrieval-augmented generation quality.

HH-RLHF: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

HotpotQA: Evaluates broad language-model knowledge, reasoning, commonsense, instruction following, or exam-style accuracy.

1. #1 Beam Retrieval (single model) 77.54
2. #2 PipNet (single model) 76.95
3. #3 Smoothing R3 (single model) 76.69

HumanEval+ code-generation leaderboard from EvalPlus.

1. #1 O1 Preview (Sept 2024) 89
2. #2 O1 Mini (Sept 2024) 89
3. #3 Qwen2.5-Coder-32B-Instruct 87.20

IDE coding-agent benchmark measuring first-attempt success rate across 80 interactive software-engineering tasks.

1. #1 Claude Sonnet 4.5 87.5
2. #2 GPT 5.2 85
3. #3 Claude Opus 4.5 83.75

Live provider-level inference latency, throughput, cost, and reliability tracker for hosted language-model APIs.

1. #1 Claude Claude Sonnet 4 22.1 tokens/s
2. #2 DeepSeek deepseek-chat 11.88 tokens/s
3. #3 OpenAI GPT-4o 10.26 tokens/s

InfiniteBench: Measures long-context retrieval, needle finding, summarization, factual grounding, or retrieval-augmented generation quality.

1. #1 GPT-4 46.099167%
2. #2 Claude 2 37.843333%
3. #3 Kimi-Chat 35.325%

InfographicVQA: Measures visual question answering, OCR, document understanding, chart comprehension, or layout-aware reasoning.

1. #1 Human Performance 0.9718
2. #2 Seed-VL-1.5 0.912
3. #3 MiMo-VL-7B-RL 0.8806

Financial decision-making benchmark for investment agents, evaluating portfolio or trading decisions rather than only financial QA.

1. #1 Qwen2.5-72B-Instruct 46.153%
2. #2 GPT-4 43.696%
3. #3 GPT-4o 39.031%

JEEBench: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 GPT-4+CoT+SC@8 0.389
2. #5 GPT-4+ (1-shot) CoT 0.292
3. #6 GPT-3.5 0.177

K-12 education benchmark for subject knowledge, problem solving, and educational-goal cognition across school-level tasks.

1. #1 Doubao-Pro-32K 81.67
2. #2 DeepSeek-V3 79.67
3. #3 Doubao-Lite-32K 74.22

L-Eval: Measures long-context retrieval, needle finding, summarization, factual grounding, or retrieval-augmented generation quality.

1. #1 GPT4-32k (2023) 73.111667%
2. #2 Llama3-70b-Instruct 68.355%
3. #3 Llama3-8b-Instruct 58.71%

Biology research-assistant benchmark spanning literature QA, database QA, protocol understanding, figure interpretation, and related lab-research tasks.

1. #1 Claude 3.5 Sonnet 0.266667
2. #2 GPT-4o 0.233333

Chinese legal-domain benchmark covering legal knowledge memorization, understanding, and application tasks across zero-shot and one-shot settings.

1. #1 lawgpt-7b-beta1.0-hf (one-shot) 62.2879
2. #2 GPT4 (one-shot) 53.8453
3. #4 GPT-3.5-turbo-0613 (one-shot) 44.5226

LeanDojo Benchmark: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 ReProver (ours) 26.3%
2. #2 w/o retrieval 23.2%
3. #3 GPT-4 7.4%

LexGLUE: Measures legal reasoning, contract review, statute interpretation, or legal-domain QA.

1. #1 Legal-BERT 70.8%
2. #2 CaseLaw-BERT 69.7%
3. #3 Longformer 69.5%

Live medical benchmark with time-stamped real-world cases and after-cutoff scoring for measuring medical model robustness over time.

1. #1 GPT-5.2 0.3923
2. #2 GPT-5.1 0.3845
3. #3 GPT-5 0.2858

Dynamic live safety benchmark for large language models across ethics, legality, privacy, factuality, and psychological health.

1. #1 Claude-Haiku-4.5 91.43
2. #2 Claude-Sonnet-4.6 85.97
3. #3 GPT-5.2 84.72

LLaVA-Bench: Evaluates multimodal understanding across image, text, chart, diagram, or cross-modal reasoning tasks.

1. #1 Vicuna-13B-v1.3 / CLIP-L-336px 70.1
2. #2 LLaMA-2-13B-Chat / CLIP-L 67.9
3. #3 LLaMA-2-7B-Chat / CLIP-L 62.8

LLaVA-Bench in the Wild: Evaluates multimodal understanding across image, text, chart, diagram, or cross-modal reasoning tasks.

1. #1 LLaVA-1.6 / Hermes-Yi-34B / full_ft-1e 89.6
2. #2 LLaVA-1.6 / Vicuna-13B / full_ft-1e 87.3
3. #3 LLaVA-1.6 / Mistral-7B / full_ft-1e 83.2

LongBench v2: Measures long-context retrieval, needle finding, summarization, factual grounding, or retrieval-augmented generation quality.

1. #1 Gemini-2.5-Pro 63.3%
2. #2 Gemini-2.5-Flash 62.1%
3. #3 Qwen3-235B-A22B-Thinking-2507 Alibaba 60.6%

Long-document understanding benchmark for locating and reasoning over evidence across very large document collections.

1. #1 GPT-4o-2024-05-13 --> GPT-4o-24-05-13 64.5%

MATH: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 gpt-3.5-turbo-0301 48.83286%
2. #3 code-davinci-002 41.017627%
3. #4 text-davinci-003 39.009888%

Mathematical visual reasoning benchmark for VLMs, unified models, and LLMs, reporting answer correctness and process scores on text and multimodal questions.

1. #1 Qwen3-VL-235B-A22B-Thinking 66.8
2. #2 Qwen3-VL-235B-A22B-Instruct 65.0
3. #3 Gemini-2.5-Pro 64.7

Open-ended math tutoring benchmark for measuring pedagogical capabilities such as mistake diagnosis, Socratic questioning, and scaffolding.

1. #1 LearnLM-1.5-Pro 0.6633
2. #2 GPT-4o 0.6378
3. #3 LLaMA3.1-70B-Instruct 0.5522

MAWPS: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 Transformer (R) 87.1
2. #2 Transformer (S) 77.9

MBPP+ code-generation leaderboard from EvalPlus.

1. #1 O1 Preview (Sept 2024) 80.20
2. #2 O1 Mini (Sept 2024) 78.80
3. #3 Qwen2.5-Coder-32B-Instruct 77

McEval: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

1. #1 GPT-4o-240513 65.2%
2. #2 GPT-4-Turbo-231106 63.4%
3. #3 DeepSeek-Coder-33b-instruct 54.3%

Med-HALT: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 Falcon 40B 44.725%
2. #2 Falcon 40B-instruct 41.205%
3. #3 Llama-2 70B 40.185%

Interactive EHR-agent benchmark with physician-written tasks over healthcare data and FHIR-style clinical workflows.

1. #1 Claude 3.5 Sonnet v2 69.67%
2. #2 GPT-4o 64.00%
3. #3 DeepSeek-V3 62.67%

Medical browsing and search benchmark for multi-hop clinical research questions over live or web-grounded medical sources.

1. #1 perplexity_deep_research 14/48
2. #2 Gemini Pro 12/48
3. #3 Gemini Pro + tools 111/453

Clinically grounded medical evaluation suite built on HELM, covering medical reasoning, safety, fairness, robustness, and specialty-specific tasks.

1. #1 DeepSeek R1 0.6625
2. #2 o3-mini (2025-01-31) 0.6410714285714286
3. #3 Claude 3.7 Sonnet (20250219) 0.6357142857142857

Clinical LLM benchmark leaderboard spanning closed-ended medical QA, open-ended clinical tasks, medical safety, summarization, note generation, HealthBench, EHRSQL, MedCalc, MedEC, general-domain, and DischargeMe evaluations.

1. #1 google/gemini-2.5-flash-preview-04-17-thinking 92.36 average normalized public table score
2. #2 openai/gpt-4.1 91.71 average normalized public table score
3. #3 openai/o4-mini 90.5 average normalized public table score

Living medical-model benchmark spanning verifiable and open-ended clinical evaluation families with model/system comparisons.

1. #1 Gemini 3 Pro Preview (verifiable) 0.6627770031943667
2. #1 GPT-5.2 (medium) (open-ended) 0.6389159522138381
3. #2 GPT-5.1 (medium) (open-ended) 0.6243980841829406

MedQA: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 BioBERT-Large 36.7
2. #2 BioRoBERTa-Base 36.1
3. #3 IR-Custom 36.1

Medical diagnostic safety benchmark measuring harm-weighted safety pass rate, coverage, diagnostic recall, and over-escalation behavior.

1. #1 openai-gpt-5.2 97.6
2. #2 anthropic-claude-haiku-4.5 95.6
3. #3 openai-gpt-5-chat 94

MIMIC-IV-Ext-Bench: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

MineDojo: Measures embodied-agent, navigation, manipulation, or simulated robotics task success.

1. #1 MineCLIP Attn 46.4667%
2. #2 Manual Reward 44.5625%
3. #3 MineCLIP Avg 39.2083%

MiniF2F: Measures mathematical reasoning, symbolic problem solving, proof construction, or competition-style problem solving.

1. #1 Lean GPT-f 24.6%
2. #2 Lean tidy 18.0%
3. #3 Metamath GPT-f 1.3%

MLAgentBench: Evaluates autonomous agent performance on multi-step tasks requiring planning, state tracking, tool use, and recovery.

1. #1 Claude v3 Opus 37.5%
2. #2 Claude v2.1 26.0%
3. #3 GPT-4-turbo 26.0%

MLCommons client-device inference benchmark for PC, Mac, and local/client form-factor AI performance.

MLCommons benchmark for API-hosted GenAI endpoints, measuring serving latency, throughput, concurrency, and endpoint efficiency.

1. #1 System 9 / Deepseek-R1 3908.3 tokens/s
2. #2 System 1 / GPT-OSS 120B 421 tokens/s
3. #3 System 2 / GPT-OSS 120B 376.2 tokens/s

Audited MLCommons inference benchmark results for datacenter, edge, and GenAI workloads, including throughput, latency, and power-efficiency views.

1. #1 AMD 87xMI355X llama2-70b-99 1042110 Tokens/s
2. #2 AMD 87xMI355X llama2-70b-99.9 1042110 Tokens/s

MMBench: Evaluates multimodal understanding across image, text, chart, diagram, or cross-modal reasoning tasks.

1. #1 Qwen3.5-9B 90.1
2. #2 Qwen3.5-4B 89.4
3. #3 InternVL3-78B 89

MMBench-CN: Evaluates multimodal understanding across image, text, chart, diagram, or cross-modal reasoning tasks.

1. #1 InternLM-XComposer2* 77.2
2. #2 Qwen-VL-Max 75.9
3. #3 GPT-4v 73.3

Fine-grained multimodal document understanding benchmark with OCR-free VQA, grounding, and document reasoning tasks.

1. #1 GPT-4o 71.99%
2. #2 Gemini-1.5-Pro 71.61%
3. #3 Qwen-VL-Max 70.49%

MMLU Medical Genetics: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 GPT-4 (5-shot) 92.0%
2. #2 GPT-4 (zero-shot) 91.0%
3. #3 Flan-PaLM 540B (few-shot) 70.0%

MMLU Professional Medicine: Evaluates clinical, biomedical, medical-exam, coding, or healthcare-document reasoning.

1. #1 GPT-4 (5-shot) 93.75%
2. #2 GPT-4 (zero-shot) 93.01%
3. #3 Flan-PaLM 540B (few-shot) 83.8%

MobileWorld: Measures browser, desktop, mobile, or GUI agents operating in interactive environments.

1. #1 GPT-5 + UI-Ins-7B 51.7%
2. #2 Gemini-3-Pro + UI-Ins-7B 46.3%
3. #3 Claude-4.5-Sonnet + UI-Ins-7B 43.8%

MoralChoice: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

1. #1 GLM4 1
2. #2 Mixtral 1
3. #3 chatgpt 1

MSRVTT-QA: Evaluates temporal, video, speech, or audio understanding beyond static text and image inputs.