CodeContests

CodeContests: Measures model capability on programming, code generation, code repair, or repository-level software tasks.

3rows
test_10_at_100k_solve_rateprimary metric
2026-05-27sampled

Metadata

ID
codecontests
Category
Coding
Release
2022-02-08

Metrics

Test 10@100k solve rate, Validation 10@1M solve rate, Test 10@1k solve rate, Test 10@10k solve rate

Latest Results

Rows are transcribed from the public AlphaCode paper Table 5. Primary score is CodeContests test-set 10@100k solve rate.

Rank Subject Test 10@100k solve rate Model Match Provenance Sampled
1 AlphaCode 41B + clustering 29.6% — Imported 2026-05-27
2 AlphaCode 41B 27.7% — Imported 2026-05-27
3 AlphaCode 9B 25.8% — Imported 2026-05-27