ProteinGym

Protein fitness prediction benchmark comparing zero-shot models on DMS substitution assays with aggregate Spearman correlations and drill-downs by function, MSA depth, taxa, and mutation depth.

97rows
average_spearmanprimary metric
2026-05-28sampled

Metadata

ID
proteingym
Category
Biology
Release
Unknown

Metrics

Average Spearman, Bootstrap Standard Error of Difference to Best Spearman (lower is better), Function Activity Spearman, Function Binding Spearman, Function Expression Spearman, Function Organismal Fitness Spearman, Function Stability Spearman, Low MSA Depth Spearman, Medium MSA Depth Spearman, High MSA Depth Spearman, Human Taxa Spearman, Other Eukaryote Taxa Spearman, Prokaryote Taxa Spearman, Virus Taxa Spearman, Mutation Depth 1 Spearman, Mutation Depth 2 Spearman, Mutation Depth 3 Spearman, Mutation Depth 4 Spearman, Mutation Depth 5+ Spearman

Latest Results

Rows are imported from the official ProteinGym DMS zero-shot substitutions Spearman summary CSV and ranked by average Spearman correlation.

Rank Subject Average Spearman Model Match Provenance Sampled
1 AIDO Protein-RAG (16B) 0.518 average Spearman — Imported 2026-05-28
2 VenusREM 0.518 average Spearman — Imported 2026-05-28
3 ProSST (K=2048) 0.507 average Spearman — Imported 2026-05-28
4 ProSST (K=4096) 0.498 average Spearman — Imported 2026-05-28
5 S3F-MSA 0.496 average Spearman — Imported 2026-05-28
6 S2F-MSA 0.488 average Spearman — Imported 2026-05-28
7 ProSST (K=1024) 0.485 average Spearman — Imported 2026-05-28
8 Protriever 0.479 average Spearman — Imported 2026-05-28
9 ESCOTT 0.476 average Spearman — Imported 2026-05-28
10 ProSST (K=512) 0.471 average Spearman — Imported 2026-05-28
11 S3F 0.470 average Spearman — Imported 2026-05-28
12 PoET (200M) 0.470 average Spearman — Imported 2026-05-28
13 ProSST (K=128) 0.469 average Spearman — Imported 2026-05-28
14 ESM3 open (1.4B) 0.466 average Spearman — Imported 2026-05-28
15 RSALOR 0.465 average Spearman — Imported 2026-05-28
16 VespaG 0.458 average Spearman — Imported 2026-05-28
17 SaProt (650M) 0.457 average Spearman — Imported 2026-05-28
18 TranceptEVE L 0.456 average Spearman — Imported 2026-05-28
19 S2F 0.456 average Spearman — Imported 2026-05-28
20 GEMME 0.455 average Spearman — Imported 2026-05-28
21 TranceptEVE M 0.455 average Spearman — Imported 2026-05-28
22 TranceptEVE S 0.452 average Spearman — Imported 2026-05-28
23 ProtSSN (ensemble) 0.449 average Spearman — Imported 2026-05-28
24 ProtSSN (k=20, h=1280) 0.442 average Spearman — Imported 2026-05-28
25 ProtSSN (k=20, h=512) 0.440 average Spearman — Imported 2026-05-28
26 EVE (ensemble) 0.439 average Spearman — Imported 2026-05-28
27 ProtSSN (k=20, h=768) 0.439 average Spearman — Imported 2026-05-28
28 ProSST (K=20) 0.438 average Spearman — Imported 2026-05-28
29 ProtSSN (k=30, h=512) 0.438 average Spearman — Imported 2026-05-28
30 ProtSSN (k=30, h=768) 0.438 average Spearman — Imported 2026-05-28
31 ProtSSN (k=30, h=1280) 0.437 average Spearman — Imported 2026-05-28
32 VESPA 0.435 average Spearman — Imported 2026-05-28
33 Tranception L 0.434 average Spearman — Imported 2026-05-28
34 ProtSSN (k=10, h=1280) 0.433 average Spearman — Imported 2026-05-28
35 EVE (single) 0.432 average Spearman — Imported 2026-05-28
36 MSA Transformer (ensemble) 0.432 average Spearman — Imported 2026-05-28
37 ProtSSN (k=10, h=512) 0.430 average Spearman — Imported 2026-05-28
38 Tranception M 0.428 average Spearman — Imported 2026-05-28
39 SiteRM 0.424 average Spearman — Imported 2026-05-28
40 ESM-IF1 0.422 average Spearman — Imported 2026-05-28
41 ProtSSN (k=10, h=768) 0.422 average Spearman — Imported 2026-05-28
42 DeepSequence (ensemble) 0.419 average Spearman — Imported 2026-05-28
43 Tranception S 0.418 average Spearman — Imported 2026-05-28
44 MSA Transformer (single) 0.417 average Spearman — Imported 2026-05-28
45 ESM2 (650M) 0.414 average Spearman — Imported 2026-05-28
46 DeepSequence (single) 0.407 average Spearman — Imported 2026-05-28
47 ESM-1v (ensemble) 0.407 average Spearman — Imported 2026-05-28
48 SaProt (35M) 0.407 average Spearman — Imported 2026-05-28
49 ESM-C (300M) 0.406 average Spearman — Imported 2026-05-28
50 ESM2 (3B) 0.406 average Spearman — Imported 2026-05-28
51 ESM-C (600M) 0.405 average Spearman — Imported 2026-05-28
52 ESM2 (15B) 0.400 average Spearman — Imported 2026-05-28
53 MIF-ST 0.400 average Spearman — Imported 2026-05-28
54 xTrimoPGLM-3B-MLM 0.399 average Spearman — Imported 2026-05-28
55 xTrimoPGLM-10B-MLM 0.396 average Spearman — Imported 2026-05-28
56 EVmutation 0.395 average Spearman — Imported 2026-05-28
57 ESM-1b 0.394 average Spearman — Imported 2026-05-28
58 VESPAl 0.393 average Spearman — Imported 2026-05-28
59 Progen3 3B 0.392 average Spearman — Imported 2026-05-28
60 Progen2 XL 0.391 average Spearman — Imported 2026-05-28
61 Progen3 1B 0.389 average Spearman — Imported 2026-05-28
62 ESM2 (150M) 0.387 average Spearman — Imported 2026-05-28
63 MIF 0.383 average Spearman — Imported 2026-05-28
64 Progen2 L 0.380 average Spearman — Imported 2026-05-28
65 Progen2 M 0.380 average Spearman — Imported 2026-05-28
66 xTrimoPGLM-1B-MLM 0.379 average Spearman — Imported 2026-05-28
67 Progen2 Base 0.378 average Spearman — Imported 2026-05-28
68 Tranception L no retrieval 0.374 average Spearman — Imported 2026-05-28
69 ESM-1v (single) 0.374 average Spearman — Imported 2026-05-28
70 Wavenet 0.373 average Spearman — Imported 2026-05-28
71 RITA XL 0.373 average Spearman — Imported 2026-05-28
72 CARP (640M) 0.369 average Spearman — Imported 2026-05-28
73 RITA L 0.366 average Spearman — Imported 2026-05-28
74 xTrimoPGLM-100B-int4 0.364 average Spearman — Imported 2026-05-28
75 Progen3 762m 0.364 average Spearman — Imported 2026-05-28
76 Site-Independent 0.359 average Spearman — Imported 2026-05-28
77 RITA M 0.350 average Spearman — Imported 2026-05-28
78 Tranception M no retrieval 0.349 average Spearman — Imported 2026-05-28
79 Unirep evotuned 0.347 average Spearman — Imported 2026-05-28
80 Progen3 339m 0.347 average Spearman — Imported 2026-05-28
81 xTrimoPGLM-1B-CLM 0.343 average Spearman — Imported 2026-05-28
82 Progen2 S 0.336 average Spearman — Imported 2026-05-28
83 CARP (76M) 0.328 average Spearman — Imported 2026-05-28
84 ESM2 (35M) 0.321 average Spearman — Imported 2026-05-28
85 Progen3 219m 0.318 average Spearman — Imported 2026-05-28
86 MULAN 0.317 average Spearman — Imported 2026-05-28
87 RITA S 0.305 average Spearman — Imported 2026-05-28
88 Tranception S no retrieval 0.303 average Spearman — Imported 2026-05-28
89 Progen3 112m 0.282 average Spearman — Imported 2026-05-28
90 CARP (38M) 0.280 average Spearman — Imported 2026-05-28
91 ProteinMPNN 0.257 average Spearman — Imported 2026-05-28
92 xTrimoPGLM-3B-CLM 0.233 average Spearman — Imported 2026-05-28
93 ESM2 (8M) 0.226 average Spearman — Imported 2026-05-28
94 Unirep 0.190 average Spearman — Imported 2026-05-28
95 ProtGPT2 0.188 average Spearman — Imported 2026-05-28
96 CARP (600K) 0.106 average Spearman — Imported 2026-05-28
97 xTrimoPGLM-7B-CLM 0.102 average Spearman — Imported 2026-05-28