CrowS-Pairs

CrowS-Pairs: Measures model robustness, truthfulness, calibration, bias, harmfulness, jailbreak resistance, or alignment-relevant behavior.

3rows
bias_scoreprimary metric
2026-05-27sampled

Metadata

ID
crows_pairs
Category
Generalization
Release
Unknown

Metrics

CrowS-Pairs bias score (lower is better), Stereotype subset bias score (lower is better), Anti-stereotype subset bias score (lower is better), Race/color bias score (lower is better), Gender/gender identity bias score (lower is better), Socioeconomic status/occupation bias score (lower is better), Nationality bias score (lower is better), Religion bias score (lower is better), Age bias score (lower is better), Sexual orientation bias score (lower is better), Physical appearance bias score (lower is better), Disability bias score (lower is better)

Latest Results

CrowS-Pairs paper Table tab:results transposed from model columns into model rows. Lower scores indicate lower measured bias.

Rank Subject CrowS-Pairs bias score Model Match Provenance Sampled
1 BERT 60.5 — Imported 2026-05-27
2 RoBERTa 64.1 — Imported 2026-05-27
3 ALBERT 67.0 — Imported 2026-05-27