Multi-IF
Multi-IF benchmarks LLMs on multi-turn and multilingual instruction following. It expands upon IFEval by incorporating multi-turn sequences and translating English prompts into 7 other languages, resulting in 4,501 multilingual conversations with three turns each. The benchmark reveals that current leading LLMs struggle with maintaining accuracy in multi-turn instructions and shows higher error rates for non-Latin script languages.
20rows
scoreprimary metric
2026-05-06sampled
Metadata
Metrics
Score, Normalized Score
| Rank | Subject | Score | Model Match | Provenance | Sampled |
|---|---|---|---|---|---|
| 1 | Qwen3-235B-A22B-Thinking-2507 | 0.81 |  Qwen3 235B A22B Thinking 2507 qwen-qwen3-235b-a22b-thinking-2507 | Self-reported | 2026-05-06 |
| 2 | o3-mini | 0.80 |  o3-mini openai-o3-mini | Self-reported | 2026-05-06 |
| 3 | Qwen3 VL 235B A22B Thinking | 0.79 | Qwen3 VL 235B A22B Thinking qwen-qwen3-vl-235b-a22b-thinking | Self-reported | 2026-05-06 |
| 4 | Qwen3 VL 32B Thinking | 0.78 | — | Self-reported | 2026-05-06 |
| 5 | Qwen3-Next-80B-A3B-Thinking | 0.78 | Qwen3 Next 80B A3B Thinking qwen-qwen3-next-80b-a3b-thinking | Self-reported | 2026-05-06 |
| 6 | Qwen3-235B-A22B-Instruct-2507 | 0.78 | Qwen3 235B A22B Instruct 2507 qwen-qwen3-235b-a22b-2507 | Self-reported | 2026-05-06 |
| 7 | Qwen3 VL 235B A22B Instruct | 0.76 | Qwen3 VL 235B A22B Instruct qwen-qwen3-vl-235b-a22b-instruct | Self-reported | 2026-05-06 |
| 8 | Qwen3-Next-80B-A3B-Instruct | 0.76 | Qwen3 Next 80B A3B Instruct qwen-qwen3-next-80b-a3b-instruct | Self-reported | 2026-05-06 |
| 9 | Qwen3 VL 8B Thinking | 0.75 | Qwen3 VL 8B Thinking qwen-qwen3-vl-8b-thinking | Self-reported | 2026-05-06 |
| 9 | Qwen3 VL 8B Instruct | 0.75 | Qwen3 VL 8B Instruct qwen-qwen3-vl-8b-instruct | Self-reported | 2026-05-06 |
| 11 | Qwen3 VL 4B Thinking | 0.74 | — | Self-reported | 2026-05-06 |
| 12 | Qwen3 VL 30B A3B Thinking | 0.73 | Qwen3 VL 30B A3B Thinking qwen-qwen3-vl-30b-a3b-thinking | Self-reported | 2026-05-06 |
| 13 | Qwen3 30B A3B | 0.72 | Qwen3 30B A3B qwen-qwen3-30b-a3b | Self-reported | 2026-05-06 |
| 14 | Qwen3 VL 32B Instruct | 0.72 | Qwen3 VL 32B Instruct qwen-qwen3-vl-32b-instruct | Self-reported | 2026-05-06 |
| 15 | GPT-4.1 | 0.71 | GPT-4.1 openai-gpt-4.1 | Self-reported | 2026-05-06 |
| 15 | GPT-4.5 | 0.71 | GPT-4.5 openai-gpt-4.5-preview | Self-reported | 2026-05-06 |
| 17 | GPT-4.1 mini | 0.67 | GPT-4.1 Mini openai-gpt-4.1-mini | Self-reported | 2026-05-06 |
| 18 | Qwen3 VL 30B A3B Instruct | 0.66 | Qwen3 VL 30B A3B Instruct qwen-qwen3-vl-30b-a3b-instruct | Self-reported | 2026-05-06 |
| 19 | GPT-4o | 0.61 | GPT-4o (2024-08-06) openai-gpt-4o-2024-08-06 | Self-reported | 2026-05-06 |
| 20 | GPT-4.1 nano | 0.57 | GPT-4.1 Nano openai-gpt-4.1-nano | Self-reported | 2026-05-06 |
No matching rows.