JMMMU-Pro

Image-based Japanese Multi-discipline Multimodal Understanding Benchmark extending JMMMU by composing question image and text into a single image for integrated visual-textual understanding.

26rows
overallprimary metric
2026-05-06sampled

Metadata

ID
jmmmu_pro
Category
Multimodal
Release
2025-12-16

Metrics

Overall, culture-specific, culture-agnostic, Japanese Art, Japanese Heritage, Japanese History, World History, Accounting, Agriculture, Architecture and Engineering, Basic Medical Science, Biology, Chemistry, Clinical Medicine, Computer Science, Design, Diagnostics and Laboratory Medicine, Economics, Electronics, Energy and Power, Finance, Manage, Marketing, Materials, Math, Mechanical Engineering, Music, Pharmacy, Physics, Psychology, Public Health

Latest Results

Rows are parsed from the public JMMMU-Pro leaderboard CSV. Source model and prompt display names are preserved; direct and CoT prompt rows remain separate source configurations.

Rank Subject Overall Model Match Provenance Sampled
1 Gemini3Pro 87.05 Gemini 3
google-gemini-3
 		Imported 2026-05-06
2 GPT-5.2 83.33 GPT-5.2
openai-gpt-5.2
 		Imported 2026-05-06
3 Qwen3-VL-8B 47.27 — Imported 2026-05-06
4 Qwen3-VL-8B 45.83 — Imported 2026-05-06
5 Qwen2.5-VL-7B 45 — Imported 2026-05-06
6 Qwen2.5-VL-7B 44.70 — Imported 2026-05-06
7 Sarashina2.2-V-3B 42.88 — Imported 2026-05-06
8 Sarashina2.2-V-3B 38.03 — Imported 2026-05-06
9 LLaVA-OV-1.5-8B 31.97 — Imported 2026-05-06
10 Phi-4-multimodal 31.82 — Imported 2026-05-06
11 InternVL2.5-8B 31.21 — Imported 2026-05-06
12 Sarashina2-V-14B 30.68 — Imported 2026-05-06
13 Sarashina2-V-14B 30 — Imported 2026-05-06
14 LLaVA-OV-1.5-8B 29.92 — Imported 2026-05-06
15 Sarashina2-V-8B 27.88 — Imported 2026-05-06
16 LLaVA-OV-7B 27.35 — Imported 2026-05-06
17 Sarashina2-V-8B 27.27 — Imported 2026-05-06
18 Heron-NVILA-Lite-15B 26.97 — Imported 2026-05-06
19 Aya-Vision-8B 26.74 — Imported 2026-05-06
20 InternVL2.5-8B 25.08 — Imported 2026-05-06
21 Phi-4-multimodal 24.17 — Imported 2026-05-06
22 Pangea-7B 23.41 — Imported 2026-05-06
23 Aya-Vision-8B 22.42 — Imported 2026-05-06
24 Pangea-7B 19.55 — Imported 2026-05-06
25 LLaVA-OV-7B 14.09 — Imported 2026-05-06
26 Heron-NVILA-Lite-15B 5.30 — Imported 2026-05-06