Open Italian LLM Leaderboard

Italian LLM leaderboard evaluating open models on Italian M-MMLU, Belebele, HellaSwag, LAMBADA, XCOPA, and ARC tasks.

25rows
average_accuracyprimary metric
2026-05-06sampled

Metadata

ID
open_italian_llm_leaderboard
Category
Language
Release
2024-01-17

Metrics

Average accuracy, M-MMLU-IT acc 3-shot, M-MMLU-IT acc 5-shot, M-MMLU-IT acc 0-shot, Belebele ita_Latn acc, Belebele ita_Latn acc norm, HellaSwag-IT acc, HellaSwag-IT acc norm, LAMBADA OpenAI MT IT perplexity (lower is better), LAMBADA OpenAI MT IT acc, XCOPA-IT acc, ARC-IT acc, ARC-IT acc norm

Latest Results

Snapshot mirrors the public Open Italian LLM Leaderboard CSV. The headline score is the mean of available source accuracy columns; LAMBADA perplexity is preserved as a lower-is-better secondary metric and excluded from the mean.

Rank Subject Average accuracy Model Match Provenance Sampled
1 mii-llm/maestrale-chat-v0.2-alpha 49.48 — Imported 2026-05-06
2 giux78/zefiro-7b-dpo-qlora-ITA-v0.7 49.39 — Imported 2026-05-06
3 FinancialSupport/saiga-7b 49.31 — Imported 2026-05-06
4 giux78/zefiro-7b-sft-qlora-ITA-v0.5 48.31 — Imported 2026-05-06
5 giux78/zefiro-7b-beta-ITA-v0.1 47.05 — Imported 2026-05-06
6 mii-11m/maestrale-chat-v0.3-alpha 46.01 — Imported 2026-05-06
7 galatolo/cerbero-7B 43.81 — Imported 2026-05-06
8 mii-llm/maestrale-chat-v0.3-beta 41.36 — Imported 2026-05-06
9 giux78/llama3-8B-usenet-merged 41.03 — Imported 2026-05-06
10 mistralai/Mistral-7B-v0.1 37.04 — Imported 2026-05-06
11 raicritis/Hermes7b_ITA 36.27 — Imported 2026-05-06
12 swap-uniba/LLaMAntino-2-7b-hf-ITA 31.30 — Imported 2026-05-06
13 swap-uniba/LLaMAntino-2-70b-hf-UltraChat-ITA 22.93 — Imported 2026-05-06
14 DeepMount00/Llama-3-8b-Ita 22.55 — Imported 2026-05-06
15 DeepMount00/Mistral-Ita-7b 21.93 — Imported 2026-05-06
16 MoxoffSpA/Volare 21.08 — Imported 2026-05-06
17 MoxoffSpA/Azzurro 20.09 — Imported 2026-05-06
18 swap-uniba/LLaMAntino-2-chat-13b-hf-UltraChat-ITA 18.77 — Imported 2026-05-06
19 seeweb/SeewebLLM-it 15.37 — Imported 2026-05-06
20 e-palmisano/Phi3-ITA-mini-4K-instruct 14.72 — Imported 2026-05-06
21 FairMind/Llama-3-8B-4bit-UltraChat-Ita 14.55 — Imported 2026-05-06
22 FairMind/Phi-3-mini-4k-instruct-bnb-4bit-Ita 14.22 — Imported 2026-05-06
23 DeepMount00/ITA_Foundation_LLM 12.72 — Imported 2026-05-06
24 walid-iguider/Minerva-3B-Instruct-v1.0 9.99 — Imported 2026-05-06
25 sapienzanlp/Minerva-3B-base-v1.0 9.86 — Imported 2026-05-06