Edge LLM Leaderboard: Raspberry Pi 5
NYU NAIRR edge LLM leaderboard measuring local LLM variants on Raspberry Pi 5 (8GB), combining MMLU accuracy with prefill/decode throughput, model size, quantization, and backend metadata.
128rows
mmlu_accuracyprimary metric
2026-05-06sampled
Metadata
Metrics
MMLU Accuracy, Prefill, Decode, Model Size (lower is better), Parameters (lower is better)
| Rank | Subject | MMLU Accuracy | Model Match | Provenance | Sampled |
|---|---|---|---|---|---|
| 1 | Mistral-7B-Instruct-v0.3 (Q8_0, llama_cpp) | 43.20 | — | Imported | 2026-05-06 |
| 2 | Mistral-7B-Instruct-v0.3 (Q4_0_4_4, llama_cpp) | 42.90 | — | Imported | 2026-05-06 |
| 3 | Mistral-7B-Instruct-v0.3 (Q4_K_M, llama_cpp) | 42.90 | — | Imported | 2026-05-06 |
| 4 | Phi-3-medium-128k-instruct (Q4_K_M, llama_cpp) | 42.70 | — | Imported | 2026-05-06 |
| 5 | Qwen2.5-14B (Q4_K_M, llama_cpp) | 42.50 | — | Imported | 2026-05-06 |
| 6 | gemma-2-9b (Q4_0_4_4, llama_cpp) | 42.40 | — | Imported | 2026-05-06 |
| 7 | gemma-2-9b (Q8_0, llama_cpp) | 42.40 | — | Imported | 2026-05-06 |
| 8 | Phi-3-medium-128k-instruct (Q4_0_4_4, llama_cpp) | 42.10 | — | Imported | 2026-05-06 |
| 9 | Qwen2.5-14B (Q4_0_4_4, llama_cpp) | 42.10 | — | Imported | 2026-05-06 |
| 10 | Mistral-Nemo-Base-2407 (Q4_0_4_4, llama_cpp) | 41.90 | — | Imported | 2026-05-06 |
| 11 | gemma-2-9b (Q4_K_M, llama_cpp) | 41.80 | — | Imported | 2026-05-06 |
| 12 | Hermes-3-Llama-3.1-8B (Q8_0, llama_cpp) | 41.80 | — | Imported | 2026-05-06 |
| 13 | Phi-3.5-mini-instruct (Q8_0, llama_cpp) | 41.80 | — | Imported | 2026-05-06 |
| 14 | Yi-1.5-9B (Q8_0, llama_cpp) | 41.80 | — | Imported | 2026-05-06 |
| 15 | internlm2_5-7b-chat (Q4_0_4_4, llama_cpp) | 41.70 | — | Imported | 2026-05-06 |
| 16 | internlm2_5-7b-chat (Q8_0, llama_cpp) | 41.70 | — | Imported | 2026-05-06 |
| 17 | aya-expanse-8b (Q4_K_M, llama_cpp) | 41.60 | — | Imported | 2026-05-06 |
| 18 | aya-expanse-8b (Q4_0_4_4, llama_cpp) | 41.50 | — | Imported | 2026-05-06 |
| 19 | aya-expanse-8b (Q8_0, llama_cpp) | 41.40 | — | Imported | 2026-05-06 |
| 20 | Phi-3-mini-128k-instruct (Q8_0, llama_cpp) | 41.40 | — | Imported | 2026-05-06 |
| 21 | Phi-3.5-mini-instruct (Q4_0_4_4, llama_cpp) | 41.40 | — | Imported | 2026-05-06 |
| 22 | internlm2_5-7b-chat (Q4_K_M, llama_cpp) | 41.30 | — | Imported | 2026-05-06 |
| 23 | Starling-LM-7B-beta (Q4_0_4_4, llama_cpp) | 41.30 | — | Imported | 2026-05-06 |
| 24 | Starling-LM-7B-beta (Q8_0, llama_cpp) | 41.30 | — | Imported | 2026-05-06 |
| 25 | Hermes-3-Llama-3.1-8B (Q4_K_M, llama_cpp) | 41.20 | — | Imported | 2026-05-06 |
| 26 | Mistral-Nemo-Base-2407 (Q4_K_M, llama_cpp) | 41.20 | — | Imported | 2026-05-06 |
| 27 | Yi-1.5-9B (Q4_K_M, llama_cpp) | 41.20 | — | Imported | 2026-05-06 |
| 28 | Phi-3.5-mini-instruct (Q4_K_M, llama_cpp) | 41.10 | — | Imported | 2026-05-06 |
| 29 | Starling-LM-7B-beta (Q4_K_M, llama_cpp) | 41 | — | Imported | 2026-05-06 |
| 30 | Phi-3-mini-128k-instruct (Q4_K_M, llama_cpp) | 40.90 | — | Imported | 2026-05-06 |
| 31 | Hermes-3-Llama-3.1-8B (Q4_0_4_4, llama_cpp) | 40.70 | — | Imported | 2026-05-06 |
| 32 | SOLAR-10.7B-Instruct-v1.0 (Q4_0_4_4, llama_cpp) | 40.70 | — | Imported | 2026-05-06 |
| 33 | Phi-3-mini-128k-instruct (Q4_0_4_4, llama_cpp) | 40.60 | — | Imported | 2026-05-06 |
| 34 | Llama-3.1-8B (Q4_K_M, llama_cpp) | 40.50 | — | Imported | 2026-05-06 |
| 35 | Yi-1.5-9B (Q4_0_4_4, llama_cpp) | 40.50 | — | Imported | 2026-05-06 |
| 36 | dolphin-2.9.3-mistral-7B-32k (Q8_0, llama_cpp) | 40.40 | — | Imported | 2026-05-06 |
| 37 | Qwen2.5-7B (Q8_0, llama_cpp) | 40.40 | — | Imported | 2026-05-06 |
| 38 | SOLAR-10.7B-Instruct-v1.0 (Q4_K_M, llama_cpp) | 40.40 | — | Imported | 2026-05-06 |
| 39 | dolphin-2.9.3-mistral-7B-32k (Q4_0_4_4, llama_cpp) | 40.30 | — | Imported | 2026-05-06 |
| 40 | Yarn-Mistral-7b-128k (Q4_0_4_4, llama_cpp) | 40.30 | — | Imported | 2026-05-06 |
| 41 | Qwen2.5-7B (Q4_K_M, llama_cpp) | 40.20 | — | Imported | 2026-05-06 |
| 42 | Yarn-Mistral-7b-128k (Q8_0, llama_cpp) | 40.20 | — | Imported | 2026-05-06 |
| 43 | Qwen2.5-7B (Q4_0_4_4, llama_cpp) | 40.10 | — | Imported | 2026-05-06 |
| 44 | Yarn-Mistral-7b-128k (Q4_K_M, llama_cpp) | 40.10 | — | Imported | 2026-05-06 |
| 45 | Yi-1.5-6B (Q8_0, llama_cpp) | 39.90 | — | Imported | 2026-05-06 |
| 46 | dolphin-2.9.3-mistral-7B-32k (Q4_K_M, llama_cpp) | 39.70 | — | Imported | 2026-05-06 |
| 47 | Llama-3.1-8B (Q4_0_4_4, llama_cpp) | 39.70 | — | Imported | 2026-05-06 |
| 48 | SOLAR-10.7B-v1.0 (Q4_0_4_4, llama_cpp) | 39.50 | — | Imported | 2026-05-06 |
| 49 | SOLAR-10.7B-v1.0 (Q4_K_M, llama_cpp) | 39.40 | — | Imported | 2026-05-06 |
| 50 | Yarn-Solar-10b-64k (Q4_0_4_4, llama_cpp) | 39.40 | — | Imported | 2026-05-06 |
| 51 | Yi-1.5-6B (Q4_K_M, llama_cpp) | 39.30 | — | Imported | 2026-05-06 |
| 52 | Yi-1.5-6B (Q4_0_4_4, llama_cpp) | 39.20 | — | Imported | 2026-05-06 |
| 53 | DeepSeek-V2-Lite (Q4_K_M, llama_cpp) | 38.90 | — | Imported | 2026-05-06 |
| 54 | Yarn-Solar-10b-64k (Q4_K_M, llama_cpp) | 38.80 | — | Imported | 2026-05-06 |
| 55 | DeepSeek-V2-Lite (Q4_0_4_4, llama_cpp) | 38.60 | — | Imported | 2026-05-06 |
| 56 | Qwen2.5-3B (Q8_0, llama_cpp) | 38.60 | — | Imported | 2026-05-06 |
| 57 | dolphin-2.9.2-qwen2-7b (Q8_0, llama_cpp) | 38.50 | — | Imported | 2026-05-06 |
| 58 | aya-23-8B (Q4_0_4_4, llama_cpp) | 38.30 | — | Imported | 2026-05-06 |
| 59 | aya-23-8B (Q8_0, llama_cpp) | 38.30 | — | Imported | 2026-05-06 |
| 60 | OLMoE-1B-7B-0924 (Q4_K_M, llama_cpp) | 38.30 | — | Imported | 2026-05-06 |
| 61 | OLMoE-1B-7B-0924 (Q8_0, llama_cpp) | 38.30 | — | Imported | 2026-05-06 |
| 62 | Qwen2.5-3B (Q4_K_M, llama_cpp) | 38.20 | — | Imported | 2026-05-06 |
| 63 | dolphin-2.9.2-qwen2-7b (Q4_K_M, llama_cpp) | 38.10 | — | Imported | 2026-05-06 |
| 64 | OLMoE-1B-7B-0924 (Q4_0_4_4, llama_cpp) | 38 | — | Imported | 2026-05-06 |
| 65 | Qwen2.5-3B (Q4_0_4_4, llama_cpp) | 37.90 | — | Imported | 2026-05-06 |
| 66 | aya-23-8B (Q4_K_M, llama_cpp) | 37.80 | — | Imported | 2026-05-06 |
| 67 | dolphin-2.9.2-qwen2-7b (Q4_0_4_4, llama_cpp) | 37.70 | — | Imported | 2026-05-06 |
| 68 | Llama-3.2-3B (Q8_0, llama_cpp) | 37.50 | — | Imported | 2026-05-06 |
| 69 | gemma-2-2b (Q4_0_4_4, llama_cpp) | 37.40 | — | Imported | 2026-05-06 |
| 70 | dolphin-2.9.4-gemma2-2b (Q4_K_M, llama_cpp) | 37.30 | — | Imported | 2026-05-06 |
| 71 | gemma-2-2b (Q8_0, llama_cpp) | 37.30 | — | Imported | 2026-05-06 |
| 72 | Llama-3.2-3B (Q4_K_M, llama_cpp) | 37.30 | — | Imported | 2026-05-06 |
| 73 | dolphin-2.9.4-gemma2-2b (Q8_0, llama_cpp) | 37.20 | — | Imported | 2026-05-06 |
| 74 | dolphin-2.9.4-gemma2-2b (Q4_0_4_4, llama_cpp) | 37.10 | — | Imported | 2026-05-06 |
| 75 | gemma-2-2b (Q4_K_M, llama_cpp) | 37 | — | Imported | 2026-05-06 |
| 76 | Llama-3.2-3B (Q4_0_4_4, llama_cpp) | 36.80 | — | Imported | 2026-05-06 |
| 77 | OLMo-7B-0724-hf (Q4_K_M, llama_cpp) | 36.20 | — | Imported | 2026-05-06 |
| 78 | OLMo-7B-0724-hf (Q8_0, llama_cpp) | 36.20 | — | Imported | 2026-05-06 |
| 79 | SmolLM2-1.7B-Instruct (Q8_0, llama_cpp) | 36 | — | Imported | 2026-05-06 |
| 80 | Qwen2.5-1.5B (Q4_K_M, llama_cpp) | 35.80 | — | Imported | 2026-05-06 |
| 81 | Qwen2.5-1.5B (Q8_0, llama_cpp) | 35.80 | — | Imported | 2026-05-06 |
| 82 | SmolLM2-1.7B-Instruct (Q4_K_M, llama_cpp) | 35.40 | — | Imported | 2026-05-06 |
| 83 | mpt-7b-instruct (Q4_K_M, llama_cpp) | 35.30 | — | Imported | 2026-05-06 |
| 84 | mpt-7b-instruct (Q8_0, llama_cpp) | 35.30 | — | Imported | 2026-05-06 |
| 85 | Qwen2.5-1.5B (Q4_0_4_4, llama_cpp) | 35.10 | — | Imported | 2026-05-06 |
| 86 | SmolLM2-1.7B-Instruct (Q4_0_4_4, llama_cpp) | 35.10 | — | Imported | 2026-05-06 |
| 87 | mpt-7b-instruct (Q4_0_4_4, llama_cpp) | 34.90 | — | Imported | 2026-05-06 |
| 88 | internlm2_5-1_8b-chat (Q8_0, llama_cpp) | 34 | — | Imported | 2026-05-06 |
| 89 | Llama-3.2-1B (Q8_0, llama_cpp) | 34 | — | Imported | 2026-05-06 |
| 90 | internlm2_5-1_8b-chat (Q4_K_M, llama_cpp) | 33.70 | — | Imported | 2026-05-06 |
| 91 | Llama-3.2-1B (Q4_K_M, llama_cpp) | 33.50 | — | Imported | 2026-05-06 |
| 92 | Llama-3.2-1B (Q4_0_4_4, llama_cpp) | 33.40 | — | Imported | 2026-05-06 |
| 93 | Amber (Q4_0_4_4, llama_cpp) | 33.10 | — | Imported | 2026-05-06 |
| 94 | Amber (Q8_0, llama_cpp) | 33.10 | — | Imported | 2026-05-06 |
| 95 | NexusRaven-V2-13B (Q4_0_4_4, llama_cpp) | 33 | — | Imported | 2026-05-06 |
| 96 | NexusRaven-V2-13B (Q4_K_M, llama_cpp) | 32.90 | — | Imported | 2026-05-06 |
| 97 | Amber (Q4_K_M, llama_cpp) | 32.70 | — | Imported | 2026-05-06 |
| 98 | SmolLM2-360M-Instruct (Q8_0, llama_cpp) | 32.30 | — | Imported | 2026-05-06 |
| 99 | internlm2_5-1_8b-chat (Q4_0_4_4, llama_cpp) | 32.20 | — | Imported | 2026-05-06 |
| 100 | SmolLM2-360M-Instruct (Q4_0_4_4, llama_cpp) | 32 | — | Imported | 2026-05-06 |
| 101 | OLMo-1B-0724-hf (Q4_K_M, llama_cpp) | 31.90 | — | Imported | 2026-05-06 |
| 102 | Qwen2.5-0.5B (Q8_0, llama_cpp) | 31.90 | — | Imported | 2026-05-06 |
| 103 | SmolLM2-360M-Instruct (Q4_K_M, llama_cpp) | 31.90 | — | Imported | 2026-05-06 |
| 104 | OLMo-1B-0724-hf (Q8_0, llama_cpp) | 31.70 | — | Imported | 2026-05-06 |
| 105 | OLMo-1B-0724-hf (Q4_0_4_4, llama_cpp) | 31.50 | — | Imported | 2026-05-06 |
| 106 | Qwen2.5-0.5B (Q4_K_M, llama_cpp) | 31.40 | — | Imported | 2026-05-06 |
| 107 | Qwen2.5-0.5B (Q4_0_4_4, llama_cpp) | 31.30 | — | Imported | 2026-05-06 |
| 108 | TinyLlama-1.1B-Chat-v1.0 (Q4_0_4_4, llama_cpp) | 30.90 | — | Imported | 2026-05-06 |
| 109 | TinyLlama-1.1B-Chat-v1.0 (Q8_0, llama_cpp) | 30.90 | — | Imported | 2026-05-06 |
| 110 | TinyLlama-1.1B-Chat-v1.0 (Q4_K_M, llama_cpp) | 30.70 | — | Imported | 2026-05-06 |
| 111 | SmolLM2-135M-Instruct (Q8_0, llama_cpp) | 29.90 | — | Imported | 2026-05-06 |
| 112 | stable-code-instruct-3b (Q8_0, llama_cpp) | 29.90 | — | Imported | 2026-05-06 |
| 113 | SmolLM2-135M-Instruct (Q4_0_4_4, llama_cpp) | 29.80 | — | Imported | 2026-05-06 |
| 114 | stable-code-instruct-3b (Q4_0_4_4, llama_cpp) | 29.80 | — | Imported | 2026-05-06 |
| 115 | stable-code-instruct-3b (Q4_K_M, llama_cpp) | 29.80 | — | Imported | 2026-05-06 |
| 116 | SmolLM2-135M-Instruct (Q4_K_M, llama_cpp) | 29.50 | — | Imported | 2026-05-06 |
| 117 | Yi-Coder-1.5B (Q4_K_M, llama_cpp) | 29.40 | — | Imported | 2026-05-06 |
| 118 | Yi-Coder-1.5B (Q8_0, llama_cpp) | 29.30 | — | Imported | 2026-05-06 |
| 119 | Yi-Coder-1.5B (Q4_0_4_4, llama_cpp) | 29.20 | — | Imported | 2026-05-06 |
| 120 | gpt2-medium (Q8_0, llama_cpp) | 29 | — | Imported | 2026-05-06 |
| 121 | gpt2-medium (Q4_0_4_4, llama_cpp) | 28.80 | — | Imported | 2026-05-06 |
| 122 | gpt2-medium (Q4_K_M, llama_cpp) | 28.80 | — | Imported | 2026-05-06 |
| 123 | TinyLlama_v1.1 (Q4_0_4_4, llama_cpp) | 28.30 | — | Imported | 2026-05-06 |
| 124 | TinyLlama_v1.1 (Q8_0, llama_cpp) | 28.20 | — | Imported | 2026-05-06 |
| 125 | TinyLlama_v1.1 (Q4_K_M, llama_cpp) | 28 | — | Imported | 2026-05-06 |
| 126 | gpt2 (Q4_0_4_4, llama_cpp) | 27.90 | — | Imported | 2026-05-06 |
| 127 | gpt2 (Q4_K_M, llama_cpp) | 27.60 | — | Imported | 2026-05-06 |
| 128 | gpt2 (Q8_0, llama_cpp) | 27.20 | — | Imported | 2026-05-06 |
No matching rows.