LawBench

Chinese legal-domain benchmark covering legal knowledge memorization, understanding, and application tasks across zero-shot and one-shot settings.

102rows
average_scoreprimary metric
2026-05-27sampled

Metadata

ID
lawbench
Category
Legal
Release
2023-09-28

Metrics

Average Score, Average Abstention Rate (lower is better), Task Count

Latest Results

Rows are aggregated by model and prompt setting from the public LawBench prediction result CSVs. Scores are the mean task score multiplied by 100.

Rank Subject Average Score Model Match Provenance Sampled
1 lawgpt-7b-beta1.0-hf (one-shot) 62.2879 — Imported 2026-05-27
2 GPT4 (one-shot) 53.8453 GPT-4
openai-gpt-4
 		Imported 2026-05-27
3 GPT4 (zero-shot) 52.3521 GPT-4
openai-gpt-4
 		Imported 2026-05-27
4 GPT-3.5-turbo-0613 (one-shot) 44.5226 GPT-3.5 Turbo
openai-gpt-3.5-turbo
 		Imported 2026-05-27
5 GPT-3.5-turbo-0613 (zero-shot) 42.1477 GPT-3.5 Turbo
openai-gpt-3.5-turbo
 		Imported 2026-05-27
6 freewilly2_70b-hf (zero-shot) 39.2275 — Imported 2026-05-27
7 qwen-7b-chat-hf (one-shot) 38.9918 — Imported 2026-05-27
8 freewilly2_70b-hf (one-shot) 38.9688 — Imported 2026-05-27
9 internlm-chat-7b-8k-hf (one-shot) 37.2783 — Imported 2026-05-27
10 qwen-7b-chat-hf (zero-shot) 36.998 — Imported 2026-05-27
11 internlm-chat-7b-hf (one-shot) 36.1131 — Imported 2026-05-27
12 internlm-chat-7b-8k-hf (zero-shot) 35.7292 — Imported 2026-05-27
13 internlm-chat-7b-hf (zero-shot) 34.6181 — Imported 2026-05-27
14 yulan-chat-2-13b-fp16-hf (one-shot) 34.5052 — Imported 2026-05-27
15 yulan-chat-2-13b-fp16-hf (zero-shot) 33.757 — Imported 2026-05-27
16 fuzi-mingcha-7b-hf (zero-shot) 33.0473 — Imported 2026-05-27
17 chatlaw-13b-hf (zero-shot) 32.7598 — Imported 2026-05-27
18 chatlaw-13b-hf (one-shot) 32.6287 — Imported 2026-05-27
19 wisdom-interrogatory-7b-hf (zero-shot) 31.4065 — Imported 2026-05-27
20 belle-2-13b-hf (one-shot) 30.786 — Imported 2026-05-27
21 belle-2-13b-hf (zero-shot) 30.4102 — Imported 2026-05-27
22 hanfei-1.0-7b-hf (zero-shot) 29.7124 — Imported 2026-05-27
23 baichuan-13b-chat-hf (one-shot) 29.5891 — Imported 2026-05-27
24 fuzi-mingcha-7b-hf (one-shot) 28.7808 — Imported 2026-05-27
25 lexilaw-6b-hf (zero-shot) 28.7769 — Imported 2026-05-27
26 wisdom-interrogatory-7b-hf (one-shot) 27.744 — Imported 2026-05-27
27 lexilaw-6b-hf (one-shot) 26.4075 — Imported 2026-05-27
28 chatlaw-33b-hf (zero-shot) 26.1448 — Imported 2026-05-27
29 tigerbot-sft-7b-hf (zero-shot) 25.886 — Imported 2026-05-27
30 tigerbot-sft-7b-hf (one-shot) 25.6232 — Imported 2026-05-27
31 chatlaw-33b-hf (one-shot) 25.4099 — Imported 2026-05-27
32 lawyer-llama-13b-hf (zero-shot) 25.3206 — Imported 2026-05-27
33 hanfei-1.0-7b-hf (one-shot) 24.9138 — Imported 2026-05-27
34 baichuan-13b-base-hf (one-shot) 24.0386 — Imported 2026-05-27
35 llama-2-chinese-13b-hf (zero-shot) 23.3556 — Imported 2026-05-27
36 lawyer-llama-13b-hf (one-shot) 23.0229 — Imported 2026-05-27
37 chatglm2-6b-hf (one-shot) 22.9426 — Imported 2026-05-27
38 internlm-7b-hf (one-shot) 22.4661 — Imported 2026-05-27
39 tigerbot-base-7b-hf (zero-shot) 21.706 — Imported 2026-05-27
40 baichuan-13b-chat-hf (zero-shot) 21.4144 — Imported 2026-05-27
41 baichuan-7b-hf (one-shot) 21.1823 — Imported 2026-05-27
42 chatglm2-6b-hf (zero-shot) 21.1546 — Imported 2026-05-27
43 llama-2-70b (one-shot) 20.8017 — Imported 2026-05-27
44 tigerbot-base-7b-hf (one-shot) 20.3355 — Imported 2026-05-27
45 qwen-7b-hf (zero-shot) 19.5785 — Imported 2026-05-27
46 chinese-alpaca-2-7b-hf (one-shot) 19.2206 — Imported 2026-05-27
47 vicuna-33b-hf (one-shot) 19.1093 — Imported 2026-05-27
48 baichuan-13b-base-hf (zero-shot) 19.0605 — Imported 2026-05-27
49 ziya-llama-13b-hf (one-shot) 18.7924 — Imported 2026-05-27
50 qwen-7b-hf (one-shot) 18.6248 — Imported 2026-05-27
51 llama-2-13b (one-shot) 18.3732 — Imported 2026-05-27
52 llama-2-7b (one-shot) 18.2648 — Imported 2026-05-27
53 ziya-llama-13b-hf (zero-shot) 17.9855 — Imported 2026-05-27
54 llama-65b (one-shot) 17.6745 — Imported 2026-05-27
55 llama-2-chinese-13b-hf (one-shot) 17.0795 — Imported 2026-05-27
56 llama-13b (one-shot) 16.6094 — Imported 2026-05-27
57 llama-30b (one-shot) 16.5186 — Imported 2026-05-27
58 internlm-7b-hf (zero-shot) 16.2684 — Imported 2026-05-27
59 gogpt-7b (one-shot) 16.2336 — Imported 2026-05-27
60 vicuna-13b-hf (one-shot) 15.8116 — Imported 2026-05-27
61 moss-moon-003-base-hf (one-shot) 15.6769 — Imported 2026-05-27
62 alpaca-7b-hf (one-shot) 15.6205 — Imported 2026-05-27
63 moss-moon-003-sft-hf (one-shot) 15.605 — Imported 2026-05-27
64 moss-moon-003-sft-hf (zero-shot) 15.22 — Imported 2026-05-27
65 baichuan-7b-hf (zero-shot) 15.0904 — Imported 2026-05-27
66 moss-moon-003-base-hf (zero-shot) 14.8432 — Imported 2026-05-27
67 llama-7b (one-shot) 14.7608 — Imported 2026-05-27
68 llama-2-70b-chat (zero-shot) 14.7059 — Imported 2026-05-27
69 llama-2-13b-chat (zero-shot) 14.64 — Imported 2026-05-27
70 wizardlm-7b-hf (one-shot) 14.4698 — Imported 2026-05-27
71 mpt-7b-hf (one-shot) 14.3262 — Imported 2026-05-27
72 wizardlm-7b-hf (zero-shot) 13.7947 — Imported 2026-05-27
73 chinese-alpaca-2-7b-hf (zero-shot) 13.7736 — Imported 2026-05-27
74 xverse-13b-hf (one-shot) 13.7059 — Imported 2026-05-27
75 vicuna-7b-hf (one-shot) 13.6461 — Imported 2026-05-27
76 llama-2-70b (zero-shot) 13.5553 — Imported 2026-05-27
77 xverse-13b-hf (zero-shot) 13.5461 — Imported 2026-05-27
78 llama-2-70b-chat (one-shot) 12.6651 — Imported 2026-05-27
79 mpt-instruct-7b-hf (one-shot) 12.6287 — Imported 2026-05-27
80 chinese-llama-2-7b-hf (one-shot) 12.5496 — Imported 2026-05-27
81 vicuna-13b-hf (zero-shot) 12.3923 — Imported 2026-05-27
82 llama-2-13b-chat (one-shot) 12.1053 — Imported 2026-05-27
83 llama-2-7b (zero-shot) 11.695 — Imported 2026-05-27
84 lawgpt-7b-beta1.1-hf (one-shot) 11.5905 — Imported 2026-05-27
85 vicuna-33b-hf (zero-shot) 11.5738 — Imported 2026-05-27
86 vicuna-7b-hf (zero-shot) 11.5164 — Imported 2026-05-27
87 mpt-7b-hf (zero-shot) 11.2057 — Imported 2026-05-27
88 llama-2-chinese-7b-hf (one-shot) 10.666 — Imported 2026-05-27
89 mpt-instruct-7b-hf (zero-shot) 10.3289 — Imported 2026-05-27
90 alpaca-7b-hf (zero-shot) 10.0245 — Imported 2026-05-27
91 lawgpt-7b-beta1.1-hf (zero-shot) 9.9129 — Imported 2026-05-27
92 llama-2-13b (zero-shot) 9.78 — Imported 2026-05-27
93 llama-7b (zero-shot) 9.7191 — Imported 2026-05-27
94 llama-30b (zero-shot) 9.2592 — Imported 2026-05-27
95 gogpt-7b (zero-shot) 8.9185 — Imported 2026-05-27
96 chinese-llama-2-7b-hf (zero-shot) 8.7914 — Imported 2026-05-27
97 llama-13b (zero-shot) 8.7448 — Imported 2026-05-27
98 llama-65b (zero-shot) 8.44 — Imported 2026-05-27
99 llama-2-7b-chat (zero-shot) 7.1598 — Imported 2026-05-27
100 llama-2-7b-chat (one-shot) 6.8263 — Imported 2026-05-27
101 llama-2-chinese-7b-hf (zero-shot) 2.957 — Imported 2026-05-27
102 lawgpt-7b-beta1.0-hf (zero-shot) 1.5146 — Imported 2026-05-27