ArchBench ADR Generation

Architectural Design Records Generation: Generate Architecture Decision Records (ADRs) from given decision contexts.

4rows
bertscore_f1primary metric
2026-05-06sampled

Metadata

ID
archbench_adr_generation
Category
Software Engineering
Release
2026-03-18

Metrics

ROUGE-1, BLEU, METEOR, BERTScore Precision, BERTScore Recall, BERTScore F1

Latest Results

Rows ranked by bertscore_f1. ArchBench paper: https://arxiv.org/abs/2603.17833. CLI: https://github.com/sa4s-serc/archbench-cli.

Rank Subject BERTScore F1 Model Match Provenance Sampled
1 GPT-4 (0-shot) 0.85 — Imported 2026-05-06
2 GPT-3.5-davinci-003 (few-shot) 0.85 — Imported 2026-05-06
3 Flan-T5-base (fine-tuned) 0.84 — Imported 2026-05-06
4 T0-3b (0-shot) 0.84 — Imported 2026-05-06