ArchBench Serverless Component Generation

Architectural Component Generation: Generate complete serverless functions (FaaS) from specifications and codebase context.

5rows
no_intervention_codebase_testsprimary metric
2026-05-06sampled

Metadata

ID
archbench_serverless_component_generation
Category
Software Engineering
Release
2026-03-18

Metrics

Codebase Tests (No Intervention), Function Tests (No Intervention), Source Lines of Code (No Intervention) (lower is better), Cyclomatic Complexity (No Intervention) (lower is better), Cognitive Complexity (No Intervention) (lower is better), Halstead Volume (No Intervention) (lower is better), CodeBLEU (No Intervention), Codebase Tests (With Intervention), Function Tests (With Intervention)

Latest Results

Rows ranked by no_intervention_codebase_tests. ArchBench paper: https://arxiv.org/abs/2603.17833. CLI: https://github.com/sa4s-serc/archbench-cli.

Rank Subject Codebase Tests (No Intervention) Model Match Provenance Sampled
1 DeepSeek-Coder-V2 81 — Imported 2026-05-06
2 GPT-4 79 — Imported 2026-05-06
3 CodeQwen1.5-7B-Chat 69 — Imported 2026-05-06
4 GPT-3.5-Turbo 68 — Imported 2026-05-06
5 Artigenz-Coder-DS-6.7B 67 — Imported 2026-05-06