用智能代理自动评估代码架构，让大模型更懂软件设计。
Beyond Correctness: Enhancing Architectural Reasoning in Code LLMs via Scalable Labeling with Agentic Judgment
- 用双判官机制自动判断代码架构复杂度与规范符合度
- 微调后在SWE-bench上正确率提升540%，超基线模型
- 适合需要提升代码架构理解力的工程团队和研究者
大语言模型虽显著提升软件工程效率，但真实开发仍需架构理解。这种理解手动标注成本过高，仅靠测试无法验证。本文提出一种基于强模型的智能代理评判流程，包含两个判官：架构复杂度判官（ACJ）评估任务所需架构理解程度，架构质量判官（AQJ）基于源码依据评估补丁是否符合项目特定架构规范。在3,360个精选样本上微调Qwen3-8B/14B/32B模型，使SWE-bench Verified任务的解决率最高达27.2%，相比基线模型提升540%，较未过滤微调提升256%。同时模型具备良好的跨语言泛化能力，且持续提升架构补丁质量。
原文摘要 · Abstract (English)
LLMs have substantially improved software engineering yet real-world development requires architectural understanding. Such understanding is prohibitively expensive to label manually and impossible to verify through tests alone. We propose an agentic judging pipeline using a strong LLM as a scalable proxy for expert architectural evaluation, comprising two judges: the Architecture Complexity Judge (ACJ), which estimates codebase-specific architectural understanding a task demands, and the Architecture Quality Judge (AQJ), which evaluates patch conformance to repository-specific architectural conventions via source-grounded rubrics. Fine-tuning Qwen3-8B/14B/32B on 3,360 curated instances achieves resolved rates of up to 27.2% on SWE-bench Verified - up to 540% over the base model and 256% over unfiltered fine-tuning. Meanwhile, the trained models achieve strong cross-language generalization and consistent improvements in architectural patch quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。