让AI理解自然语言规则,自动发现代码隐藏问题。
MetaLint: Easy-to-Hard Generalization for Code Linting
- 用自然语言描述规范,让模型判断代码是否合规
- 在新规则下无需重训,检测准确率提升2.7倍
- 适合需要灵活适配新编码规范的开发者
大语言模型在代码生成上表现优异,但在代码检查方面难以泛化到训练中未见或演进中的最佳实践。我们提出MetaLint,一种元学习框架,将代码检查任务建模为指令遵循问题:模型根据自然语言描述的最佳实践规范评估代码是否合规。与以往依赖固定规则集的方法不同,MetaLint可在测试时动态控制需执行的规范,无需重训即可适应新规则。我们在仅使用自动生成的合成数据训练的模型上,验证了其对复杂、上下文相关的最佳实践(传统lint工具无法覆盖)仍具备良好泛化能力。为此,我们构建了一个由人工标注的硬规则基准,灵感来自Python增强提案(PEPs)。在该基准上,模型无需针对目标规范微调即显著提升性能:Qwen3-4B的检测F-score提升2.7倍(25.9% → 70.4%),召回率最高,定位F-score达26.7%,媲美更大模型如o3-mini。该优势在多种编程语言、模型架构、规模、推理模式和linter来源间均保持一致。代码与基准已开源。
原文摘要 · Abstract (English)
Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practices beyond those observed during training. We introduce MetaLint, a meta-learning framework that formulates code linting as an instruction-following task, where a model evaluates whether code adheres to a natural language specification of best practices. In contrast to prior work that trains models to detect violations from a fixed set of best practices, MetaLint evaluates code against a provided natural language specification, enabling test-time control over which practices to enforce and generalization to unseen or evolving rules without retraining. We demonstrate that models trained solely on synthetic data generated from automatic linters still generalize to harder, context-dependent best practices for which such linters are not available. To evaluate generalization beyond such easy signals, we introduce a human-curated benchmark of hard best practices inspired by Python Enhancement Proposals (PEPs). On this benchmark, MetaLint substantially improves performance without explicit fine-tuning on target best practices and exhibits strong, easy-to-hard generalization. Qwen3-4B achieves a 2.7x detection F-score gain (25.9% -> 70.4%), the highest recall, and a 26.7% localization F-score, matching larger models such as o3-mini. These gains generalize across programming languages, model families, scales, reasoning settings, and linter sources. We release the code and benchmark to support reproducibility and future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。