无需参考文本，四维度评估代码注释质量
CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
- 不依赖参考注释，从语义、信息量等四方面综合评分
- 在人工标注数据集上表现优于现有指标，交叉熵更低
- 适合用于高质量代码注释数据集构建与模型训练
高效生成结构化代码注释需要可靠的评价指标以支持数据集构建，但现有方法（SIDE、MIDQ、STASIS）在代码-注释分析上存在局限。本文提出CIDRe，一种语言无关的无参考质量评估准则，融合四个协同作用的维度：(1) 相关性（代码与注释语义对齐）、(2) 信息量（功能覆盖程度）、(3) 完整性（所有结构部分存在）、(4) 描述长度（细节充分性）。我们在人工标注的数据集上验证该准则。实验表明，CIDRe在交叉熵评估中优于现有指标；将其用于过滤注释后，基于筛选数据微调的模型在GPT-4o-mini评估中表现出统计显著的质量提升。
原文摘要 · Abstract (English)
Effective generation of structured code comments requires robust quality metrics for dataset curation, yet existing approaches (SIDE, MIDQ, STASIS) suffer from limited code-comment analysis. We propose CIDRe, a language-agnostic reference-free quality criterion combining four synergistic aspects: (1) relevance (code-comment semantic alignment), (2) informativeness (functional coverage), (3) completeness (presence of all structure sections), and (4) description length (detail sufficiency). We validate our criterion on a manually annotated dataset. Experiments demonstrate CIDRe's superiority over existing metrics, achieving improvement in cross-entropy evaluation. When applied to filter comments, the models finetuned on CIDRe-filtered data show statistically significant quality gains in GPT-4o-mini assessments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。