用信息论分析代码与文档的可追溯性,发现文档信息量常不足。
Lost in Transmission: An Information-Theoretic Account of Unsupervised Software Traceability
- 基于自信息和互信息量化代码与文档的信息量与匹配度
- 工业数据集显示代码平均信息量高于文档,互信息普遍偏低
- 提醒开发者重视文档质量而非盲目提升模型复杂度
可追溯性是保障现代软件系统可靠性、可维护性和合规性的关键能力。尽管无监督信息检索(IR)和机器学习(ML)技术广泛用于自动化追溯链接恢复,但其效果常受限于底层构件的质量与结构。实践中,这些方法假设文本中隐含有意义的追溯信号,但在工业场景中,文档稀疏、不一致或不平衡时该假设往往不成立。此外,传统评估指标(如精确率、召回率、F1)在未考虑数据特征时可能误判性能。本文提出TraceXplainer,一种基于信息论的框架,用于评估无监督追溯的可靠性与极限。该方法利用自信息和互信息(MI)量化源代码与目标文档的信息量及对齐程度。通过对工业数据集的全面实证分析,我们发现典型追溯语料库存在显著信息失衡:源代码平均信息量高于对应文档。同时,观测到的互信息水平、损失值与噪声揭示了无监督技术恢复准确追溯链接的固有局限。研究结果表明,实际提升可追溯性需转向以数据为中心的工程实践,关注构件质量、一致性与信息对齐,而非仅追求模型复杂度。本研究为从业者评估可追溯性准备度并优化文档与开发流程提供洞见。
原文摘要 · Abstract (English)
Traceability remains a critical capability to ensure system reliability, maintainability, and compliance in modern software development. Although unsupervised Information Retrieval (IR) and Machine Learning (ML) techniques are widely adopted for automated trace link recovery, their effectiveness is often limited by the quality and structure of the underlying artifacts. In practice, these approaches assume that meaningful traceability signals are embedded in textual data, an assumption that rarely holds in industrial settings with sparse, inconsistent, or unbalanced documentation. Furthermore, conventional evaluation metrics (e.g., precision, recall, F1) can misrepresent performance when data characteristics are not explicitly considered. We introduce TraceXplainer, an information-theoretic framework for evaluating the reliability and limits of unsupervised traceability. Our approach leverages self-information and mutual information (MI) to quantify the informativeness and alignment of source and target artifacts. Through a comprehensive empirical analysis of industry datasets, we show that typical traceability corpora exhibit significant information imbalances, where the source code contains on average more information than the corresponding documentation. In addition, the observed levels of mutual information, loss, and noise reveal inherent constraints on the ability of unsupervised techniques to recover accurate trace links. These findings suggest that improving traceability in practice requires a shift to data-centric engineering, focusing on artifact quality, consistency, and information alignment; rather than solely advancing model sophistication (or complexity). Our results provide insights for practitioners to better assess traceability readiness and guide improvements in documentation and development workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。