用110万条Gherkin步骤检测重复，提升软件测试可维护性。
Deja Vu at Scale: Paraphrase-Robust Detection of Duplicate Gherkin Steps in Behaviour-Driven Software Testing with Sentence-Transformer Embeddings and a 1.1M-Step Open Benchmark
- 基于四种策略融合的相似度检测，支持语义和语法双重匹配。
- 发现80.2%步骤存在重复，单个仓库平均58.6%可消除。
- 开源百万级数据集与标注基准，适合测试工程与质量优化研究者。
行为驱动开发（BDD）在Gherkin中积累步骤文本重复，导致维护成本上升。现有检测方法或需可运行测试，或局限于单一组织，缺乏静态、抗改写、细粒度的检测工具及公开基准。本文发布：(i) 当前最大跨组织的BDD步骤语料库；(ii) 标注的成对校准基准；(iii) 四策略融合的检测器，并构建了关联聚类与ISO/IEC 25010可维护性子特性的节省模型。语料库涵盖347个公共GitHub仓库、23,667个.feature文件、1,113,616条Gherkin步骤，含SPDX标签。检测器结合精确哈希、归一化莱文斯坦、Sentence-Transformer余弦相似度与带莱文斯坦带的混合策略。校准采用1,020对人工标注的步骤对，遵循公开评分标准（60对重叠，Fleiss kappa = 0.84）。报告主评分标准与无分重标下的精确率、召回率与F1值（95%置信区间），并与SourcererCC与NiCad风格的词汇基线对比。结果显示，步骤加权的完全重复率为80.2%；中位仓库重复率为58.6%（斯皮尔曼相关系数rho = 0.51）。最优混合聚类共出现20,737次，覆盖2,245个文件。近似匹配在无分重标下达到F1 = 0.822；语义匹配在主标准下为F1 = 0.906，反映披露的分层偏差。词汇基线分别达F1 = 0.761与0.799。节省模型估算语料库中共有893,357条可删除重复步骤；中位仓库中62.5%的步骤行可被消除。
原文摘要 · Abstract (English)
Context. Behaviour-Driven Development (BDD) suites in Gherkin accumulate step-text duplication with documented maintenance cost. Prior detectors either require runnable tests or are single-organisation, leaving a gap: a static, paraphrase-robust, step-level detector and a public benchmark to calibrate it. Objective. We release (i) the largest cross-organisational BDD step corpus to date, (ii) a labelled pair-level calibration benchmark, and (iii) a four-strategy detector with a consolidation-savings model linking clusters to ISO/IEC 25010 maintainability sub-characteristics. Method. The corpus contains 347 public GitHub repositories, 23,667 .feature files, and 1,113,616 Gherkin steps, SPDX-tagged. The detector layers exact hashing, normalised Levenshtein, sentence-transformer cosine, and a Levenshtein-banded hybrid. Calibration uses 1,020 manually labelled step pairs under a released rubric (60-pair overlap, Fleiss kappa = 0.84). We report precision, recall, and F1 with bootstrap 95% CIs under the primary rubric and a score-free relabelling, and benchmark against SourcererCC-style and NiCad-style lexical baselines. Results. Step-weighted exact-duplicate rate is 80.2%; median-repository rate is 58.6% (Spearman rho = 0.51). The top hybrid cluster has 20,737 occurrences across 2,245 files. Near-exact reaches F1 = 0.822 on score-free labels; semantic F1 = 0.906 under the primary rubric reflects a disclosed stratification artefact. Lexical baselines reach F1 = 0.761 and 0.799. The savings model estimates 893,357 corpus-wide eliminable step occurrences; on the median repository 62.5% of step lines are eliminable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。