提出细粒度代码模型污染检测框架，精准识别语义相似代码。
TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMs
- 构建三层次语义重叠模型，分粗到精检测代码污染。
- 在多模型上实现0.91的细粒度F1，优于现有方法42%-217%。
- 首个代码污染检测基准，适合评估代码LLM可靠性。
数据污染是影响模型评估可靠性的已知威胁，但在代码大语言模型中仍研究不足，污染常超出精确重复范畴。本文提出TRACER，一种面向细粒度代码污染检测的语义感知框架。该框架通过函数功能相同、几乎相同、共享逻辑三个语义重叠层级建模污染，并采用粗到精的检测流程。我们还构建了首个细粒度代码污染检测基准，涵盖三个主流基准和三个典型后训练数据集。TRACER在多个代码LLM骨干网络上表现强劲且稳定，GPT-5在细粒度检测中取得0.91的F1分数；在二分类设置下，F1达0.92，较现有方法提升42%-217%。我们进一步通过消融实验与错误分析，评估各组件贡献。
原文摘要 · Abstract (English)
Data contamination is a known threat to the reliability of model evaluation. However, it remains underexplored in code large language models (LLMs), where contamination often goes beyond exact duplication. We present TRACER, a semantic-aware framework for fine-grained code contamination detection. TRACER models contamination using three levels of semantic overlap - Functionally Identical, Nearly Identical, and Shared Logic - and detects them through a coarse-to-fine pipeline. We also introduce the first benchmark for fine-grained code contamination detection, spanning three widely used benchmarks and three representative post-training datasets. TRACER achieves strong and consistent performance across multiple LLM backbones, with GPT-5 reaching an F1 score of 0.91 in fine-grained detection. In the binary setting, TRACER attains an F1 of 0.92, outperforming existing methods by 42%-217%. We further conduct ablation studies and error analysis to assess the contributions of individual components in TRACER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。