量化大模型推理中时间泄漏的影响，实现可解释的检测与修复。
All Leaks Count, Some Count More: Interpretable Temporal Contamination Detection and Mitigation in LLM Backtesting
- 通过拆解推理逻辑为原子声明，用谢林值衡量每部分对决策的贡献。
- 提出Shapley-DCLR指标，精确计算决策依赖的泄露信息比例。
- 设计TimeSPEC架构，在推理时强制使用截止前证据，适合严格合规场景。
在已解决事件上测试大语言模型时，假设模型仅基于截止前的知识进行推理，但预训练模型不可避免地会泄露截止后的知识。本文提出一种基于声明级别的评估框架，将预测理由分解为原子声明，并运用谢林值量化每个声明对决策的影响，得到可解释的指标Shapley-DCLR（谢林加权决策关键泄露率），用于衡量决策驱动推理中被污染的比例。进一步提出TimeSPEC（时间监督的声明提取预测）架构，在推理阶段融合时间过滤检索与声明级监督，确保预测完全基于截止前证据。在三个大模型上的消融实验表明，检索与监督机制需协同使用；三任务探测实验还显示，时间约束带来的性能损失随任务对截止后信息的依赖程度而增加。
原文摘要 · Abstract (English)
Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim's decision impact, yielding \textbf{Shapley-DCLR} (\textbf{Shapley}-weighted \textbf{D}ecision-\textbf{C}ritical \textbf{L}eakage \textbf{R}ate) -- an interpretable metric measuring what fraction of decision-driving reasoning is contaminated. We further propose \textbf{TimeSPEC} (\textbf{Time}-\textbf{S}upervised \textbf{P}rediction with \textbf{E}xtracted \textbf{C}laims), an inference-time architecture that interleaves temporally-filtered retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm retrieval and supervision are jointly necessary; and a three-task probe further illstrates that the performance cost of temporal enforcement scales with each task's reliance on post-cutoff information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。