测试金融大模型在时间点任务中的前瞻偏差，发现主流模型存在记忆性预测问题。
Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
- 通过真实金融场景评估模型前瞻偏差，区分预测能力与记忆。
- 标准LLM在不同市场周期中表现衰减明显，而PiT模型随规模提升泛化更好。
- 适合关注金融AI可信性、模型部署风险的开发者和研究人员。
我们提出Look-Ahead-Bench，一个标准化基准，用于衡量金融领域点对点（Point-in-Time, PiT）大语言模型中的前瞻偏差。不同于现有主要通过问答测试内部前瞻知识的方法，本基准在真实金融工作流中评估模型行为。为区分真正的预测能力与基于记忆的表现，我们分析模型在不同时段市场环境下的性能衰减，并引入多个量化基线以建立性能阈值。评估了Llama 3.1（8B和70B）、DeepSeek 3.2等主流开源模型，以及PiT-Inference系列的Pitinf-Small、Pitinf-Medium和前沿级的Pitinf-Large模型。结果表明，标准大模型存在显著的前瞻偏差（以alpha衰减衡量），而PiT模型则随着规模增大展现出更强的泛化与推理能力。本工作为金融大模型的时间偏差标准化评估奠定基础，并提供实际部署可用的识别框架。代码已公开于GitHub：https://github.com/benstaf/lookaheadbench
原文摘要 · Abstract (English)
We introduce Look-Ahead-Bench, a standardized benchmark measuring look-ahead bias in Point-in-Time (PiT) Large Language Models (LLMs) within realistic and practical financial workflows. Unlike most existing approaches that primarily test inner lookahead knowledge via Q\\&A, our benchmark evaluates model behavior in practical scenarios. To distinguish genuine predictive capability from memorization-based performance, we analyze performance decay across temporally distinct market regimes, incorporating several quantitative baselines to establish performance thresholds. We evaluate prominent open-source LLMs -- Llama 3.1 (8B and 70B) and DeepSeek 3.2 -- against a family of Point-in-Time LLMs (Pitinf-Small, Pitinf-Medium, and frontier-level model Pitinf-Large) from PiT-Inference. Results reveal significant lookahead bias in standard LLMs, as measured with alpha decay, unlike Pitinf models, which demonstrate improved generalization and reasoning abilities as they scale in size. This work establishes a foundation for the standardized evaluation of temporal bias in financial LLMs and provides a practical framework for identifying models suitable for real-world deployment. Code is available on GitHub: https://github.com/benstaf/lookaheadbench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。