测试大模型何时开始‘过时’，帮用户避开知识盲区。
LLMLagBench: Identifying Temporal Training Boundaries in Large Language Models
- 通过评估模型对近期事件的认知，定位其训练数据的时间边界。
- 在多个大模型上验证，发现部分模型未公开训练截止时间。
- 适合关注模型时效性、需避免过时信息的使用者。
大型语言模型（LLMs）在其训练数据中存在明确的时间截止点，这导致模型在该时间点之后的信息无法准确提供，必须依赖外部查询。更隐蔽的问题是，当这一限制被忽视时，模型可能在推理过程中将过时的时间敏感信息与通用知识混淆，影响回答准确性。本文提出 LLMLagBench，一个系统性的大模型时效性基准，通过评估模型对近期事件的知识来识别其训练数据最早可能的时间边界。我们对该基准在大量模型上的有效性进行了评估，包括具有明确和未声明训练截止时间的模型。通过人工验证和与公开预训练信息对比，确认了该基准的可靠性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are pretrained on textual data up to a specific temporal cutoff. This creates a strict knowledge boundary beyond which models cannot provide accurate information without querying external sources. More subtly, when this limitation is unknown or ignored, LLMs may inadvertently blend outdated time-sensitive information with general knowledge during reasoning tasks, potentially compromising response accuracy. We introduce LLMLagBench, an LLM freshness benchmark, as a systematic approach for identifying the earliest probable temporal boundaries of an LLM's training data by evaluating its knowledge of recent events. We then apply this benchmark to evaluate a large set of LLMs, including models with both explicitly declared and undeclared training cutoffs. The reliability of the benchmark is assessed by manual validation and comparison with publicly released information about LLM pretraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。