测试大模型处理时间约束能力，发现多数模型表现不稳定且易出错。
Empirical Characterization of Temporal Constraint Processing in LLMs
- 通过任务测试8个大模型的时间约束判断能力，发现性能呈两极分化。
- 模型准确率在95%与50%间波动，格式变化可导致性能下降30-60个百分点。
- 现有模型缺乏持续时间状态表示，适合对时间敏感应用的开发者警惕。
在需要实时决策的智能体架构中部署大语言模型时，我们通常假设其能可靠判断动作窗口是否开放或已关闭，但这一假设未经验证。本文通过截止时间检测任务，对八款生产级模型（参数量2.8-8B）进行了时间约束处理能力的实证分析，揭示出系统性部署风险：性能分布呈现双峰特征（模型准确率要么达95%，要么仅50%），提示存在根本性缺陷；提示格式微调即可引发30-60个百分点的性能波动，显示极端脆弱性；失败模型表现出100%的误报率，存在系统性行动偏差。参数规模在此范围内与能力无相关性——3.8B模型表现可媲美7B模型，而部分7B模型则完全失效。在200个合成示例上进行微调可使具备部分能力的模型提升12-37个百分点。研究证明，仅靠自然语言的下一个词预测，即使经过针对性微调，也无法可靠习得时间约束满足能力。该能力需依赖三类机制：(1) 连续的时间状态表征，(2) 与语言模式匹配分离的显式约束检查，(3) 对时间关系的系统性组合推理。当前自回归架构缺乏这些机制。在时间敏感场景中直接部署此类系统，若不引入融合符号推理模块的混合架构，将构成不可接受的风险。
原文摘要 · Abstract (English)
When deploying LLMs in agentic architectures requiring real-time decisions under temporal constraints, we assume they reliably determine whether action windows remain open or have closed. This assumption is untested. We characterize temporal constraint processing across eight production-scale models (2.8-8B parameters) using deadline detection tasks, revealing systematic deployment risks: bimodal performance distribution (models achieve either 95% or 50% accuracy), extreme prompt brittleness (30-60 percentage point swings from formatting changes alone), and systematic action bias (100% false positive rates in failing models). Parameter count shows no correlation with capability in this range-a 3.8B model matches 7B models while other 7B models fail completely. Fine-tuning on 200 synthetic examples improves models with partial capability by 12-37 percentage points. We demonstrate that temporal constraint satisfaction cannot be reliably learned through next-token prediction on natural language, even with targeted fine-tuning. This capability requires architectural mechanisms for: (1) continuous temporal state representation, (2) explicit constraint checking separate from linguistic pattern matching, (3) systematic compositional reasoning over temporal relations. Current autoregressive architectures lack these mechanisms. Deploying such systems in time-critical applications without hybrid architectures incorporating symbolic reasoning modules represents unacceptable risk.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。