LLM代理缺乏时间感知,导致工具调用时机与人类判断严重不符。
Your LLM Agents are Temporally Blind: The Misalignment Between Tool Use Decisions and Human Time Perception
- 构建时序数据集TicToc,模拟76种动态场景下的对话轨迹。
- 现有模型在有时间戳时与人类偏好对齐率最高仅65%。
- 特定后训练方法可有效提升模型对人类时间感知的对齐能力。
大型语言模型(LLM)代理在动态环境中执行任务时,普遍存在「时间盲视」问题——默认假设上下文静态不变,忽略消息间真实时间流逝。这导致工具调用决策偏差:要么依赖过时上下文而漏调工具,要么重复调用。为此,我们构建了TicToc数据集,涵盖76个高、中、低时间敏感度的多轮对话场景,收集人类对「调用工具」与「直接回答」的选择偏好。评估显示,现有模型在给出时间戳的情况下,与人类偏好对齐率均未超过65%。简单提示工程效果有限,但特定后训练方法可有效改善多轮对话中的时间感知对齐。本研究为理解并缓解时间盲视提供了基础,推动更具备时间意识与人类对齐的智能体发展。
原文摘要 · Abstract (English)
Large language model (LLM) agents are increasingly used to interact with and execute tasks in dynamic environments. However, a critical yet overlooked limitation of these agents is that they, by default, assume a stationary context, failing to account for the real-world time elapsed between messages. We refer to this as "temporal blindness". This limitation hinders decisions about when to invoke tools, leading agents to either over-rely on stale context and skip needed tool calls, or under-rely on it and redundantly repeat tool calls. To study this challenge, we constructed TicToc, a diverse dataset of multi-turn user-agent message trajectories across 76 scenarios, spanning dynamic environments with high, medium, and low time sensitivity. We collected human preferences between "calling a tool" and "directly answering" on each sample, and evaluated how well LLM tool-calling decisions align with human preferences under varying amounts of elapsed time. Our analysis reveals that existing models display poor alignment with human temporal perception, with no model achieving a normalized alignment rate better than 65% when given time stamp information. We also show that naive, prompt-based alignment techniques have limited effectiveness for most models, but specific post-training alignment can be a viable way to align multi-turn LLM tool use with human temporal perception. Our data and findings provide a first step toward understanding and mitigating temporal blindness, offering insights to foster the development of more time-aware and human-aligned agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。