提出时间后悔机制,让智能体自动识别并纠正因果模型错误。
Trivium: Temporal Regret as a First-Class Objective for Causal-Memory Controllers

- 用时间后悔+认知后悔替代传统奖励优化,诊断错误的根源与时机。
- 在500次实验中,传统方法持续误判,而新方法仅7.8±2.9次误识别。
- 适合需要长期可解释性与自我修正能力的复杂决策系统。
许多代理系统和大模型流水线通过优化结果奖励来纠正错误,但仅关注‘结果’而忽略‘原因’与‘时机’,导致错误重复出现。本文提出长期时间后悔、结果后悔与认知后悔三类诊断性指标,非传统在线学习中的比较型后悔。时间后悔衡量错误因果模型被容忍的时长,认知后悔反映该模型后验误差。在E个实验流上,基于显式探测、持续记录与可检测假设,证明三个条件结论:第一,在观测等价混淆下,仅优化结果奖励无法区分因果与虚假结构,误校准可能在结果后悔为零后仍持续;第二,在持久因果日志与预算探测下,总探测复杂度为O(log E),诱导出O(log E)的延迟识别时间后悔;第三,当存在K个可检测变化点时,速率扩展至O(K log E)。实例化Trivium系统并提出五项可验证预测。在CausalBench-Seq上,硬结构读取显示每种子运行500次共20种子时,平均7.8±2.9次误识别,无观测平稳误差,而仅优化结果的方法始终误判。审计消融实验表明,后验更新持续进行而非检测器重开或局部修复驱动恢复;局部修复反而降低已提交图的暴露。此前基于截断软得分的对数包络结论被撤回。一次真实大模型流的初步实验提供外部证据。此处自学习指修订外部因果模型,而非重新训练语言模型权重。
原文摘要 · Abstract (English)
Many agentic systems and LLM pipelines correct mistakes by optimizing outcome reward. This addresses only the what of failure; the why and when may go unlogged, allowing the same error to recur across episodes. We propose long-horizon temporal regret alongside outcome regret and epistemic regret. These are diagnostic quantities, not standard comparator-based online-learning regrets. Temporal regret captures how long an unresolved or incorrect causal model is tolerated; epistemic regret captures posterior error over that model. Over a stream of E episodes, we prove three conditional results under explicit probing, persistence, and detectability assumptions. First, under observationally equivalent confounding, outcome-only learning cannot separate causal from spurious structure, so miscalibration can persist after outcome regret reaches zero. Second, with a persistent causal log and budgeted probes, total probe complexity is logarithmic in E, inducing O(log E) delayed-identification temporal regret. The implemented clipped soft score adds a linear numerical floor, so this logarithmic claim applies only to identification delay. Third, under K detectable change-points, the rate extends to O(K log E). We instantiate Trivium and state five falsifiable predictions. On CausalBench-Seq, a hard structural readout records 7.8+-2.9 misidentified episodes per seed over 500 episodes and 20 seeds, with zero observed stationary errors, while outcome-only controllers remain misidentified throughout. Audit ablations show that continued posterior updating, not detector reopening or local repair, drives posterior recovery; local repair instead reduces committed-graph dispatch exposure. The prior logarithmic-envelope verdict based on a clipped soft score is withdrawn. A pilot real-LLM stream provides external evidence. Self-learning here means revising an external causal model, not retraining LLM weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。