测试大模型在无法解决的记忆冲突下的应对能力，发现现有模型常强行选答案。
When Personal Memory Has No Single Answer: Evaluating LLM Agents under Irreducible Conflict
- 构建新基准TANGLE，模拟三种真实不可解记忆冲突场景
- 模型能识别冲突但难合理应对，尤其在端到端对话中记忆丢失严重
- 提出冲突感知动作策略，鼓励保留矛盾、主动追问而非武断决策
大型语言模型代理在多轮会话中维护个人记忆，但记忆可能冲突。由于偏好依赖上下文、行为演变和信息源矛盾，当查询缺乏上下文、时间或权威来源以解释冲突时，将某条记忆视为唯一正确答案会将未决矛盾转化为无根据的过度自信行为。现有基准仅恢复单一答案，忽视模型是否意识到不确定性、保留备选方案、主动寻求缺失信息或采取恰当行动。我们提出TANGLE基准，涵盖40个角色、541个实例，包含三种真实不可解冲突类型：上下文分割冲突（CPC）、行为波动冲突（BOC）和来源矛盾冲突（SCC）。评估两个赛道——有标注记忆的真值赛道与从多轮对话中提取记忆的流水线赛道——在冲突感知、因果推理、置信度校准、澄清提问和记忆忠实性五个维度的表现。实验显示流水线挑战显著：即使在真值记忆下，模型识别冲突的能力优于行为校准与精准追问；而在端到端流水线中，记忆提取无法保留下游推理所需的冲突关系。政策比较表明，固定规则不足以应对需反映冲突的决策。由此提出冲突感知动作策略（CAAP），依据可用证据动态调整行为。TANGLE将冲突处理定义为识别不确定性、保留矛盾证据并行动而不强制给出唯一答案。
原文摘要 · Abstract (English)
LLM agents increasingly maintain personal memory across sessions, but it can conflict. Preferences depend on context, behavior evolves, and sources can conflict. When a query lacks context, time, or source authority to interpret conflict, treating one memory as definitive converts unresolved conflict into an unjustified, overconfident action. Existing benchmarks recover one answer from conflicting evidence, overlooking whether agents recognize underdetermination, preserve alternatives, seek missing information, and choose appropriate actions. We introduce \underline{T}esting \underline{A}gents' \underline{N}avigation of \underline{G}enuine, \underline{L}atent, and \underline{E}ntangled Memory Conflicts (\textsc{TANGLE}), a benchmark for genuinely unresolvable memory conflicts. It comprises 541 instances across 40 personas and three types: Context-Partitioned Conflict (CPC), Behavior-Oscillation Conflict (BOC), and Source-Contradiction Conflict (SCC). We evaluate two tracks---an oracle track with curated memory and a pipeline track that extracts memory from multi-session dialogues---on five dimensions: conflict perception, causal reasoning, confidence calibration, clarification seeking, and memory faithfulness. Experiments reveal pipeline challenges. With curated memory, models recognize conflicts more reliably than they calibrate actions or seek targeted clarification. With end-to-end pipeline memory, extraction fails to preserve conflict-bearing relations needed for downstream reasoning. Policy comparisons show fixed rules are insufficient when actions must reflect conflict. These findings motivate Conflict-Aware Action Policy (CAAP), which adapts actions to each conflict using available evidence. \textsc{TANGLE} frames conflict handling as recognizing underdetermination, retaining conflicting evidence, and acting without forcing a definitive answer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。