arXiv:2603.23231cs.AI2026-03被引 11

评测大模型长期记忆中个性化的持续一致性。

PERMA: Benchmarking Personalized Memory Agents via Event-Driven Preference and Realistic Task Environments

  • 按时间顺序设计多轮跨域交互事件,模拟真实用户偏好演化。
  • 引入文本变异与语言风格对齐,提升评估场景真实性。
  • 揭示当前记忆系统在跨域干扰下仍难保持连贯个性。

赋予大模型长期记忆对构建能适应用户需求变化的智能体至关重要。现有评估常将偏好对话与无关内容混杂,使任务退化为大海捞针,忽略驱动偏好演化的事件关联。真实个性化中,偏好是逐步积累、在嘈杂上下文中形成的。为此,我们提出PERMA基准,用于评估模型在时间维度上的个性一致性,而非静态偏好召回。该基准包含跨会话、跨领域的时序交互事件,随时间插入偏好相关查询,并设计多项选择与交互式任务以探测模型对人物轨迹的理解。实验表明,通过关联相关交互,先进记忆系统可精准提取偏好并降低令牌消耗,优于传统语义检索;但其在长时间跨度和跨域干扰下仍难以维持连贯个性,凸显个性化记忆管理的不足。代码与数据已开源:https://github.com/PolarisLiu1/PERMA。

原文摘要 · Abstract (English)

Empowering large language models with long-term memory is crucial for building agents that adapt to users' evolving needs. Existing evaluations of this capability typically interleave preference-related dialogues with irrelevant conversations, reducing the task to needle-in-a-haystack retrieval while ignoring relationships between events driving user preference evolution. Such settings overlook a fundamental characteristic of real-world personalization: preferences emerge gradually and accumulate across interactions within noisy contexts. To bridge this gap, we introduce PERMA, a benchmark designed to evaluate persona consistency over time beyond static preference recall. Additionally, we incorporate (1) text variability and (2) linguistic alignment to simulate erratic user inputs and individual idiolects in real-world data. PERMA consists of temporally ordered interaction events spanning multiple sessions and domains, with preference-related queries inserted over time. We design both multiple-choice and interactive tasks to probe the model's understanding of persona along the interaction timeline. Experiments demonstrate that by linking related interactions, advanced memory systems extract precise preferences and reduce token consumption, outperforming traditional semantic retrieval of raw dialogues. Nevertheless, they still struggle to maintain a coherent persona across temporal depth and cross-domain interference, highlighting the need for more robust personalized memory management in agents. Our code and data are open-sourced at https://github.com/PolarisLiu1/PERMA.

个性化长期记忆基准测试对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。