arXiv:2604.13074cs.CLcs.CV2026-04中稿 · CVPR被引 9

让AI长期记住用户偏好,持续调整回应风格。

PersonaVLM: Long-Term Personalized Multimodal LLMs

论文配图:PersonaVLM: Long-Term Personalized Multimodal LLMs
图 1 · 摘自论文原文
  • 构建记忆库动态捕捉用户多轮交互的个性化信息。
  • 在128k上下文下比基线提升22.4%,超越GPT-4o 5.2%。
  • 适合需要长期互动的智能助手场景,如私人顾问。

多模态大语言模型(MLLMs)作为数百万用户的日常助手,但其生成内容与个人偏好对齐的能力有限。现有方法仅支持静态、单轮个性化,无法捕捉用户随时间演化的偏好与人格特征。本文提出PersonaVLM,一种面向长期个性化的多模态代理框架,通过三大能力实现个性化:(a) 记忆:主动从交互中提取并总结时序多模态记忆,构建个性化数据库;(b) 推理:通过检索和融合数据库中的相关记忆进行多轮推理;(c) 响应对齐:在长期互动中推断用户演变的人格,确保输出保持一致性。为评估,我们建立Persona-MME基准,包含超过2000个精心设计的交互案例,涵盖七个核心维度与14项细粒度任务。大量实验验证该方法有效性,在128k上下文下相较基线提升22.4%(Persona-MME)和9.8%(PERSONAMEM),且优于GPT-4o 5.2%和2.0%。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) serve as daily assistants for millions. However, their ability to generate responses aligned with individual preferences remains limited. Prior approaches enable only static, single-turn personalization through input augmentation or output alignment, and thus fail to capture users' evolving preferences and personality over time (see Fig.1). In this paper, we introduce PersonaVLM, an innovative personalized multimodal agent framework designed for long-term personalization. It transforms a general-purpose MLLM into a personalized assistant by integrating three key capabilities: (a) Remembering: It proactively extracts and summarizes chronological multimodal memories from interactions, consolidating them into a personalized database. (b) Reasoning: It conducts multi-turn reasoning by retrieving and integrating relevant memories from the database. (c) Response Alignment: It infers the user's evolving personality throughout long-term interactions to ensure outputs remain aligned with their unique characteristics. For evaluation, we establish Persona-MME, a comprehensive benchmark comprising over 2,000 curated interaction cases, designed to assess long-term MLLM personalization across seven key aspects and 14 fine-grained tasks. Extensive experiments validate our method's effectiveness, improving the baseline by 22.4% (Persona-MME) and 9.8% (PERSONAMEM) under a 128k context, while outperforming GPT-4o by 5.2% and 2.0%, respectively. Project page: https://PersonaVLM.github.io.

个性化长时记忆多模态智能助手

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。