用检索加权历史行为,让图像生成更懂用户偏好。
RAGAR: Retrieval Augmented Personalized Image Generation Guided by Recommendation
- 根据历史项与参考图的相似度动态加权,提取更精准偏好
- 引入多模态排序任务优化生成,提升个性化表现
- 在三个真实数据集上优于五种基线方法
个性化图像生成对提升用户体验至关重要,它能根据用户的视觉偏好将参考图像转化为心仪样式。然而现有方法存在两大问题:一是将用户历史序列中所有项目同等对待,忽略其与参考图像的语义差异,低相似度项目被赋予过高权重,扭曲用户偏好;二是过度依赖生成图像与参考图像的一致性进行优化,导致用户偏好表达不足,个性化受限。为此,我们提出基于推荐引导的检索增强个性化图像生成方法(RAGAR)。该方法通过检索机制按历史项与参考图像的相似度分配不同权重,更精准提取用户偏好;并引入基于多模态排序模型的新排名任务,替代原有依赖一致性的优化方式,强化个性化生成能力。在三个真实世界数据集上的大量实验及人工评估表明,RAGAR在个性化与语义指标上均显著优于五种基线方法。
原文摘要 · Abstract (English)
Personalized image generation is crucial for improving the user experience, as it renders reference images into preferred ones according to user visual preferences. Although effective, existing methods face two main issues. First, existing methods treat all items in the user historical sequence equally when extracting user preferences, overlooking the varying semantic similarities between historical items and the reference item. Disproportionately high weights for low-similarity items distort users' visual preferences for the reference item. Second, existing methods heavily rely on consistency between generated and reference images to optimize the generation, which leads to underfitting user preferences and hinders personalization. To address these issues, we propose Retrieval Augment Personalized Image GenerAtion guided by Recommendation (RAGAR). Our approach uses a retrieval mechanism to assign different weights to historical items according to their similarities to the reference item, thereby extracting more refined users' visual preferences for the reference item. Then we introduce a novel rank task based on the multi-modal ranking model to optimize the personalization of the generated images instead of forcing depend on consistency. Extensive experiments and human evaluations on three real-world datasets demonstrate that RAGAR achieves significant improvements in both personalization and semantic metrics compared to five baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。