arXiv:2603.01493cs.IRcs.AI2026-03KDD被引 1

构建首个真实个人相册检索基准,推动从视觉匹配到意图驱动的跨源推理。

PhotoBench: Beyond Visual Matching Towards Personalized Intent-Driven Photo Retrieval

  • 基于多源信息融合的画像框架,整合视觉、时空、社交与事件数据。
  • 发现统一嵌入模型在非视觉约束下表现退化,代理系统工具调度能力差。
  • 适合研究个性化多媒体检索、智能代理与跨模态推理的学者使用。

个人相册不仅是静态图像的集合,更是由时间连续性、社会关联和丰富元数据定义的动态生态档案,使得个性化图片检索极具挑战。然而,现有检索基准严重依赖孤立的网络快照,无法捕捉解决真实用户意图查询所需的多源推理。为此,我们提出 PhotoBench,首个基于真实个人相册构建的基准,旨在推动从视觉匹配向个性化多源意图驱动推理的范式转变。基于严谨的多源画像框架,该框架整合每张图片的视觉语义、时空元数据、社交身份与时间事件信息,生成根植于用户人生轨迹的复杂意图驱动查询。在 PhotoBench 上的广泛评估揭示了两个关键局限:模态鸿沟——统一嵌入模型在非视觉约束下失效;源融合悖论——代理系统在工具编排上表现不佳。这些发现表明,个人多模态检索的下一前沿超越统一嵌入,亟需具备精确约束满足与多源融合能力的稳健代理推理系统。我们的 PhotoBench 已公开可用。

原文摘要 · Abstract (English)

Personal photo albums are not merely collections of static images but living, ecological archives defined by temporal continuity, social entanglement, and rich metadata, which makes the personalized photo retrieval non-trivial. However, existing retrieval benchmarks rely heavily on context-isolated web snapshots, failing to capture the multi-source reasoning required to resolve authentic, intent-driven user queries. To bridge this gap, we introduce PhotoBench, the first benchmark constructed from authentic, personal albums. It is designed to shift the paradigm from visual matching to personalized multi-source intent-driven reasoning. Based on a rigorous multi-source profiling framework, which integrates visual semantics, spatial-temporal metadata, social identity, and temporal events for each image, we synthesize complex intent-driven queries rooted in users' life trajectories. Extensive evaluation on PhotoBench exposes two critical limitations: the modality gap, where unified embedding models collapse on non-visual constraints, and the source fusion paradox, where agentic systems perform poor tool orchestration. These findings indicate that the next frontier in personal multimodal retrieval lies beyond unified embeddings, necessitating robust agentic reasoning systems capable of precise constraint satisfaction and multi-source fusion. Our PhotoBench is available.

图像检索多源融合意图驱动个人相册

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。