金融文档视觉检索中，单向量聚合会丢失关键细节。
A Picture is Worth a Thousand Words? An Empirical Study of Aggregation Strategies for Visual Financial Document Retrieval
- 用图像块拼接代替整图，检测微小数字变化的语义影响
- 单向量聚合使差异文档向量几乎相同，信息严重丢失
- 适用于金融领域视觉检索系统设计者，警示聚合风险
视觉RAG将文档视为图像，使用视觉编码器生成图像块令牌。然而，每份文档数百个块令牌带来向量数据库的存储与检索挑战。实际部署需将其聚合为单一向量。这引发关键问题：单向量聚合是否损失金融文档的关键信息？我们构建诊断基准，利用金融文档中仅数字变化即导致语义显著转变的特性。实验表明，单向量聚合使不同文档的向量几乎一致；块级指标可检测语义变化，证实聚合掩盖了这些细节。根源在于全局纹理主导。该发现贯穿不同模型规模、优化检索的嵌入方法及多种缓解策略，凸显金融应用中单向量视觉文档检索的重大风险。
原文摘要 · Abstract (English)
Visual RAG has offered an alternative to traditional RAG. It treats documents as images and uses vision encoders to obtain vision patch tokens. However, hundreds of patch tokens per document create retrieval and storage challenges in a vector database. Practical deployment requires aggregating them into a single vector. This raises a critical question: does single-vector aggregation lose key information in financial documents? We develop a diagnostic benchmark using financial documents where changes in single digits can lead to significant semantic shifts. Our experiments show that single-vector aggregation collapses different documents with almost identical vectors. Metrics show that the patch level detects semantic changes, and confirm that aggregation obscures these details. We identify global texture dominance as the root cause. Our findings are consistent across model scales, retrieval-optimized embeddings, and multiple mitigation strategies, highlighting significant risks for single-vector visual document retrieval in financial applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。