小模型通过精调可超越大模型，揭示嵌入表示的粒度困境
Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddings
- 构建新数据集CapRetrieval，测试细粒度语义匹配能力
- 0.1B小模型经微调后超越7B主流模型性能
- 发现嵌入需平衡细粒度特征与整体语义对齐的矛盾
本文源于对文本编码器的一个观察：嵌入表示可能无法识别编码语义中的细粒度实体或事件，导致简单查询也失败。为检验此类行为，我们引入新评估数据集CapRetrieval，其段落为图像描述，查询为指向不同形式实体或事件概念的短语。零样本评估显示，无论训练来源或模型规模如何，编码器在细粒度匹配上普遍表现不佳。为提升性能，我们采用提出的数据生成策略微调编码器，使小型0.1B模型超越当前最优的7B模型。在此过程中，进一步揭示了粒度困境：嵌入需同时捕捉细粒度显著性与整体语义一致性。相关数据集、代码与模型已公开于https://github.com/lxucs/CapRetrieval。
原文摘要 · Abstract (English)
This work stems from an observed limitation of text encoders: embeddings may not be able to recognize fine-grained entities or events within encoded semantics, resulting in failed retrieval even in simple cases. To examine such behaviors, we first introduce a new evaluation dataset, CapRetrieval, in which passages are image captions and queries are phrases targeting entity or event concepts in diverse forms. Zero-shot evaluation suggests that encoders often struggle with these fine-grained matching, regardless of training sources or model size. Aiming for enhancement, we proceed to finetune encoders with our proposed data generation strategies, enabling a small 0.1B encoder to outperform the state-of-the-art 7B model. Within this process, we further uncover the granularity dilemma, a challenge for embeddings to capture fine-grained salience while aligning with overall semantics. Our dataset, code and models in this work are publicly released at https://github.com/lxucs/CapRetrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。