arXiv:2412.11404cs.CLcs.AI2024-12ACL被引 6

用依存句法增强注意力,让大模型生成内容的每处引用都精准可追溯。

Attention with Dependency Parsing Augmentation for Fine-Grained Attribution

  • 用集合并操作保留证据粒度,避免信息丢失
  • 引入依存句法提升目标片段语义完整性,定位更准
  • 适用于各类基于模型内部特征的方法,通用性强

为帮助人类高效验证RAG生成内容,构建细粒度溯源机制、为每个回答片段提供来自检索文档的支撑证据至关重要。现有方法依赖模型内部相似性度量(如显著性分数和隐藏状态相似性),但存在计算复杂度高或表征粗粒度的问题。此外,多数工作采用仅解码器的Transformer结构,难以在目标片段后融入上下文信息。为此,我们提出两种适用于所有基于模型内部特征的方法:首先,通过集合并操作聚合词级证据,保持表征粒度;其次,融合依存句法分析以增强目标片段的语义完整性。实际实现中,采用注意力权重作为相似性度量。实验表明,该方法在各项指标上均持续优于已有方法。

原文摘要 · Abstract (English)

To assist humans in efficiently validating RAG-generated content, developing a fine-grained attribution mechanism that provides supporting evidence from retrieved documents for every answer span is essential. Existing fine-grained attribution methods rely on model-internal similarity metrics between responses and documents, such as saliency scores and hidden state similarity. However, these approaches suffer from either high computational complexity or coarse-grained representations. Additionally, a common problem shared by the previous works is their reliance on decoder-only Transformers, limiting their ability to incorporate contextual information after the target span. To address the above problems, we propose two techniques applicable to all model-internals-based methods. First, we aggregate token-wise evidence through set union operations, preserving the granularity of representations. Second, we enhance the attributor by integrating dependency parsing to enrich the semantic completeness of target spans. For practical implementation, our approach employs attention weights as the similarity metric. Experimental results demonstrate that the proposed method consistently outperforms all prior works.

细粒度溯源注意力机制依存句法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。