arXiv:2609.01645cs.IR2026-09

检索系统因语言风格差异,让低收入用户难获公共福利信息。

The Vocabulary Gap Is an Equity Gap: Register Mismatch in Retrieval Systems for Public-Benefits Access

  • 构建双语对照基准,对比官方与普通人提问的检索效果
  • 普通表达查询召回率仅36-44%,差距达56个百分点
  • 发现词汇匹配度下降是主因,可用简单词表修复

检索增强型问答系统被用于帮助公众查询福利资格,但系统检索的文档使用政府正式语体,而用户常以通俗、非正式或非母语英语提问。我们证明这种语言风格不匹配会将高性能检索系统变为不公平工具。构建了包含51条联邦福利规则和25个信息需求的受控基准,每个需求在官方语体与普通用户语体下各表述一次,固定黄金段落。在BM25、TF-IDF和词图检索器上,正式语体测试几乎完美(Recall@5 96-100%),而普通语体检索大幅下滑(Recall@5 36-44%)。以BM25为例,Recall@1从84%降至16%,Recall@5从100%降至44%,形成56点公平性差距。机制分析显示:正式查询与原文共享0.63内容词,普通查询仅0.11,降低5.9倍。设计一个简洁可审计的普通到正式词表桥接方案,使普通查询的BM25 Recall@5从44%提升至80%。贡献在于评估协议、基准、机理诊断与透明缓解方法,揭示标准检索评估所掩盖的高影响社会性失效模式。

原文摘要 · Abstract (English)

Retrieval-augmented question answering is increasingly used to help people navigate public-benefits eligibility, yet the documents these systems retrieve from are written in agency register while intended users often ask questions in plain, informal, or non-native English. We show that this register mismatch can turn a high-performing retrieval system into an inequitable one. We construct a controlled benchmark of 51 publicly documented federal benefit-eligibility rules and 25 information needs, each phrased in both agency register and plain user register while keeping the gold passage fixed. Across BM25, TF-IDF, and a term-graph retriever, formal-register evaluation is nearly perfect (Recall@5 96-100%), but plain-register retrieval collapses (Recall@5 36-44%). For BM25, Recall@1 falls from 84% to 16% and Recall@5 from 100% to 44%, a 56-point equity gap on identical information needs. We trace the mechanism to a measurable vocabulary gap: formal queries share 0.63 of their content terms with the gold passage, while plain queries share only 0.11, a 5.9x reduction. A deliberately simple, auditable plain-to-formal lexicon bridge recovers much of the failure, lifting plain-query BM25 Recall@5 from 44% to 80%. The contribution is not a new retriever; it is an evaluation protocol, benchmark, mechanistic diagnosis, and transparent mitigation for a high-stakes social-impact failure mode that standard retrieval evaluation hides.

检索公平性语言风格公共政策信息获取

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。