arXiv:2609.02913cs.IR2026-09

针对教育问答中的课程逻辑错位问题,提出前置过滤与融合检索框架。

CHSR-RRF: A curriculum-gated hybrid retrieval framework with reciprocal rank fusion and leakage-aware benchmarking for educational RAG

论文配图:CHSR-RRF: A curriculum-gated hybrid retrieval framework with reciprocal rank fusion and leakage-aware benchmarking for educational RAG
图 1 · 摘自论文原文
  • 用课程元数据提前过滤候选文档,避免错误知识点引入
  • 预检索阶段门控使泄漏降低4.6倍,召回率不变;后置约束则导致召回归零
  • 构建126例带层级标签的评测集,揭示数据缺失是主要失败原因

检索增强生成(RAG)在教育问答中日益普及,但标准检索器仅优化主题相关性,未考虑课程有效性。在教学场景中,若内容来自错误学科、年级或考试背景,即使语义相关也属不当,这种现象称为课程泄漏。本文提出CHSR-RRF框架,通过元数据约束在检索前实施课程门控,结合稀疏与稠密检索并使用倒数排名融合与确定性重排序。同时构建了包含126个案例的CERB评测基准,具备层次感知的相关性标签和明确的泄漏标注。在61例初步测试中,预检索门控使泄漏降低4.6倍(p<0.001),且保持排名召回率;而检索后施加相同约束则导致召回率与精确范围成功率降至零(p=0.039)。全基准下界分析进一步表明,诸多失败源于语料库和元数据缺口,而非检索设计本身。结果表明,结构化教育领域的检索应视为受限选择,有效性应在候选池形成时即被强制执行,而非排序之后。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) is increasingly used in educational question answering, but standard retrievers optimize topical relevance without enforcing curriculum validity. In school settings, a passage can be relevant yet inappropriate if it comes from the wrong subject, level, or examination context; we call this failure mode curriculum leakage. We present CHSR-RRF, a curriculum-gated hybrid retrieval framework that applies metadata constraints before retrieval, then combines sparse and dense search with reciprocal rank fusion and deterministic reranking. We also introduce CERB, a 126-case benchmark for curriculum-constrained retrieval with hierarchy-aware relevance labels and explicit leakage annotations. On a 61-case pilot, pre-retrieval gating reduces leakage by 4.6x ($p<0.001$) while preserving ranked recall, whereas applying the same constraints after retrieval collapses recall and exact-scope success to zero ($p=0.039$). A full-benchmark lower-bound analysis further shows that many remaining failures arise from corpus and metadata gaps rather than retrieval design alone. These results show that retrieval in structured educational domains should be treated as constrained selection, with validity enforced when the candidate pool is formed rather than after ranking.

教育AI检索增强课程对齐评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。