解决大模型技能检索中的通用描述干扰问题,提升精准度与速度。
SkillSight: Calibrating Generic Content Bias for Skill Retrieval

- 通过语义和词法双空间校准,消除技能描述中共有的冗余背景信息。
- 在SRA-Bench上召回率提升20.21个百分点,比基线快1248倍。
- 无需训练即可显著改善多智能体系统中的技能选择效果。
随着大型语言模型代理访问的技能库不断增大,准确检索所需技能成为可靠能力选择与执行的关键。现有检索器通常将技能内容视为普通文档,忽视了其高度规律的结构特征:大量技能共享相似的描述模式,但这些模式对区分所需能力帮助甚微。我们发现,这种共有的描述背景会反映在密集相关性得分中,导致查询与技能文档间出现显著的能量差距,并掩盖可区分信号,尤其在结构相似的难例负样本中更为明显。基于此,我们提出SkillSight,一种无需训练的检索框架,可在语义和词法空间中校准共有的背景信息。语义背景校准通过识别由IDF确定的通用词项,估计出一个背景子空间,降低由共享描述模式带来的相似性;词法证据校准则降低共享背景词项的权重,以恢复具有判别性的词级证据。在SRA-Bench和SkillBench-Supp上的实验表明,该方法在各项检索指标上均实现稳定提升,相比原始密集检索器,召回率@10最高提升20.21个百分点。其速度比密集+重排序基线快达1248倍。端到端评估中,SkillSight在三个代理模型上表现最佳,优于LLM Selection高达4.97个百分点。结果表明,共有的描述背景是技能检索中的排名干扰源,而对其校准可实现无需额外训练的精确高效技能选择。代码已公开于https://github.com/xiaojinying/SkillSight。
原文摘要 · Abstract (English)
As large language model agents gain access to increasingly large skill libraries, retrieving the right skill becomes critical to reliable capability selection and execution. Existing retrievers often treat skill contents as ordinary documents, overlooking their highly regular structure: shared descriptive patterns recur across many skills while providing little evidence for distinguishing the required capability. We show that this shared descriptive background is reflected in dense relevance scores, induces a pronounced energy gap between queries and skill documents, and obscures discriminative signals, especially for structurally similar hard negatives. Based on this observation, we propose SkillSight, a training-free retrieval framework that calibrates shared background in both semantic and lexical spaces. Semantic Background Calibration estimates a background subspace from generic tokens identified by IDF, reducing similarity induced by shared descriptive patterns, while Lexical Evidence Calibration downweights shared background tokens to recover discriminative token-level evidence. Experiments on SRA-Bench and SkillBench-Supp demonstrate consistent improvements across retrieval metrics, with SkillSight improving Recall@10 by up to 20.21 percentage points over the original dense retriever. It is up to 1,248 times faster than the Dense + Reranker baseline. In end-to-end evaluation, SkillSight achieves the best overall performance across three agent models and outperforms LLM Selection by up to 4.97 percentage points. These results identify shared descriptive background as a source of ranking interference in skill retrieval and demonstrate that calibrating it enables accurate and efficient skill selection without additional training. Our code can be found at https://github.com/xiaojinying/SkillSight.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。