用少量示例示范即可控制搜索结果多样性与公平性，无需重新训练。
Modeling Ranking Properties with In-Context Learning
- 通过演示样例引导大模型调整排序行为，实现无训练控制。
- 在多个任务中提升最多23%的题材多样性，公平性提升20.6%。
- 适合需要灵活调节排序策略的研究者和实际搜索系统开发者。
标准信息检索模型主要优化相关性，但现实搜索还需平衡多样性与公平性等目标。这些目标依赖文档间相互作用，通常通过事后启发式或监督学习解决，需为每种场景和数据集单独训练。本文提出一种基于上下文学习（ICL）的列表级大模型重排序方法，无需额外训练。只需提供少量展示理想权衡的参考排序样例，即可引导模型适应当前查询。我们在TREC Fairness、Touché及TREC Deep Learning 2019/2020等常见测试集上评估，验证了该方法可通过示范工程精确调控排序行为，实现细粒度调整。实验表明，该方法在保持基准相关性的同时，在辅助目标上取得显著提升：主题多样性最高提升23%，公平性最高提升20.6%。
原文摘要 · Abstract (English)
While standard IR models are primarily designed to optimize relevance, real-world search often needs to balance additional objectives such as diversity and fairness. These objectives depend on inter-document interactions and are commonly addressed using post-hoc heuristics or supervised learning methods, which require task-specific training for each ranking scenario and dataset. In this work, we propose an in-context learning (ICL) approach for listwise LLM rerankers that eliminates the need for such training. Instead, our method relies on a small number of example rankings that demonstrate the desired trade-offs between objectives for past queries similar to the current input. We evaluate our approach on common IR test collections to investigate multiple auxiliary objectives: group fairness (TREC Fairness), polarity diversity (Touché), and topical diversity (TREC Deep Learning 2019/2020). We empirically validate that our method enables control over ranking behavior through demonstration engineering, allowing nuanced behavioral adjustments without explicit optimization. Our experiment demonstrates significant improvements in auxiliary objectives, with up to 23\% in topical diversity and 20.6\% fairness gains across different tasks, while maintaining relevance across benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。