不确定时才重排序，省计算还更准
When Reranking Hurts: Uncertainty-Based Gating for Few-Shot Reranking
- 根据模型不确定性决定是否重排序，避免无意义计算
- 在NLU任务上降低54%计算量，机器翻译降20%
- 适合资源有限但需高效推理的场景
少样本选择通常假设重排序总能提升性能。我们指出，昂贵的重排序步骤反而可能降低效果。为此提出无需训练的门控重排序方法，依据模型不确定性判断是否进行重排序。在8个大语言模型、7个自然语言理解数据集和18个机器翻译领域-方向组合上的实验表明，在校准操作点下，该方法使机器翻译平均计算成本降低约20%，自然语言理解降低54%，同时保持与全量重排序相当的性能。结果表明，更高计算开销不等于更好性能，重排序仅在高不确定性实例上最有效。
原文摘要 · Abstract (English)
Few-shot selection typically assumes that reranking retrieved examples always improves performance. We challenge this view by identifying that the expensive reranking step can in fact degrade performance. Instead, we propose \emph{Training-Free Gated Reranking}, which decides whether to rerank the few-shot examples based on the model's uncertainty. Extensive experiments across 8 LLMs, covering 7 NLU datasets and 18 MT domain-direction combinations, demonstrate that our approach reduces average computational costs by approximately 20\% for MT and 54\% for NLU at a calibrated operating point, while matching the performance of full reranking. These findings indicate that higher computational cost does not guarantee better performance, and that reranking is most beneficial when targeted at high-uncertainty instances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。