根据输入自适应组合提示词,提升社会偏见检测效果
Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection
- 针对不同输入动态选择最优提示组合,而非固定模板
- 在三个数据集上表现优于单个提示方法,最高提升12.3%
- 适用于依赖语境的复杂任务,适合做偏见检测研究者参考
指令微调的进展催生了多种大模型提示技术,如显式推理步骤。然而,这些技术的效果受任务类型、语言模型及上下文影响,有效提示往往需反复试验。现有自动提示方法多优化单一技术,未考虑技术组合及其对输入的依赖性。为此,我们提出一种自适应提示方法,能为给定输入即时预测最优提示组合。应用于高度依赖语境的社会偏见检测任务,我们在三个大模型和三个数据集上进行评估,对比组合方法与单一技术及其他基线。结果表明,寻找有效提示组合至关重要。所提方法在多个设置中稳定保持高检测性能,表现最佳。初步实验显示其在其他任务上也具泛化能力。
原文摘要 · Abstract (English)
Recent advances on instruction fine-tuning have led to the development of various prompting techniques for large language models, such as explicit reasoning steps. However, the success of techniques depends on various parameters, such as the task, language model, and context provided. Finding an effective prompt is, therefore, often a trial-and-error process. Most existing approaches to automatic prompting aim to optimize individual techniques instead of compositions of techniques and their dependence on the input. To fill this gap, we propose an adaptive prompting approach that predicts the optimal prompt composition ad-hoc for a given input. We apply our approach to social bias detection, a highly context-dependent task that requires semantic understanding. We evaluate it with three large language models on three datasets, comparing compositions to individual techniques and other baselines. The results underline the importance of finding an effective prompt composition. Our approach robustly ensures high detection performance, and is best in several settings. Moreover, first experiments on other tasks support its generalizability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。