通过重写提示范例风格，低成本提升大模型对齐能力
RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars
- 基于风格分析重构提示范例，增强模型对齐表现
- 在Alpaca、Just-eval等数据集上分别提升0.10~0.32分
- 无需训练，适合快速部署与安全可控的对话系统
对齐调优对确保大语言模型行为符合伦理和有用性至关重要。现有方法依赖高质量标注和大量训练资源。本文提出一种低成本、无需调参的基于上下文学习（ICL）的对齐增强方法。通过对高质量ICL示范样本的分析，发现风格是影响模型对齐能力的关键因素，据此构建风格框架并显式重写示范样本。进一步将重写后的示范样本组合，平衡模型对齐中的事实性与安全性矛盾。将重写后的示例作为提示以触发少样本学习，有效提升模型对齐性能。相比最优基线方法（最高得分5.00），本方法在Alpaca任务上最高提升0.10分（4.50→4.60），在Just-eval基准上提升0.22分（4.34→4.56），在MT-Bench数据集上最大提升0.32分（3.53→3.85）。代码与数据已公开于https://github.com/AnonymousCode-ComputerScience/RIDE。
原文摘要 · Abstract (English)
Alignment tuning is crucial for ensuring large language models (LLMs) behave ethically and helpfully. Current alignment approaches require high-quality annotations and significant training resources. This paper proposes a low-cost, tuning-free method using in-context learning (ICL) to enhance LLM alignment. Through an analysis of high-quality ICL demos, we identified style as a key factor influencing LLM alignment capabilities and explicitly restyled ICL exemplars based on this stylistic framework. Additionally, we combined the restyled demos to achieve a balance between the two conflicting aspects of LLM alignment--factuality and safety. We packaged the restyled examples as prompts to trigger few-shot learning, improving LLM alignment. Compared to the best baseline approach, with an average score of 5.00 as the maximum, our method achieves a maximum 0.10 increase on the Alpaca task (from 4.50 to 4.60), a 0.22 enhancement on the Just-eval benchmark (from 4.34 to 4.56), and a maximum improvement of 0.32 (from 3.53 to 3.85) on the MT-Bench dataset. We release the code and data at https://github.com/AnonymousCode-ComputerScience/RIDE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。