小模型也能自我修正对话状态，省算力还高效
CorrectionLM: Self-Corrections with SLM for Dialogue State Tracking
- 用上下文示例让小模型自己纠错，无需大模型参与
- 低资源下对话状态追踪效果接近顶尖大模型
- 适合算力有限但需高精度对话系统的场景
大型语言模型（LLMs）通过反馈和优化展现了自我提升能力，但现有小型语言模型（SLMs）在该领域进展有限。当前的纠错方法通常依赖从大模型中蒸馏知识，带来巨大计算开销。本文提出 CORRECTIONLM，一种新型纠错框架，使 SLM 能够利用上下文示例实现自我修正，无需 LLM 参与。该方法应用于两个低资源条件下的对话状态追踪（DST）任务，在仅消耗极小计算成本的情况下，达到与前沿 LLM 相当的性能。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated self-improvement capabilities via feedback and refinement, but current small language models (SLMs) have had limited success in this area. Existing correction approaches often rely on distilling knowledge from LLMs, which imposes significant computation demands. In this work, we introduce CORRECTIONLM, a novel correction framework that enables SLMs to self-correct using in-context exemplars without LLM involvement. Applied to two dialogue state tracking (DST) tasks in low-resource settings, CORRECTIONLM achieves results similar to a state-of-the-art LLM at a small fraction of the computation costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。