研究大模型对话状态追踪中上下文学习的影响力因素
Factors affecting the in-context learning abilities of LLMs for dialogue state tracking
- 用句向量+K近邻筛选演示样本，构造提示模板输入大模型
- 在MultiWoZ2.4数据集上验证了不同演示选择策略对性能的影响
- 为对话系统设计提供了可操作的上下文学习优化方向
本研究探索将上下文学习（ICL）应用于对话状态跟踪（DST）问题，并分析影响其效果的因素。采用基于句向量的K近邻方法检索适合的演示样本，将选定的演示与测试样本按模板组合成输入，送入大语言模型。在MultiWoZ2.4数据集上，针对OLMo-7B-instruct、Mistral-7B-Instruct-v0.3和Llama3.2-3B-Instruct模型开展系统性实验，分析演示选择与提示上下文相关因素对DST性能的影响。研究结果揭示了大模型在对话状态追踪中上下文学习能力的关键影响因素，为实际应用提供实用洞见。
原文摘要 · Abstract (English)
This study explores the application of in-context learning (ICL) to the dialogue state tracking (DST) problem and investigates the factors that influence its effectiveness. We use a sentence embedding based k-nearest neighbour method to retrieve the suitable demonstrations for ICL. The selected demonstrations, along with the test samples, are structured within a template as input to the LLM. We then conduct a systematic study to analyse the impact of factors related to demonstration selection and prompt context on DST performance. This work is conducted using the MultiWoZ2.4 dataset and focuses primarily on the OLMo-7B-instruct, Mistral-7B-Instruct-v0.3, and Llama3.2-3B-Instruct models. Our findings provide several useful insights on in-context learning abilities of LLMs for dialogue state tracking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。