让大模型通过无监督训练学会查资料、推理，回答更准确。
Learning to Reason and Use Tools through Unsupervised Fine-Tuning in Task-Oriented Dialog Systems
- 用上下文学习自动收集推理路径，再用大模型筛选高质量数据。
- 80亿参数模型超越700亿参数的原生系统，提升推理与工具使用能力。
- 适合需要精准问答、动态信息检索的对话系统开发者。
当前对话系统在动态信息检索中表现不佳，常出现幻觉且响应准确率低。本文针对任务导向对话场景，改进ReAct框架，使大语言模型能调用外部知识并生成事实性回复。核心提出一种无监督微调流程：通过上下文学习推理获取推理轨迹，再用大模型作为判别器筛选高质量样本，构建稳健训练集；并引入无监督自提升循环，使优化后的检查点持续生成更优轨迹用于下一轮微调。在SIMMC数据集上的实验表明，基于ReAct的系统因更强的推理和工具使用能力优于基线。值得注意的是，经过微调的80亿参数模型性能超过未微调的700亿参数系统。最后还进行了错误分析、场景复杂度影响及跨领域泛化研究。
原文摘要 · Abstract (English)
Current dialogue systems struggle with dynamic information retrieval, often leading to hallucinations and lower response accuracy. We address this by adapting the ReAct framework for Task-Oriented Dialogue, enabling Large Language Models (LLMs) to access external knowledge and produce factual responses. Mainly, we propose an unsupervised fine-tuning pipeline that harvests reasoning trajectories via in-context learning inference. High-quality samples are filtered using an LLM-based judge to construct a robust training set. This is enhanced by a unsupervised self-improvement loop, where improved checkpoints generate increasingly better trajectories for subsequent fine-tuning iterations. Experiments on the SIMMC dataset demonstrate that ReAct-based systems outperform baselines due to superior reasoning and tool use. Notably, our fine-tuned 8B model surpasses a 70B in-context system. Finally, we present an error analysis, impact of scene complexity, and cross-domain generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。