给Text-to-SQL加思维链，让直接偏好优化有效提升模型性能
Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL
- 在Text-to-SQL数据中添加合成思维链，使DPO方法生效
- 加入思维链后，模型性能显著提升，且避免奖励劫持
- 适合想提升文本转SQL模型鲁棒性的研究者参考
直接偏好优化（DPO）在数学应用题和代码生成等复杂推理任务中表现优异，但在Text-to-SQL数据集上常无效甚至降低性能。我们发现根本原因在于：与数学和代码任务不同，Text-to-SQL数据通常只包含最终答案（黄金SQL查询），缺乏详细的思维链（CoT）解题过程。通过为Text-to-SQL数据集添加合成的思维链，我们首次实现了使用DPO的一致且显著的性能提升。分析表明，思维链推理对释放DPO潜力至关重要，它能缓解奖励劫持、增强判别能力并提升可扩展性。这些发现为构建更鲁棒的Text-to-SQL模型提供了重要启示。为促进后续研究，我们公开了代码和增强后的数据集。
原文摘要 · Abstract (English)
Direct Preference Optimization (DPO) has proven effective in complex reasoning tasks like math word problems and code generation. However, when applied to Text-to-SQL datasets, it often fails to improve performance and can even degrade it. Our investigation reveals the root cause: unlike math and code tasks, which naturally integrate Chain-of-Thought (CoT) reasoning with DPO, Text-to-SQL datasets typically include only final answers (gold SQL queries) without detailed CoT solutions. By augmenting Text-to-SQL datasets with synthetic CoT solutions, we achieve, for the first time, consistent and significant performance improvements using DPO. Our analysis shows that CoT reasoning is crucial for unlocking DPO's potential, as it mitigates reward hacking, strengthens discriminative capabilities, and improves scalability. These findings offer valuable insights for building more robust Text-to-SQL models. To support further research, we publicly release the code and CoT-enhanced datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。