用模块化课程学习提升复杂SQL生成能力,效果优于传统微调。
LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL

- 分层训练不同难度适配器,逐步构建复杂逻辑生成能力。
- 在Spider和BIRD数据集上,复杂查询准确率显著提升。
- 灵活可组合的架构适合不同数据库复杂度场景使用。
近期,面向代码的大型语言模型在将自然语言转化为可执行代码方面展现出强大能力。文本转SQL是其重要应用,使非技术人员可通过自然语言与关系型数据库交互。然而,当前先进模型在处理高度复杂的逻辑时仍表现不佳,尤其是涉及多重嵌套连接与条件的语句,以及噪声大或结构不良的真实数据库模式。本文探究课程学习是否能提升基于代码的LLM在文本转SQL任务上的表现。在Spider和BIRD基准上,我们采用不同课程策略对模型进行微调。实验表明,简单按复杂度排序的课程学习因灾难性遗忘而无法超越标准微调。为此,我们提出模块化适配器组合(MAC)策略:在逐级复杂度层次(简单至超难)上顺序训练专属适配器,构建分步学习环境,有效提升复杂查询生成性能。该方法不仅在Spider和BIRD上取得可测量的性能提升,还提供一种灵活的‘乐高式’架构,支持根据具体模式难度需求组合与部署模型。结果表明,结构化、模块化的学习方式是掌握复杂代码生成语法与逻辑的更优选择。
原文摘要 · Abstract (English)
Recently, code-oriented large language models (LLMs) have demonstrated strong capabilities in translating natural language into executable code. Text-to-SQL is a significant application of this ability, enabling non-technical users to interact with relational databases using natural language. However, state-of-the-art models continue to struggle with highly complex logic, particularly deeply nested statements involving multiple joins and conditions, as well as with real-world database schemas that are noisy or poorly structured. In this paper, we investigate whether curriculum learning can improve the performance of code-based LLMs on Text-to-SQL tasks. Employing benchmarks including Spider and BIRD, we fine-tune models under different curriculum strategies. Our experiments show that naive curriculum, simply ordering training samples by complexity in a single epoch, fails to surpass standard fine-tuning due to catastrophic forgetting. To overcome this, we propose a Modular Adapter Composition (MAC) strategy. By sequentially training tier-specific adapters on incremental complexity levels (Easy to Extra-Hard), we create a scaffolded learning environment that improves performance on complex queries. Our approach not only produces measurable performance gains on the Spider and BIRD benchmarks but also provides a flexible, "Lego-like" architecture, allowing models to be composed and deployed based on specific schema difficulty requirements. These findings demonstrate that structured, modular learning is a superior alternative to monolithic fine-tuning for mastering the syntax and logic of complex code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。