低资源下提升Text2SQL模型效率，减少40%显存占用
LR-SQL: A Supervised Fine-Tuning Method for Text2SQL Tasks under Low-Resource Scenarios
- 分块数据库训练，动态组合表以降低上下文长度
- 显存减少40%，表预测准确率仅降2%，整体执行准确率下降0.6%
- 适合资源受限场景下的Text2SQL模型微调，尤其对大库任务友好
大型语言模型通过监督微调推动Text2SQL发展，但数据库复杂性导致上下文过长，显著增加显存需求。为解决此问题，本文提出LR-SQL，包含模式链接（schema_link）与SQL生成（SQL_generation）两个微调模型，其中模式链接模型为核心。在微调过程中，LR-SQL将完整数据库拆分为可灵活调整数量的表片段，使模型从分散切片中学习全局关系；同时，在推理阶段引入链式思维能力，增强对离散片段间关系的感知。实验表明，相比现有方法，LR-SQL可降低40%的总显存使用量，模式链接任务中表预测准确率仅下降2%，整体Text2SQL任务的执行准确率下降0.6%。项目代码已开源。
原文摘要 · Abstract (English)
Large language models revolutionize Text2SQL through supervised fine-tuning, yet a crucial limitation is overlooked: the complexity of databases leads to an increased context length, consequently resulting in higher GPU memory demands for model fine-tuning. To address this issue, we propose LR-SQL. LR-SQL comprises two supervised fine-tuning models: the schema\_link model and the SQL\_generation model, with the schema\_link model serving as the focal point for streamlining the overall process. During the fine-tuning of the schema\_link model, LR-SQL breaks down the complete database into flexible combinations of tables with adjustable quantities, enabling the model to learn the relationships within the entire database from these dispersed slices. Furthermore, to enhance the model's ability to perceive the relationships among various discrete slices during inference, LR-SQL trains the model's Chain-of-Thought capability for this task. Experimental results demonstrate that LR-SQL can reduce the total GPU memory usage by 40\% compared to existing fine-tuning methods, while only losing 2\% of table prediction accuracy in schema\_link task. For the overall Text2SQL task, the Execution Accuracy decrease by 0.6\%.Our project is now available on https://github.com/hongWin/LR-SQL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。