arXiv:2412.07246cs.CLcs.DB2024-12AAAI被引 5

不靠真实数据重放,用语法差异引导大模型缓解持续学习遗忘问题。

Filling Memory Gaps: Enhancing Continual Semantic Parsing via SQL Syntax Variance-Guided LLMs without Real Data Replay

  • 基于SQL语法差异分析,引导大模型重建关键记忆
  • 任务感知双教师蒸馏提升知识积累与迁移能力
  • 在无数据重放下超越理想设置方法,适配未知任务

持续语义解析(CSP)旨在用少量标注样本训练解析器,将自然语言问题转为SQL,以适应数据库动态更新的真实场景。以往方法通过重放历史数据或参数高效微调缓解此问题,但常违反数据隐私或依赖理想持续学习设定。为此,我们提出一种新型大语言模型增强的持续语义解析方法LECSP,无需真实数据重放或理想设定即可缓解遗忘并促进泛化。具体地,该方法从SQL语法角度分析任务间共性与差异,指导大模型重构关键记忆,并通过校准策略提升记忆准确性;随后采用任务感知的双教师蒸馏框架,促进序列训练中的知识累积与迁移。在两个CSP基准上的实验表明,我们的方法显著优于现有方法,甚至超越使用数据重放或理想设定的方法。此外,我们实现了超出上限的泛化性能,更优地适应未见任务。

原文摘要 · Abstract (English)

Continual Semantic Parsing (CSP) aims to train parsers to convert natural language questions into SQL across tasks with limited annotated examples, adapting to the real-world scenario of dynamically updated databases. Previous studies mitigate this challenge by replaying historical data or employing parameter-efficient tuning (PET), but they often violate data privacy or rely on ideal continual learning settings. To address these problems, we propose a new Large Language Model (LLM)-Enhanced Continuous Semantic Parsing method, named LECSP, which alleviates forgetting while encouraging generalization, without requiring real data replay or ideal settings. Specifically, it first analyzes the commonalities and differences between tasks from the SQL syntax perspective to guide LLMs in reconstructing key memories and improving memory accuracy through a calibration strategy. Then, it uses a task-aware dual-teacher distillation framework to promote the accumulation and transfer of knowledge during sequential training. Experimental results on two CSP benchmarks show that our method significantly outperforms existing methods, even those utilizing data replay or ideal settings. Additionally, we achieve generalization performance beyond the upper limits, better adapting to unseen tasks.

持续学习语义解析大模型SQL生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。