构建渐进式文本转SQL数据集，用于研究大模型的可解释性机制。
TinySQL: A Progressive Text-to-SQL Dataset for Mechanistic Interpretability Research
- 设计从简单到复杂的合成数据集TinySQL，支持渐进式研究。
- 在33M至1B参数模型上验证了最小电路的可识别性与可靠性。
- 适合研究模型内部结构、可解释性方法对比的学者使用。
机制可解释性研究面临从简单玩具任务分析到大型模型特征发现之间的鸿沟。为弥合这一差距，我们提出将文本转SQL生成作为理想研究任务，因其兼具玩具任务的形式结构与真实世界复杂性。本文引入合成数据集TinySQL，逐步涵盖基础至高级的SQL操作，并训练33M至1B参数规模的模型，建立全面的可解释性测试基准。应用多种互补的可解释性技术，包括边缘归因修补（Edge Attribution Patching）与稀疏自编码器（Sparse Autoencoders），识别支持SQL生成的最小电路与组件。通过对比不同SQL子技能对应的电路，评估其最小性、可靠性和可识别性。最后，采用逐层对数透镜分析揭示模型在各层如何组合生成查询：从意图识别，到模式解析，再到结构化生成。本工作为在结构化、渐进复杂场景下探查与比较可解释性方法提供了稳健框架。
原文摘要 · Abstract (English)
Mechanistic interpretability research faces a gap between analyzing simple circuits in toy tasks and discovering features in large models. To bridge this gap, we propose text-to-SQL generation as an ideal task to study, as it combines the formal structure of toy tasks with real-world complexity. We introduce TinySQL, a synthetic dataset, progressing from basic to advanced SQL operations, and train models ranging from 33M to 1B parameters to establish a comprehensive testbed for interpretability. We apply multiple complementary interpretability techniques, including Edge Attribution Patching and Sparse Autoencoders, to identify minimal circuits and components supporting SQL generation. We compare circuits for different SQL subskills, evaluating their minimality, reliability, and identifiability. Finally, we conduct a layerwise logit lens analysis to reveal how models compose SQL queries across layers: from intent recognition to schema resolution to structured generation. Our work provides a robust framework for probing and comparing interpretability methods in a structured, progressively complex setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。