让大模型像人一样理解表格数据，提升推理能力。
Toward Structured Knowledge Reasoning: Contrastive Retrieval-Augmented Generation on Experience
- 构建经验记忆库，通过对比学习增强模型泛化能力
- 在Text-to-SQL和TableQA任务上平均提升3.44%至4.24%
- 无需训练，持续优化，适合结构化数据推理场景
大型语言模型（LLMs）在纯文本任务中表现优异，但在表格、数据库等结构化数据任务上表现不佳，原因在于预训练阶段对结构化数据接触不足，以及文本到结构的转换机制僵化。与人类能跨模态灵活迁移知识不同，现有模型难以推断表格中的隐含关系，尤其缺乏显式结构引导时。为此，我们提出对比检索增强生成经验框架（CoRE），通过构建经验记忆表示，并利用对比上下文学习（ICL）模拟人类知识迁移，以弥合认知差距。在Text-to-SQL和TableQA任务上的实验表明，CoRE显著提升性能，平均增益达3.44%和4.24%，部分挑战任务提升高达17.2%。基于蒙特卡洛树搜索（MCTS）生成的经验记忆库使训练数据扩展8-9倍，提升多样性与领域覆盖。该方法无需训练、可持续迭代，推动大模型向结构化知识专家演进。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong performance on plain text tasks but underperform on structured data like tables and databases. Potential challenges arise from their underexposure during pre-training and rigid text-to-structure transfer mechanisms. Unlike humans who seamlessly apply learned patterns across data modalities, LLMs struggle to infer implicit relationships embedded in tabular formats, especially in the absence of explicit structural guidance. To bridge this cognitive gap, we introduce Contrastive Retrieval-Augmented Generation on Experience (CoRE), a framework that builds experience memory representations and enhances generalization through contrastive In-Context Learning (ICL) to simulate human-like knowledge transfer. Experiments on Text-to-SQL and TableQA show CoRE significantly improves performance, achieving average gains of 3.44% and 4.24%, with up to 17.2% on challenging tasks. Our Monte Carlo Tree Search (MCTS)-generated Experience Memory expands training data 8-9x, enhancing diversity and domain coverage. This training-free and continual method propels LLMs toward structured knowledge expertise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。