解决企业级多表复杂表格的智能推理难题,提升大模型理解能力。
From Rows to Reasoning: A Retrieval-Augmented Multimodal Framework for Spreadsheet Understanding
- 将表格分解为行、列、块嵌入,结合文本与视觉信息检索增强生成
- 在400万单元格数据上达74%准确率,较之前提升50个百分点
- 适合处理含图表的大型多表工作簿,对金融、审计等场景特别有用
大型语言模型在包含数千行数值数据、多张关联表格及嵌入图表、收据等视觉内容的企业级电子表格上难以进行有效推理。现有先进方法多依赖单表压缩或全上下文编码,限制了可扩展性,且无法反映真实用户操作方式。我们提出首个大规模多模态表格推理基准FRTR-Bench,涵盖30个企业级Excel工作簿,覆盖近四百万单元格和超过50个嵌入图像。为此,我们提出从行到推理(FRTR)框架,将工作簿分解为细粒度的行、列、块嵌入,采用混合词法-密集检索与倒数排名融合(RRF),整合多模态嵌入以同时处理数值与视觉信息。在六个LLM上测试,使用Claude Sonnet 4.5在FRTR-Bench上达到74%答案准确率,显著优于此前24%的最优水平;在SpreadsheetLLM基准上,使用GPT-5实现87%准确率,同时相比直接序列化方法减少约50%的令牌使用量。
原文摘要 · Abstract (English)
Large Language Models (LLMs) struggle to reason over large-scale enterprise spreadsheets containing thousands of numeric rows, multiple linked sheets, and embedded visual content such as charts and receipts. Prior state-of-the-art spreadsheet reasoning approaches typically rely on single-sheet compression or full-context encoding, which limits scalability and fails to reflect how real users interact with complex, multimodal workbooks. We introduce FRTR-Bench, the first large-scale benchmark for multimodal spreadsheet reasoning, comprising 30 enterprise-grade Excel workbooks spanning nearly four million cells and more than 50 embedded images. To address these challenges, we present From Rows to Reasoning (FRTR), an advanced, multimodal retrieval-augmented generation framework that decomposes Excel workbooks into granular row, column, and block embeddings, employs hybrid lexical-dense retrieval with Reciprocal Rank Fusion (RRF), and integrates multimodal embeddings to reason over both numerical and visual information. We tested FRTR on six LLMs, achieving 74% answer accuracy on FRTR-Bench with Claude Sonnet 4.5, a substantial improvement over prior state-of-the-art approaches that reached only 24%. On the SpreadsheetLLM benchmark, FRTR achieved 87% accuracy with GPT-5 while reducing token usage by roughly 50% compared to direct serialization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。