构建股票问答新基准，评估模型对历史与预测数据的综合推理能力
STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Data
- 基于4417只股票构建端到端问答基准，涵盖历史与预测数据
- 大模型在历史查询上表现良好，但预测推理仍存显著挑战
- 适合研究金融智能体、工具增强型推理的学者使用
股票市场分析需要对历史记录和未来预测进行综合推理，但现有基准分散于孤立任务。我们提出STQA（面向股票的表格问答），一个端到端基准，系统评估自然语言问答在历史数据、数值预测及基于预测的推理能力。基于大规模金融数据集，STQA覆盖4,417只股票，包含31,400个由专家设计模板生成的问答对，并附有细粒度意图与槽位标注。为落地该基准，我们提出SQFRS（股票查询-预测-推理系统），一种基于代理的统一框架，协调SQL检索与时间序列预测工具。实验表明，当前大模型在历史查询上表现良好，但预测推理面临重大挑战，暴露出工具协同与不确定性下的推理瓶颈。数据与代码已公开于https://github.com/xuxubaobaoan/STQA_Project。STQA为可信、工具增强型金融智能体的研究提供了严格测试平台。
原文摘要 · Abstract (English)
Stock market analysis inherently requires composite reasoning over historical records and future projections, yet existing benchmarks remain fragmented across isolated tasks. We introduce STQA (Stock-focused Tabular Question Answering), an end-to-end benchmark designed to systematically evaluate natural-language question answering over historical data, numerical forecasts, and forecast-based reasoning. Built on a large-scale financial dataset, STQA covers 4,417 stocks and contains 31,400 question-answer pairs derived from expert-crafted templates, accompanied by fine-grained intent and slot annotations. To operationalize this benchmark, we present SQFRS (Stock Query-Forecast-Reasoning System), an agent-based unified framework that orchestrates SQL retrieval and time-series forecasting tools. Experiments demonstrate that while current large language models perform well on historical queries, forecast-based reasoning poses a substantial challenge, revealing critical bottlenecks in tool coordination and reasoning under uncertainty. The dataset and code are available at https://github.com/xuxubaobaoan/STQA_Project. STQA thus serves as a rigorous testbed for future research on trustworthy, tool-augmented financial agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。