针对IPO尽职调查设计新基准，提升大模型金融分析能力。
IPO Finance Agent: Benchmark of LLM Financial Analysts Beyond Finance Agent v2, with Automated Rubric Generation, on the SpaceX (SPCX) IPO
- 构建基于S-1文件的IPO尽调任务框架，改进长文档检索机制。
- 引入自动评估规则生成流水线，提升评测严谨性与可复现性。
- 在太空探索公司SPCX案例中验证，模型成本效率超越现有基准。
Finance Agent v2（Vals AI）已成为评估Anthropic Claude和OpenAI ChatGPT等前沿语言模型在金融任务上表现的参考基准。然而，该基准仅聚焦公开上市公司定期报告（如SEC 10-K和10-Q），其代理架构依赖原始分块检索，未能应对IPO尽职调查的独特挑战。SEC S-1文件包含历史财务报表、治理结构、拟议会计处理、资本形成叙事及承销敏感型风险披露，文档长度远超常规周期性报告。为此，本文提出IPO Finance Agent，从任务领域和检索架构两方面扩展Finance Agent v2。实验发现，原框架对SpaceX（SPCX）S-1文件几乎无输出，因文档过长。因此我们引入上下文感知检索，更贴近行业实践。同时构建了1000个IPO尽调问题数据集，公开70个SPCX S-1问题以支持复现，其余保密以防基准污染。此外，提出评估-优化流水线：从模型回答中提取候选事实，整合为初版评分标准，经LLM反馈自动审计遗漏、幻觉、混淆项与冗余，驱动迭代修正与精炼，人类专家仅审核最终版本。结果表明，最优模型Zhipu GLM-5.2达79.8%准确率；最高效模型Xiaomi MiMo-2.5 Pro以77.2%准确率实现0.05美元/查询成本，显著优于当前Finance Agent v2榜首（Google Gemini 3.5 Flash，57.9%，2.51美元/查询），并超越其最低成本条目（MiniMax M3，48.3%，0.32美元/查询）的成本效益。代码与数据已开源于GitHub。
原文摘要 · Abstract (English)
Finance Agent v2 (by Vals AI) has emerged as the reference benchmark for evaluating both Anthropic Claude and OpenAI ChatGPT frontier language models on financial tasks. However, it narrowly deals with periodic reporting from publicly traded companies (SEC 10-K and 10-Q filings), and its agentic harness relies on naive, unenriched chunk retrieval. Neither the task design nor the retrieval approach addresses the distinct challenges of IPO due diligence. SEC S-1 filings combine historical financial statements, governance structures, pro forma and common-control accounting treatments, capital-formation narratives, and underwriting-sensitive risk disclosures within substantially longer documents than typical periodic filings. That is why we introduce IPO Finance Agent, which extends the Finance Agent v2 framework along two directions: task domain and retrieval architecture. During our experiments, the original Finance Agent v2 harness basically failed to deliver any output related to the SpaceX S-1 filing, due to document length. We therefore had to improve the agentic harness with contextual retrieval, a more realistic and industry-standard approach for long documents. We also built a dataset of 1,000 IPO-diligence questions, and publicly release 70 questions on the SpaceX (SPCX) S-1 filing to support reproducibility, while the remainder are held private to guard against benchmark contamination. In addition, we introduce an evaluator-optimizer pipeline to automatically generate evaluation rubrics for the benchmark: candidate facts are extracted from model answers, consolidated into draft criteria, then automatically audited for omissions, hallucinations, mistiered items, and redundancy, with LLM feedback driving iterative repair, targeted enrichment, and deduplication. Human experts only review final rubrics before deployment. Results show that the best-performing evaluated model, Zhipu GLM-5.2, reaches 79.8% accuracy, and the most cost-efficient model on the resulting Pareto frontier, Xiaomi MiMo-2.5 Pro, reaches slightly lower accuracy (77.2%) at 0.05 USD per query, while exceeding the current Finance Agent v2 leaderboard ceiling, Google Gemini 3.5 Flash at 57.9% for 2.51 USD per query, and undercutting even FABv2's cheapest entry (MiniMax M3: 48.3% at 0.32 USD) on cost-efficiency. Code and data are released on GitHub https://github.com/benstaf/ipoagent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。