金融多模态检索生成新基准，支持时序分析与跨模态理解。
Towards Temporal-Aware Multi-Modal Retrieval Augmented Generation in Finance
- 将表格、新闻、股价、图表转为文本，构建时序感知图索引。
- 覆盖日、周、月、季、年五类时间维度的10种金融任务。
- 适合金融智能问答、财报分析等场景的模型开发者使用。
金融决策依赖对多种数据源的深度分析，包括财务表格、新闻文章、股价走势等。本文提出FinTMMBench，首个面向金融领域时序感知多模态检索增强生成（RAG）系统的综合性评估基准。该基准基于纳斯达克100公司异构数据构建，具有三大优势：1）多模态语料库：包含财务表格、新闻文章、每日股价及视觉技术图表；2）时序感知问题：每个问题需在特定时间段（日、周、月、季度、年度）内检索并解读相关数据；3）多样化的金融分析任务：由领域专家设计的10类任务，涵盖信息抽取、趋势分析、情感分析和事件检测等。此外，我们提出TMMHybridRAG方法，先利用大模型将表格、图像、时间序列等多模态数据转化为文本，再在构建图结构与密集索引时引入时间信息。实验验证其有效性，但仍存在显著差距，揭示了本基准所提出挑战的复杂性。
原文摘要 · Abstract (English)
Finance decision-making often relies on in-depth data analysis across various data sources, including financial tables, news articles, stock prices, etc. In this work, we introduce FinTMMBench, the first comprehensive benchmark for evaluating temporal-aware multi-modal Retrieval-Augmented Generation (RAG) systems in finance. Built from heterologous data of NASDAQ 100 companies, FinTMMBench offers three significant advantages. 1) Multi-modal Corpus: It encompasses a hybrid of financial tables, news articles, daily stock prices, and visual technical charts as the corpus. 2) Temporal-aware Questions: Each question requires the retrieval and interpretation of its relevant data over a specific time period, including daily, weekly, monthly, quarterly, and annual periods. 3) Diverse Financial Analysis Tasks: The questions involve 10 different financial analysis tasks designed by domain experts, including information extraction, trend analysis, sentiment analysis and event detection, etc. We further propose a novel TMMHybridRAG method, which first leverages LLMs to convert data from other modalities (e.g., tabular, visual and time-series data) into textual format and then incorporates temporal information in each node when constructing graphs and dense indexes. Its effectiveness has been validated in extensive experiments, but notable gaps remain, highlighting the challenges presented by our FinTMMBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。