用知识图谱构建科学文献的自对弈推理框架,提升多跳问答能力。
SPARK: Self-Play with Asymmetric Reward from Knowledge Graphs

- 基于多文档知识图谱生成关系推理题,结构化路径支撑问题构造。
- 在跨文档多跳问答上超越基线,跳跃次数越多优势越明显。
- 适合研究多模态推理、知识增强AI的学者关注。
自对弈强化学习在数学和编程等具有明确规则的领域表现优异,但在科学文献中面临挑战:文本中多模态元素间的关系通常未显式表达,导致关系推理问题难以自动生成,奖励信号可靠性弱。我们提出SPARK(基于知识图谱的非对称奖励自对弈框架),通过自动构建统一知识图谱(KG)作为结构基础,利用KG路径生成关系推理问题,以图中存储的结构化事实提供可验证的奖励计算依据。一个小型视觉-语言模型(sVLM)在固定KG下交替扮演提议者与求解者角色,实现信息不对称下的自对弈。我们在公开基准和自建跨文档多跳问答数据集上评估,结果表明SPARK持续优于基于扁平语料的自对弈基线,且随着跳跃层数增加,性能差距扩大,说明知识图谱结构能有效支持超越无结构语料的复杂关系推理。
原文摘要 · Abstract (English)
Self-play reinforcement learning has shown strong performance in domains with formally verifiable structure, such as mathematics and coding, where both problem generation and reward computation can be grounded in explicit rules. Extending this paradigm to scientific literature is more challenging: the relationships among multi-modal elements within and across documents are rarely made explicit in text, which makes automatic generation of relational reasoning questions difficult and weakens the reliability of reward signals. We propose SPARK (Self-Play with Asymmetric Reward from Knowledge Graphs), a framework that automatically constructs a unified knowledge graph (KG) from multi-document scientific literature and uses it as the structural basis for self-play. KG paths over multimodal nodes serve as a source for generating relational reasoning questions, and structured facts stored in the KG provide a basis for verifiable reward computation. A single small vision-language model (sVLM) alternates between Proposer and Solver roles under information asymmetry against a fixed KG, a design that we believe can be naturally extended toward online adaptation in future work. We evaluate SPARK on public benchmarks and a self-constructed cross-document multi-hop QA dataset. Results show that SPARK consistently outperforms flat-corpus-based self-play baselines, and the performance gap widens as hop count increases, suggesting that KG-structure grounding contributes to relational multi-hop reasoning beyond what unstructured corpus grounding can provide.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。