用强化学习让大模型零样本理解文本网络关系。
TRN-R1-Zero: Text-rich Network Reasoning via LLMs with Reinforcement Learning Only
- 仅靠强化学习优化基础大模型，不依赖标注数据或思维链训练。
- 在多类文本网络上实现零样本推理，跨任务泛化能力强。
- 新设计的奖励机制能动态评估邻居信息价值，促进关系推理。
文本丰富网络（TRN）的零样本推理仍是挑战，因模型需在无特定任务监督下融合文本语义与关系结构。传统图神经网络依赖固定标签空间和监督目标，近期基于大语言模型（LLM）的方法常忽略图上下文或依赖大模型蒸馏，限制泛化能力。本文提出TRN-R1-Zero，一种仅通过强化学习进行后训练的框架。该方法直接优化基础LLM，采用邻近感知的组相对策略优化目标，基于新颖的边际增益度量动态调整奖励，以有效引导模型进行关系推理。相比以往方法，TRN-R1-Zero无需监督微调或来自大型推理模型的思维链数据。在引用、超链接、社交及共购买等多类TRN基准上的大量实验表明其优越性与鲁棒性。此外，仅依赖节点级训练，即可实现边级与图级任务的零样本推理，扩展至跨领域迁移。代码已开源。
原文摘要 · Abstract (English)
Zero-shot reasoning on text-rich networks (TRNs) remains a challenging frontier, as models must integrate textual semantics with relational structure without task-specific supervision. While graph neural networks rely on fixed label spaces and supervised objectives, recent large language model (LLM)-based approaches often overlook graph context or depend on distillation from larger models, limiting generalisation. We propose TRN-R1-Zero, a post-training framework for TRN reasoning trained solely via reinforcement learning. TRN-R1-Zero directly optimises base LLMs using a Neighbour-aware Group Relative Policy Optimisation objective that dynamically adjusts rewards based on a novel margin gain metric for the informativeness of neighbouring signals, effectively guiding the model toward relational reasoning. Unlike prior methods, TRN-R1-Zero requires no supervised fine-tuning or chain-of-thought data generated from large reasoning models. Extensive experiments across citation, hyperlink, social and co-purchase TRN benchmarks demonstrate the superiority and robustness of TRN-R1-Zero. Moreover, relying strictly on node-level training, TRN-R1-Zero achieves zero-shot inference on edge- and graph-level tasks, extending beyond cross-domain transfer. The codebase is publicly available at https://github.com/superallen13/TRN-R1-Zero.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。