解决生成模型训练中的模式坍缩问题，提升分子生成多样性与有效性。
Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training
- 通过根部吸收前缀轨迹平衡，增强早期生成阶段的学习信号。
- 结合子模重播策略，显著提升分子生成的多样性与有效性。
- 适用于大语言模型生成化学分子结构，适合药物研发场景。
生成流网络（GFlowNets）可微调大语言模型以近似奖励比例后验分布，但易发生模式坍缩，表现为前缀坍缩和长度偏差。我们归因于两点：(i) 早期前缀信用分配弱；(ii) 重播机制导致训练流分布偏移。提出根部吸收前缀轨迹平衡（RapTB），在根部锚定子轨迹监督，并通过吸收后缀回溯将终端奖励传递至中间前缀，提供密集的前缀级学习信号。为缓解重播引发的分布偏移，进一步引入子模重播（SubM）策略，促进高奖励与多样性。实验表明，在使用SMILES字符串生成分子的任务中，RapTB结合SubM持续提升优化性能与分子多样性，同时保持高有效率。代码已开源。
原文摘要 · Abstract (English)
Generative Flow Networks (GFlowNets) enable fine-tuning large language models to approximate reward-proportional posteriors, but they remain prone to mode collapse, manifesting as prefix collapse and length bias. We attribute this to two factors: (i) weak credit assignment to early prefixes, and (ii) biased replay that induces a shifted, non-representative training flow distribution. We propose Rooted absorbed prefix Trajectory Balance RapTB, an objective that anchors subtrajectory supervision at the root and propagates terminal rewards to intermediate prefixes via absorbed suffix-based backups, providing dense prefix-level learning signals. To mitigate replay-induced distribution shift, we further introduce SubM, a submodular replay refresh strategy that promotes both high reward and diversity. Empirically, on tasks such as molecule generation with LLM using SMILES strings, RapTB combined with SubM consistently improves optimization performance and molecular diversity while preserving high validity. The code is released on https://github.com/ComDec/ChemGFN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。