提出测试多步离线目标条件强化学习分割不变性的基准，发现人为分割点严重影响性能。
SegBench-GC: Testing Segmentation Invariance in Multi-Step Offline Goal-Conditioned Reinforcement Learning
- 通过控制人工分割边界设计压力测试，隔离分割对学习的影响。
- 未处理分割时成功率降至19.1%，而保留延续价值的方案仍保持39.1%。
- 适用于研究离线强化学习中轨迹结构敏感性与算法鲁棒性的研究人员。
离线目标条件强化学习（GCRL）常依赖轨迹结构进行未来目标采样和多步目标计算，但已记录轨迹可能因管理原因被人为分割，而非真实终止。本文提出SegBench-GC，一种受控的压力测试，固定转移、源轨迹、目标采样、优化设置和评估方式，仅改变人工回溯边界及是否保留延续价值。延续有效目标（CVT）提供分割一致性对照：奖励累积在人工切口处停止，但目标从存储的后续状态回溯。在匹配计数的PointMaze实验中，35,000个人工切口、三种分割实现和三个优化种子下，最终每任务50次试验的成功率分别为：未切割50.5%、使用CVT为39.1%、若相同切口视为吸收态则为19.1%；不同分割实现下，朴素平均成功率介于4.8%至31.9%之间。独立发布的n=25基线（来自Decoupled Q-Chunking代码库）在Puzzle-4x5上同样显示失败：未切割47.2%、CVT为58.5%、朴素方法仅为0.27%。目标级诊断验证了分析目标差与数值精度一致；学习批评器诊断表明，朴素处理导致显著乐观偏差，而CVT则与未切割批评器基本对齐。CVT采用标准延续回溯，无需新贝尔曼规则；贡献在于受控基准、失败归因与跨学习器证据，证明行政分割可显著影响多步离线GCRL结果。
原文摘要 · Abstract (English)
Offline goal-conditioned reinforcement learning (GCRL) often uses trajectory structure for future-goal sampling and multi-step targets, yet logged trajectories may be partitioned for administrative reasons that do not correspond to termination. We introduce SegBench-GC, a controlled stress test of segmentation invariance that holds transitions, source trajectories, goal sampling, optimization settings, and evaluation fixed while varying only artificial backup boundaries and whether those boundaries retain continuation value. Continuation-valid targets (CVT) provide the segmentation-consistent control: reward accumulation stops at an artificial cut, but the target bootstraps from its stored successor. In a matched-count PointMaze study with 35,000 artificial cuts, three segmentation realizations, and three optimization seeds, final 50-episode-per-task success is 50.5% uncut, 39.1% with CVT, and 19.1% when the same cuts are treated as absorbing; across segmentation realizations, naive mean success ranges from 4.8% to 31.9%. An independent published n-step baseline (n=25) from the Decoupled Q-Chunking codebase shows the same failure on Puzzle-4x5: 47.2% uncut, 58.5% CVT, and 0.27% naive across three optimization seeds. A target-level diagnostic verifies the analytic target difference to numerical precision, and learned-critic diagnostics show a large optimistic shift under naive handling while CVT remains approximately aligned with the uncut critic. CVT applies standard continuation bootstrapping rather than a new Bellman rule; the contribution is the controlled benchmark, failure isolation, and cross-learner evidence that administrative segmentation can materially change multi-step offline GCRL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。