发现评估通道切换会导致奖励方向反转，提出可定位问题的量化指标
Agent Step Value: Auditing Evaluator-Channel Reversals in Black-Box Agent Traces
- 定义代理步值ASV，衡量不同评估通道下的奖励变化
- 在医学问答任务中，生成视角奖励均值为负，直接评分则为正
- 可定位到生成视角坐标处的奖励反转，适合模型审计与可靠性研究
对评估通道切换下步骤奖励方向是否保持的问题进行审计。相同冻结的转换过程可能违背奖励方向不变的假设。过程奖励受代理状态影响，评估审计则随评分配置变化；两者的一阶差分无法分离其交互。本文定义了代理步值（ASV），作为通道索引的目标边际增益，并在完整匹配的样本上识别出状态-通道交互。在冻结的PubMed问答转换中，直接评分的平均ASV为正，而生成视角的平均ASV为负。两次匹配重播实验重现了这一反转现象，跨通道符号不一致比同通道重试不一致高出48.0个百分点。匹配检索样本将反转定位至生成视角坐标，并追踪其通过读取-堆叠桥的传播路径。仅源端生成协议在含伪影检索中恢复正向平均方向，并从无伪影前状态视图中移除解析器检测到的实质性支持主张。ASV将通道敏感性转化为可定位、可干预的测量问题，可在步骤奖励复用前进行检验。
原文摘要 · Abstract (English)
Pooling, substituting, or reusing evaluator-derived step rewards assumes that their direction survives a change of evaluation channel. The same frozen transition can violate that assumption. Process rewards vary agent states, while evaluator audits vary scoring configurations; neither first difference isolates their interaction. We define Agent Step Value (ASV) as a channel-indexed target-margin gain and identify the state-by-channel interaction on complete matched faces. Across frozen PubMed question-answering transitions, direct scoring yields a positive mean ASV, while the generated-view channel yields a negative mean. Two matched replay waves reproduce this reversal, and cross-channel sign disagreement exceeds same-channel retry disagreement by 48.0 percentage points. Matched retrieval faces localize the reversal to the generated-view coordinate and trace its direction across a readout-and-stack bridge. A source-only generation contract restores the positive mean direction on artifact-bearing retrievals and removes parser-detected substantive support claims from artifact-free before-state views. ASV turns channel sensitivity into an identified measurement problem that can be localized and tested by intervention before step rewards are reused.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。