提出CoCo方法，精准解析专家模型如何评判回答。
Beyond Routing Weights: Faithful Response-Level Interpretation of Mixture-of-Experts Reward Models via Contribution Contrast
- 用对比响应对捕捉专家实际判断行为，而非仅看路由权重。
- 在自动与人工评估中，解释结果更一致、更专注。
- 适合研究奖励模型可解释性或改进强化学习系统的人。
奖励模型在基于人类偏好的学习中至关重要，但其预测依据难以识别。现有的稀疏混合专家（MoE）奖励模型通过将提示路由至专用专家并利用高路由权重示例来刻画专家，但路由权重仅反映专家接收了哪些提示，无法揭示其如何评判回答，解释不完整。为此，我们提出贡献对比（CoCo）方法，通过选择-拒绝响应对中贡献差异最大的样本，联合捕捉专家的路由与偏好行为，实现响应级别的忠实解释。在自动与人工评估中，CoCo生成的解释比基于路由、得分或稀疏自编码器的方法更连贯、更真实、更具专业性，同时保持竞争力的奖励建模精度。据我们所知，这是首个系统研究MoE奖励模型解释方法的工作。
原文摘要 · Abstract (English)
Reward models are central to learning from human preferences, yet identifying what drives their predictions remains challenging. Recent sparse Mixture-of-Experts (MoE) reward models seek to improve interpretability by routing prompts to specialized experts and characterizing experts through examples with high routing weights. However, routing weights only reveal which prompts an expert $\textit{receives}$, not how it $\textit{judges}$ responses, providing only a partial account of expert behavior. We therefore propose $\textbf{Co}$ntribution-$\textbf{Co}$ntrast ($\textbf{CoCo}$) response-level interpretation, which faithfully characterizes experts' roles using chosen-rejected response pairs with the largest contribution contrasts, jointly capturing routing and preference behavior. Across automatic and human evaluations, CoCo yields more coherent, faithful, and specialized interpretations than router-based, score-based, and sparse autoencoder-based alternatives while maintaining competitive reward modeling accuracy. To the best of our knowledge, this is the first systematic study of interpretation methods for MoE reward models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。