发现大模型评分受输出格式影响，根源在于判断信号与格式分支的耦合。
Judge Circuits
- 通过因果分析定位共享判断子图，存在于中后层MLP中。
- 零消融该子图使评分失效，但保留世界知识，证明其关键作用。
- 揭示评分不一致本质是格式分支脆弱性，适合模型可解释性研究者。
大语言模型作为评分器已成为大规模评估模型输出的主流范式，但同一模型在不同输出格式下（如1-5分制与真/假标签）会给出系统性差异评分。现有研究仅停留在输入输出层面的诊断。我们采用位置感知边归因修补法（PEAP），对Gemma-3、Qwen2.5和Llama-3进行因果探究，发现结构化理解与开放式偏好任务中的判断共享一个稀疏且通用的潜在评估子图，位于中至后层多层感知机（MLPs）中；对该子图进行零消融会导致评分崩溃，但保留了模型的常识知识。通过结构性解耦抽象判断与输出格式，我们揭示了所研究开源模型中格式诱导不一致的机制：连续的判断信号在共享主干中计算，再经由脆弱的、格式特定的终端分支映射，使独立于格式的偏好可下游分离。研究暗示，跨格式的基准可靠性比较部分反映的是格式几何，而非评估质量。
原文摘要 · Abstract (English)
LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same model assigns systematically different scores when its output format changes (e.g., a 1-5 rating vs. a True/False label). Existing diagnoses of these format-induced inconsistencies stop at the input-output level. Using Position-aware Edge Attribution Patching (PEAP), we causally investigate the internal mechanism in Gemma-3, Qwen2.5, and Llama-3. We find that judgments across structured understanding and open-ended preference tasks share a sparse, generalized Latent Evaluator sub-graph in the mid-to-late multi-layer perceptrons (MLPs); zero-ablating it collapses judgment while preserving world knowledge in architecturally modular models. By structurally decoupling abstract judging from output formatting, we provide a mechanistic account of format-induced inconsistency on the open-weight models we study: a continuous judgment signal computed in the shared trunk is mapped through fragile, format-specific terminal branches, enabling format-independent preference to be isolated downstream of the requested output format. Our findings imply that benchmark-level reliability comparisons across formats are partially measuring formatter geometry rather than evaluation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。