用双向上下文评估文本，消除传统模型的顺序偏差。
DiffScore: Text Evaluation Beyond Autoregressive Likelihood
- 采用掩码重建机制，每个词都基于完整上下文评分。
- 在十项基准测试中均优于自回归模型，零样本和微调皆有效。
- 可分解流畅性与忠实度，适合质量分析与模型调试。
自回归语言模型广泛用于文本评估，但其从左到右的因子分解引入了位置偏差：早期词仅依赖左侧上下文，将架构不对称误认为文本质量。我们提出掩码重建作为替代范式，使每个词都基于完整双向上下文评分。为此，我们构建DiffScore框架，基于掩码大扩散语言模型，通过测量不同掩码率下的文本可恢复性，消除位置偏差，并自然建立从局部流畅性到全局连贯性的评估层级。我们还提供了自回归框架无法实现的诊断工具：多时间步质量曲线（跨掩码率分解评分）与双向PMI分解（分离流畅性与忠实度）。在十个基准上的实验表明，DiffScore在零样本与微调设置下均持续优于自回归基线。代码已公开：https://github.com/wenlai-lavine/DiffScore。
原文摘要 · Abstract (English)
Autoregressive language models are widely used for text evaluation, however, their left-to-right factorization introduces positional bias, i.e., early tokens are scored with only leftward context, conflating architectural asymmetry with true text quality. We propose masked reconstruction as an alternative paradigm, where every token is scored using full bidirectional context. We introduce DiffScore, an evaluation framework built on Masked Large Diffusion Language Models. By measuring text recoverability across continuous masking rates, DiffScore eliminates positional bias and naturally establishes an evaluation hierarchy from local fluency to global coherence. We further provide diagnostic tools unavailable to autoregressive frameworks: multi-timestep quality profiles that decompose scores across masking rates, and bidirectional PMI decomposition that disentangles fluency from faithfulness. Experiments across ten benchmarks show that DiffScore consistently outperforms autoregressive baselines in both zero-shot and fine-tuned settings. The code is released at: https://github.com/wenlai-lavine/DiffScore.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。