针对扩散语言模型量化中决策不稳问题，提出新校准方法提升稳定性。
FAIR-Calib: Frontier-Aware Instability-Reweighted Calibration for Post-Training Quantization of Diffusion Large Language Models
- 分两阶段校准：先估计易错决策位置，再加权保护脆弱状态
- 在LLaDA和Dream数据集上显著减少决策翻转，性能优于现有方法
- 无需复杂推理即可实现，适合部署资源受限的扩散语言模型
扩散大语言模型（dLLMs）通过迭代优化词元并不可逆地固化结果，导致早期决策即使写入后仍存在脆弱性，形成‘稳定性滞后’。我们发现，后训练量化（PTQ）误差容易在写入边界处翻转这些临界决策，且错误被永久锁定并放大。为此，提出前沿感知不稳定性重加权校准（FAIR-Calib），一种针对dLLMs的两阶段PTQ框架。第一阶段利用全精度教师模型估算位置先验，融合前沿命中与掩码阶段可靠性；第二阶段采用离策略、逐层校准，最小化重加权隐藏状态MSE，有效优先保护脆弱的前沿状态，且无需昂贵的端到端扩散推理。我们进一步从理论上证明该加权目标是输出KL散度的代理。实验表明，FAIR-Calib在LLaDA和Dream（W4A4）上持续超越最先进基线，显著降低前沿决策翻转，并抑制多种基准上的提交后不匹配现象。
原文摘要 · Abstract (English)
Diffusion Large Language Models (dLLMs) refine tokens iteratively but commit them irreversibly, leading to a "stability lag" where early decisions remain fragile even after being written. We reveal that Post-Training Quantization (PTQ) error easily flips these borderline decisions at the write frontier, which are then permanently locked in and amplified. To address this, we propose Frontier-Aware Instability-Reweighted Calibration (FAIR-Calib), a two-stage PTQ framework for dLLMs. Stage I probes a full-precision teacher to estimate a position prior that combines frontier hits and masked-stage reliability. Stage II performs off-policy, layer-wise calibration by minimizing a reweighted hidden-state MSE, effectively prioritizing the protection of fragile frontier states without requiring expensive end-to-end diffusion rollouts. We further theoretically justify our weighted objective as a surrogate for output KL divergence. Empirically, FAIR-Calib consistently outperforms state-of-the-art baselines on LLaDA and Dream (W4A4), significantly reducing frontier decision flips and suppressing post-commit mismatches across diverse benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。