通过几何约束提升量化模型高温度推理时的准确性与真实性。
Tethered Reasoning: Decoupling Entropy from Hallucination in Quantized LLMs via Manifold Steering
- 用预计算的可信流形约束隐藏状态轨迹，分离熵与幻觉。
- 4比特量化下高温推理准确率下降仅1.24-2.81个百分点。
- 仅修改0.2%-2.5%令牌即可实现高熵创意输出，适合高效生成场景。
量化语言模型面临根本困境：低采样温度导致重复与模式坍缩，高温度（T > 2.0）引发轨迹发散与语义不连贯。我们提出HELIX，一种几何框架，通过将隐藏状态轨迹锚定至预计算的可信流形，实现输出熵与幻觉的解耦。HELIX计算融合词级语义熵与流形距离的统一可信得分（UTS），当检测到轨迹偏离时，渐进式引导向结构一致区域，仅影响0.2%-2.5%的令牌。在4比特量化Granite 4.0 H Small（32B/9B活跃，混合Mamba-Transformer）上，GSM8K在T=3.0时保持88.84%准确率（较T=0.5仅降2.81pp）；MMLU在14,042个问题上维持72.49%准确率（降1.24pp）。结果表明，高温度幻觉主要源于轨迹发散而非语义坍缩。值得注意的是，仅引导约10%的稀疏Transformer注意力层即可修正Mamba-2状态空间的漂移。几何锚定揭示了此前被掩盖的高熵创意储备：在T > 2.0时，引导输出的创意重复率仅为5%-20%，远低于保守设置下的70%-80%。跨架构验证（Qwen3-30B-A3B MOE）证实该现象与架构无关，独特概念生成量提升46.7%。HELIX作为语法锚点，允许在不破坏逻辑骨架的前提下探索语义多样性，实现多温度合成，生成的独特概念比单温度推断高出200%。
原文摘要 · Abstract (English)
Quantized language models face a fundamental dilemma: low sampling temperatures yield repetitive, mode-collapsed outputs, while high temperatures (T > 2.0) cause trajectory divergence and semantic incoherence. We present HELIX, a geometric framework that decouples output entropy from hallucination by tethering hidden-state trajectories to a pre-computed truthfulness manifold. HELIX computes a Unified Truth Score (UTS) combining token-level semantic entropy with Mahalanobis distance from the manifold. When UTS indicates trajectory divergence, graduated steering vectors redirect activations toward structurally coherent regions while affecting only 0.2-2.5% of tokens. On 4-bit quantized Granite 4.0 H Small (32B/9B active, hybrid Mamba-Transformer): GSM8K maintains 88.84% accuracy at T = 3.0 (2.81pp degradation from T = 0.5); MMLU maintains 72.49% across 14,042 questions (1.24pp degradation). This demonstrates that high-temperature hallucination is primarily trajectory divergence rather than semantic collapse. Notably, steering the sparse Transformer attention layers (~10% of layers) is sufficient to correct drift in the Mamba-2 state-space formulation. Geometric tethering reveals a previously-masked High-Entropy Creative Reservoir. At T > 2.0, steered outputs exhibit 5-20% idea duplication versus 70-80% at conservative settings. Cross-architecture validation (Qwen3-30B-A3B MOE) confirms this phenomenon is architecture-independent, with 46.7% higher unique concept generation. HELIX acts as a syntax tether, enabling exploration of semantic diversity without violating the logical backbone required for valid output. This enables Multi-Temperature Synthesis, generating 200% more unique concepts than single-temperature inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。