为代码生成设计专用不确定性评估方法，提升安全性与可靠性。
Code Is More Than Text: Uncertainty Estimation for Code Generation
- 提出三轴不确定性评估：词法、算法、功能一致性
- 平均AUROC提升至0.776，较基线高8.1点
- 单次计算的词法熵已媲美多轮方法，成本更低
大型语言模型（LLMs）被越来越多地用于代码生成，但生成错误代码会带来真实的安全与可靠性风险。可靠的不确定性估计（UE）对选择性预测、人机协同审查及下游智能体决策至关重要。然而，现有代码不确定性评估方法大多源自自然语言生成，忽略了代码的独特属性。我们指出代码与自然语言有三方面差异：单个错误标记可导致整个程序失效（标记脆弱性）；算法意图与具体实现可独立偏离（意图-代码差距）；程序可执行（可执行性）。据此构建三个正交的不确定性维度：词法（Top-K标记熵）、算法（伪代码一致性）、功能（行为一致性）。在五种代码LLM上，三轴集成方法将平均AUROC从最强的自然语言基线0.696提升至0.776（+8.1点）。尤其在Qwen3-14B上，单次计算的Top-K词法熵已达到最强多轮基线性能，且成本降低超3倍；跨模型表现始终具有竞争力。结果表明，代码不确定性评估应采用专用设计，而非直接沿用自然语言方法。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly deployed as code generators, where silently wrong programs pose real safety and reliability risks. Reliable uncertainty estimation (UE) is essential for selective prediction, human-in-the-loop review, and downstream agentic decisions. Yet most existing code UE methods are inherited from natural language (NL) generation and ignore properties that make code distinct. We argue that code differs from NL in three ways: a single wrong token can break an entire program (token fragility); algorithmic intent and concrete implementation can disagree independently (intent-code gap); and programs can be executed (executability). We instantiate these properties as three orthogonal uncertainty axes: lexical (Top-K token entropy), algorithmic (pseudo-code consistency), and functional (behavioral consistency). Across five code LLMs, our three-axis ensemble improves average AUROC from 0.696 for the strongest NL-derived baseline to 0.776 (+8.1 points). Notably, on Qwen3-14B, our single-pass Top-K token entropy matches the strongest multi-pass baseline while being over 3x cheaper; across models, it remains a competitive low-cost signal. These results suggest that code UE deserves code-specific design rather than direct NL ports.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。