arXiv:2601.19934cs.CLcs.AI2026-01被引 1

发现大模型即使固定参数也会输出不一致,揭示了系统性随机漂移现象。

Quantifying non deterministic drift in large language models

  • 通过重复运行实验,量化相同提示下模型输出的非确定性漂移。
  • 在温度0.0和0.7下,gpt-4o-mini与llama3.1-8b均存在显著输出差异。
  • 适用于关注模型可复现性、部署稳定性的研究人员与工程师。

大型语言模型广泛应用于摘要生成到决策支持等任务。实践中,即使固定温度等解码参数,相同提示也常产生不同输出。本文通过重复运行实验,实证量化了无操作干预条件下的基准行为漂移,即相同提示多次输入时的输出变异性。评估了两个公开模型gpt-4o-mini和llama3.1-8b,在五类提示下分别采用精确重复、扰动输入和重用模式,温度为0.0和0.7。使用唯一输出比例、词汇相似度和词数统计衡量漂移,实现跨模型、提示模式与部署方式的直接比较。结果表明,即使在温度0.0下,非确定性仍持续存在,且随模型规模、部署方式和提示类型呈现不同漂移模式。研究将发现置于概念漂移、行为漂移及基础设施引发非确定性现有工作背景中,讨论词汇度量局限性,并强调新兴语义方法。通过建立无稳定化技术下的系统性实证基线,为未来漂移缓解与控制方法提供参考点。

原文摘要 · Abstract (English)

Large language models (LLMs) are widely used for tasks ranging from summarisation to decision support. In practice, identical prompts do not always produce identical outputs, even when temperature and other decoding parameters are fixed. In this work, we conduct repeated-run experiments to empirically quantify baseline behavioural drift, defined as output variability observed when the same prompt is issued multiple times under operator-free conditions. We evaluate two publicly accessible models, gpt-4o-mini and llama3.1-8b, across five prompt categories using exact repeats, perturbed inputs, and reuse modes at temperatures of 0.0 and 0.7. Drift is measured using unique output fractions, lexical similarity, and word count statistics, enabling direct comparison across models, prompting modes, and deployment types. The results show that nondeterminism persists even at temperature 0.0, with distinct variability patterns by model size, deployment, and prompt type. We situate these findings within existing work on concept drift, behavioural drift, and infrastructure-induced nondeterminism, discuss the limitations of lexical metrics, and highlight emerging semantic approaches. By establishing a systematic empirical baseline in the absence of stabilisation techniques, this study provides a reference point for evaluating future drift mitigation and control methods.

大模型非确定性漂移分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。