通过分清事实、假设和情绪,让大模型决策更稳定一致。
Measuring and Improving Behavioral Consistency in Large Language Models through Fact-Heuristic-Emotion State Enforcement

- 用三元状态结构强制模型在决策前区分事实、假设与情绪信号。
- 使决策翻转率降低82%,重复输出变异减少近1.1倍。
- 不改模型权重,仅靠提示层设计就显著提升行为一致性。
大型语言模型在多次运行中对同一决策问题会给出不同答案,甚至因自身先前回答作为上下文而反转判断。本文提出认知内核模型(CKM),一种无需修改模型权重的提示级状态强化机制。该方法要求模型在决策前将输入分为三类:事实(可验证信息)、启发式(推断或假设)和情绪(评价或优先级信号)。通过维护结构化状态 S_t = {F_t, H_t, E_t} 并由转移函数更新,实现状态追踪。在韩语决策场景(模糊性、伦理冲突、资源分配、错误处理)下,对4家厂商共26个模型进行37,403次观测评估,涵盖四组核心实验、四臂消融、五臂虚假限制消融及温度探针测试。结果表明:(1) CKM显著降低重复输出变异性(随机效应Hedges' g=1.09,95%置信区间[0.83, 1.35],31组模型对);(2) 状态持久性使新模型决策翻转率下降82%(g=1.52);(3) 效果非仅由JSON格式导致(仅值重计算即g=2.24);(4) 固定状态下的内在随机性可忽略;(5) 采样不确定性增加时优势更明显(温度0.7时g=2.87);(6) 虚假消融显示约45%增益源于结构支撑,55%来自事实/启发式/情绪内容,且仅CKM同时提升一致性并减少翻转。模型推理正确性未改善。结论:行为一致性可度量,存在模型差异,可通过强制分离三类信息部分提升。
原文摘要 · Abstract (English)
Large language models (LLMs) can give different answers to the same decision problem across runs, and reverse a decision when their own prior answer returns as context. We ask whether this instability can be measured and partially reduced without changing model weights. We test the Cognitive Kernel Model (CKM), a prompt-level state-enforcement layer. Before deciding, the model must separate its input into three epistemic roles: Fact (given or verifiable), Heuristic (inferred or assumed), and Emotion (evaluative or priority signal). CKM adds no capability; it forces the model to track what kind of information it uses before acting. Formally it maintains a structured state S_t = {F_t, H_t, E_t} updated by a transition function. We evaluate CKM on Korean-language decision scenarios (ambiguity, ethical conflict, resource allocation, error handling) across 26 LLMs from four vendors and 37,403 observations, via four core experiments, a 4-arm ablation, a 5-arm sham-restriction ablation, and a temperature probe. Findings: (1) CKM reduces repeated-output variability (random-effects Hedges' g=1.09, 95% CI [0.83, 1.35], 31 model pairs); (2) state persistence cuts the decision-flip rate by 82% in newer models (g=1.52); (3) the effect is not JSON formatting alone (value-only recomputation, g=2.24); (4) intrinsic randomness under fixed anchor states is negligible; (5) the advantage grows under sampling stochasticity (g=2.87 at temperature 0.7); (6) a sham ablation attributes about 45% of the gain to structural scaffolding and 55% to Fact/Heuristic/Emotion content, and CKM is the only arm that both raises consistency and reduces flipping. CKM does not improve reasoning correctness. The narrower result: behavioral consistency is measurable, varies across models, and is partially improvable by forcing models to separate facts, assumptions, and evaluative signals before deciding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。