让大模型摆脱千篇一律,通过人格锚定和温度调控提升回答多样性。
Beyond the Hivemind: Escaping LLM Homogeneity via Meta-Persona Anchoring and Sequential Temperature Scaling

- 先让模型自选独特人格,再用分层采样策略扩大输出范围。
- 回答相似度从0.85降至0.65,多数问题低于0.7的多样性阈值。
- 适合需要创意、多角度输出的场景,如内容生成与思维实验。
近期研究发现大型语言模型存在‘人工蜂群效应’,导致模型对开放性问题趋于同质化共识,即使采用高温度采样,响应间语义相似度仍高达0.80–0.90。本文提出一种新框架——元人格锚定结合过滤型温度缩放(FTS),以缓解该问题。方法分两阶段:首先引导模型自选独特个性作为生成起点;其次采用双阶段采样筛法,先用Top-$p$过滤确保语法正确性,再对保留候选进行极端温度缩放(T ≥ 4.0)以探索更广概率分布。我们在约200亿参数的开源模型上,基于INFINITY-CHAT数据集进行评估。结果表明,平均成对余弦相似度从0.85显著下降至0.65,多数问题响应相似度低于0.7,有效缩小了人工模式坍缩与人类多样性的差距。代码已开源,助力更具创造力的AI应用部署。
原文摘要 · Abstract (English)
Recent studies have identified an ``Artificial Hivemind'' effect in Large Language Models (LLMs) causing models to converge on a narrow, homogenized consensus even for open questions. This semantic collapse limits the diversity of AI, resulting in high inter-response similarity ($\approx 0.80-0.90$) even under high-temperature sampling. In this paper, we propose a novel mitigation framework to increase diversity: Meta-Persona Anchoring combined with Filtered Temperature Scaling (FTS). Our approach utilizes a two-stage generation process: first, the model is prompted to self-select a unique, idiosyncratic persona to anchor its starting point; second, we apply a dual-stage sampling sieve, utilizing Top-$p$ filtering to preserve grammatical validity followed by extreme temperature scaling ($T \ge 4.0$) on the surviving candidates to explore the broadened probability distribution. We evaluate our method using the INFINITY-CHAT dataset on state-of-the-art open weight models under $\sim$20B parameters. Our results demonstrate a significant reduction in semantic convergence, with average pairwise cosine similarity dropping from ($\approx 0.85$) to ($\approx 0.65$). Our scheme achieves a majority of questions below the 0.7 threshold, effectively reducing the gap between artificial mode collapse and human-level typological diversity. We provide our implementation as an open-source framework to enable more diverse and creative AI deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。