让一个大模型生成多种思维模式,提升多样性与表现
Many Minds from One Model: Bayesian-Inspired Transformers for Population Diversity
- 在归一化层注入随机性,实现单模型多思维采样
- 零样本生成与强化学习任务中,多样性与性能双优
- 适合需要多视角推理的智能系统开发
尽管现代Transformer模型规模庞大且表现优异,但通常被训练为单一思维系统:优化过程产生确定性参数,代表对数据的单一假设。受人类群体智慧的启发——群体智能源于个体行为的多样性——我们提出基于贝叶斯思想的群体式Transformer(B-Trans),可从单个预训练大语言模型中采样出多样且连贯的模型实例(称作‘思维’)。B-Trans通过在归一化层直接引入随机性,构建贝叶斯后验近似,避免全贝叶斯神经网络训练的高昂成本。从该近似中采样可获得具有多样化行为的思维群体,同时保持通用能力。在生成每条响应时,从随机分布中采样一个固定实例,确保时间一致性与推理连贯性。在零样本生成和可验证奖励强化学习(RLVR)任务上的实验表明,B-Trans有效利用了模型的随机多样性,在提升响应多样性的同时,相比确定性基线取得更优的任务性能。
原文摘要 · Abstract (English)
Despite their scale and success, modern transformers are usually trained as single-minded systems: optimization produces a deterministic set of parameters, representing a single functional hypothesis about the data. Motivated by the analogy to human populations, in which population-level intelligence emerges from diverse individual behaviors, we propose Population Bayesian Transformers (B-Trans), which enable sampling diverse yet coherent transformer large language model instances (hereafter referred to as a 'mind') from a single pre-trained LLM. B-Trans introduces a Bayesian-inspired posterior proxy by injecting stochasticity directly into normalization layers, avoiding the prohibitive cost of training full Bayesian neural networks. Sampling from this proxy yields a population of minds with diverse behaviors while maintaining general competence. During the generation of each response, we sample a single realization from the random distribution and hold it fixed, ensuring temporal consistency and reasoning coherence. Experiments on zero-shot generation and Reinforcement Learning with Verifiable Rewards (RLVR) demonstrate that B-Trans effectively leverages the stochastic model diversity, yielding superior response diversity while achieving better task performance compared to deterministic baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。