大模型预训练后，周围密集分布着各类任务专家，可直接提取提升性能。
Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
- 将预训练权重视为分布，从中随机采样并筛选最优解
- 在大模型中，任务专家密度高，少量扰动即可获得优秀解
- 无需迭代优化，简单并行采样+投票即媲美复杂强化学习方法
预训练得到的参数向量通常被视为微调的起点。本文提出，预训练结果应被看作一个参数向量的分布，其支撑集已包含特定任务的专家解。在小模型中，这些专家解占据分布体积极小，需依赖梯度下降等结构化优化方法才能发现；而在大且充分预训练的模型中，任务专家密度显著提升，多样化的任务优化专家广泛存在于预训练权重的邻域内。受此启发，我们提出一种完全并行的后训练方法：随机生成 N 个参数扰动，选取表现最好的 K 个，通过多数投票集成预测。尽管方法简单，但在大规模模型上，该方法在性能上可与 PPO、GRPO、ES 等标准后训练方法相媲美。
原文摘要 · Abstract (English)
Pretraining produces a learned parameter vector that is typically treated as a starting point for further iterative adaptation. In this work, we instead view the outcome of pretraining as a distribution over parameter vectors, whose support already contains task-specific experts. We show that in small models such expert solutions occupy a negligible fraction of the volume of this distribution, making their discovery reliant on structured optimization methods such as gradient descent. In contrast, in large, well-pretrained models the density of task-experts increases dramatically, so that diverse, task-improving specialists populate a substantial fraction of the neighborhood around the pretrained weights. Motivated by this perspective, we explore a simple, fully parallel post-training method that samples $N$ parameter perturbations at random, selects the top $K$, and ensembles predictions via majority vote. Despite its simplicity, this approach is competitive with standard post-training methods such as PPO, GRPO, and ES for contemporary large-scale models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。