用低精度数值系统从单模型生成多成员集成，无需训练即可提升泛化性能。
Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systems
- 从单个模型在低精度数制中衍生多个集成成员，无需额外训练
- 实验证明该方法优于现有集成策略，有效提升模型泛化能力
- 适合资源受限场景下快速部署高鲁棒性模型
尽管集成深度神经网络在提升泛化性能方面表现出色，但将其扩展至大模型仍面临挑战。随着深度学习的进步主要依赖于模型规模（如大规模神经网络架构的广泛采用），可扩展性成为大模型时代机器学习算法的关键问题。本文首次展示低精度集成的潜力：在无训练条件下，通过低精度数制从单一模型生成多个集成成员。实验分析表明，所提出的低精度集成方法在性能上优于现有集成方法。
原文摘要 · Abstract (English)
While ensembling deep neural networks has shown promise in improving generalization performance, scaling current ensemble methods for large models remains challenging. Given that recent progress in deep learning is largely driven by the scale, exemplified by the widespread adoption of large-scale neural network architectures, scalability emerges an increasingly critical issue for machine learning algorithms in the era of large-scale models. In this work, we first showcase the potential of low precision ensembling, where ensemble members are derived from a single model within low precision number systems in a training-free manner. Our empirical analysis demonstrates the effectiveness of our proposed low precision ensembling method compared to existing ensemble approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。