用种子和量化隐变量代替模型权重，大幅压缩存储体积。
Kilobyte Models: Neural Networks as a Seed and a Quantized Latent
- 用种子+隐变量生成网络权重，无需存储完整参数
- 模型仅需数字节存储，精度媲美每权几比特量化
- 适合带宽受限场景，如设备端更新与嵌入式部署
训练好的神经网络的存储与传输成本随参数量增长，成为空中更新、设备端库等带宽受限场景的瓶颈。本文研究一种极端模型压缩方式：部署对象不是权重本身，而是一份生成权重的短配方。基于映射网络，将网络权重表示为紧凑可训练隐变量与固定随机基的非线性函数，发现仅需存储隐变量，因基与初始化中心可通过整数种子复现。模型变为种子加量化隐变量，其大小由隐变量维度和位宽决定，而非参数总数。我们形式化该部署体，并引入可扩展至无法内存容纳投影的大网络的种子块级基。实验表明，映射模型精度接近对同一网络进行极端比特量化（每权仅数比特），且存储开销远低于传统方法。达到最激进位宽需在量化闭环中微调隐变量。结果不依赖特定随机基，结构化基可近乎零成本重生成大网络权重。
原文摘要 · Abstract (English)
The cost of storing and transmitting a trained neural network scales with its parameter count, a bottleneck for over-the-air updates, on-device libraries, and other bandwidth-bound deployments. We study an extreme form of model compression in which the deployable artifact is not the weights but a short recipe for regenerating them. Building on Mapping Networks, which express a network's weights as a nonlinear function of a compact trainable latent and a fixed random basis, we observe that only the latent need be stored, because the basis and initialization center are reproducible from an integer seed. A model becomes a seed together with a quantized latent, whose size is set by the latent dimension and bit width rather than the parameter count. We formalize this artifact and introduce a seeded block-wise basis that scales to networks whose projection cannot be held in memory. In our experiments, a mapped model is as accurate as the same network quantized aggressively to a few bits per weight, while taking far fewer bytes to store. Reaching the most aggressive bit widths depends on fine-tuning the latent with quantization in the loop. The results do not depend on the particular random basis, and a structured basis lets the weights be regenerated almost for free even for large networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。