一个无需训练的神经网络用数字拼接实现高效编码与还原
A Synthetic Pseudo-Autoencoder Invites Examination of Tacit Assumptions in Neural Network Design
- 用数字拼接而非压缩实现多值编码，依赖硬件截断机制
- 不依赖学习，直接设计结构，打破传统自动编码器范式
- 适合反思机器学习中隐含假设，尤其对生物演化研究有启发
我们提出一个手工设计的神经网络，无需训练即可将任意整数集合编码为单一数值，并完整恢复原始元素。该网络仅使用标准操作——带偏置的加权求和与恒等激活函数，但设计选择挑战了领域内关于表示、域连续性、计算方式和可学习性的常见观念。例如，其结构是人为设计而非学习所得；通过简单拼接数字实现多值表示，不进行压缩；利用硬件层面的右端舍入作为位操作机制。该网络并非用于实际应用，而是因其与标准训练型自编码器的相似与差异，促使我们反思可能过度约束机器学习与自动编码系统发展的隐含假设。研究部分受自然自编码在物种特征演化中的理论启发，最终从生物学视角深化了讨论。
原文摘要 · Abstract (English)
We present a handcrafted neural network that, without training, solves the seemingly difficult problem of encoding an arbitrary set of integers into a single numerical variable, and then recovering the original elements. While using only standard neural network operations -- weighted sums with biases and identity activation -- we make design choices that challenge common notions in this area around representation, continuity of domains, computation, learnability and more. For example, our construction is designed, not learned; it represents multiple values using a single one by simply concatenating digits without compression, and it relies on hardware-level truncation of rightmost digits as a bit-manipulation mechanism. This neural net is not intended for practical application. Instead, we see its resemblance to -- and deviation from -- standard trained autoencoders as an invitation to examine assumptions that may unnecessarily constrain the development of systems and models based on autoencoding and machine learning. Motivated in part by our research on a theory of biological evolution centered around natural autoencoding of species characteristics, we conclude by refining the discussion with a biological perspective.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。