用果蝇嗅觉机制启发，用复数权重提升句子表示
Comply: Learning Sentences with Complex Weights inspired by Fruit Fly Olfaction
- 受果蝇嗅觉启发，用复数权重编码位置信息
- 单层网络实现优于飞蝇模型，媲美大型先进模型
- 结果可解释，且无需增加参数
生物启发的神经网络为建模数据分布提供了新路径。FlyVec 是近期一个基于果蝇嗅觉回路设计的模型，用于学习词嵌入，其性能甚至可与专门设计用于文本编码的深度学习方法竞争，且计算效率最高。我们提出是否能进一步提升其性能。为此，本文引入 Comply：通过复数权重引入位置信息，使单层神经网络能够学习序列表示。实验表明，Comply 不仅超越 FlyVec，还达到显著更大规模的前沿模型水平，且未增加额外参数。Comply 生成稀疏的上下文句向量，可通过神经元权重直接解释。
原文摘要 · Abstract (English)
Biologically inspired neural networks offer alternative avenues to model data distributions. FlyVec is a recent example that draws inspiration from the fruit fly's olfactory circuit to tackle the task of learning word embeddings. Surprisingly, this model performs competitively even against deep learning approaches specifically designed to encode text, and it does so with the highest degree of computational efficiency. We pose the question of whether this performance can be improved further. For this, we introduce Comply. By incorporating positional information through complex weights, we enable a single-layer neural network to learn sequence representations. Our experiments show that Comply not only supersedes FlyVec but also performs on par with significantly larger state-of-the-art models. We achieve this without additional parameters. Comply yields sparse contextual representations of sentences that can be interpreted explicitly from the neuron weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。