优化稀疏随机嵌入设计，显著提升相似性搜索效果
On Design Choices in Similarity-Preserving Sparse Randomized Embeddings
- 借鉴昆虫与哺乳动物神经结构，通过扩展与稀疏化生成嵌入
- 不同参数组合使搜索性能差异可达数倍以上
- 适合需要高效相似性检索的工程应用
Expand & Sparsify 是在昆虫蘑菇体和哺乳动物小脑中发现的神经回路原理：感官数据先被随机映射到高维空间（扩展），再仅保留最强激活的少数神经元（稀疏化）。该原理被用于设计 FlyHash 算法，生成保持相似性的稀疏嵌入，适用于新奇检测、模式识别和相似性搜索。尽管算法简单，其性能受多个设计选择影响，包括输入预处理、稀疏激活函数选择及随机投影矩阵构造。本文系统研究这些设计对 FlyHash 相似性搜索性能的影响，发现合适的组合可带来性能的显著提升。
原文摘要 · Abstract (English)
Expand & Sparsify is a principle that is observed in anatomically similar neural circuits found in the mushroom body (insects) and the cerebellum (mammals). Sensory data are projected randomly to much higher-dimensionality (expand part) where only few the most strongly excited neurons are activated (sparsify part). This principle has been leveraged to design a FlyHash algorithm that forms similarity-preserving sparse embeddings, which have been found useful for such tasks as novelty detection, pattern recognition, and similarity search. Despite its simplicity, FlyHash has a number of design choices to be set such as preprocessing of the input data, choice of sparsifying activation function, and formation of the random projection matrix. In this paper, we explore the effect of these choices on the performance of similarity search with FlyHash embeddings. We find that the right combination of design choices can lead to drastic difference in the search performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。