用小波与余弦变换压缩句子向量，提升效率且保持性能
Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding
- 结合小波变换与余弦变换，无参数地压缩句子向量
- 在多个下游任务中表现媲美甚至优于原始嵌入
- 适合需要高效低维句子表示的NLP应用
小波变换在图像与信号处理中表现出色，其在自然语言处理中捕捉语言特性的潜力逐渐显现。本文将离散小波变换（DWT）应用于词向量与句向量，通过内在与外在评估验证其在降低维度的同时保留关键信息的有效性。进一步将DWT与离散余弦变换（DCT）结合，提出一种无参数模型，基于局部变化的词特征，将富含信息的句子压缩为固定长度向量。实验表明，该方法在下游任务中表现与原始嵌入相当，部分任务更优。
原文摘要 · Abstract (English)
Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in Image and Signal processing suggest that wavelets can be effectively applied to Natural Language Processing (NLP) tasks that capture a variety of linguistic properties. In this paper, we leverage the power of applying Discrete Wavelet Transforms (DWT) to word and sentence embeddings. We first evaluate, intrinsically and extrinsically, how wavelets can effectively be used to consolidate important information in a word vector while reducing its dimensionality. We further combine DWT with Discrete Cosine Transform (DCT) to propose a non-parameterized model that compresses a sentence with a dense amount of information in a fixed size vector based on locally varying word features. We show the efficacy of the proposed paradigm on downstream applications models yielding comparable and even superior (in some tasks) results to original embeddings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。