用自编码器压缩文本嵌入，提升下游任务效率与性能
Factor Augmented Supervised Learning with Text Embeddings
- 将自编码器融入预训练模型流程，学习任务相关的低维潜在因子
- 在分类、异常检测等任务中显著优于原始嵌入和传统降维方法
- 适合需要高效处理高维文本表示的研究者与工程师
大型语言模型从文本数据生成嵌入，产生捕捉词义与上下文关系的向量表示。然而，这些嵌入的高维性常导致下游任务效率低下并增加计算成本。为此，我们提出一种监督式因子增强框架——文本自编码器增强学习（AEALT），将降维直接整合进预训练语言模型工作流。首先提取文本文档的嵌入；随后通过一个监督增强型自编码器学习低维、任务相关的潜在因子。通过建模复杂嵌入的非线性结构，AEALT 在性能上超越依赖原始嵌入的传统深度学习方法。我们在多个真实世界公开数据集上进行了广泛实验，验证了其在分类、异常检测与预测任务中的广泛应用性。数值结果表明，AEALT 在多项任务中均显著优于原始嵌入及若干标准降维方法。
原文摘要 · Abstract (English)
Large language models (LLMs) generate text embeddings from text data, producing vector representations that capture the semantic meaning and contextual relationships of words. However, the high dimensionality of these embeddings often impedes efficiency and drives up computational cost in downstream tasks. To address this, we propose AutoEncoder-Augmented Learning with Text (AEALT), a supervised, factor-augmented framework that incorporates dimension reduction directly into pre-trained LLM workflows. First, we extract embeddings from text documents; next, we pass them through a supervised augmented autoencoder to learn low-dimensional, task-relevant latent factors. By modeling the nonlinear structure of complex embeddings, AEALT outperforms conventional deep-learning approaches that rely on raw embeddings. We validate its broad applicability with extensive experiments on classification, anomaly detection, and prediction tasks using multiple real-world public datasets. Numerical results demonstrate that AEALT yields substantial gains over both vanilla embeddings and several standard dimension reduction methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。