用注意力机制改进文本分类中的HAL表示，提升准确率与可解释性。
Enhancing Hyperspace Analogue to Language (HAL) Representations via Attention-Based Pooling for Text Classification
- 引入可学习的加性注意力池化，动态加权重要词
- 在IMDB数据集上达到82.38%准确率，比均值池化高6.74个百分点
- 能抑制停用词，聚焦情感关键词，适合需要解释性的任务
HAL模型依赖全局词共现矩阵构建分布语义表示，但传统均值池化对所有词等权处理，导致上下文关键词信息被稀释。本文在HAL表示流程中引入可学习、温度缩放的加性注意力机制，并在注意力层前使用截断奇异值分解（Truncated SVD）降低原始共现矩阵的稀疏性与高维性。在IMDB情感分析数据集上的实验表明，该方法测试准确率达82.38%，相比均值池化基线（75.64%）提升6.74个百分点。定性分析显示，注意力权重有效抑制了停用词，聚焦于情感承载词，兼顾性能提升与模型可解释性。
原文摘要 · Abstract (English)
The Hyperspace Analogue to Language (HAL) model relies on global word co-occurrence matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, aggregating them into sentence-level embeddings via standard mean pooling often results in information loss. Mean pooling assigns equal weight to all tokens, thereby diluting the impact of contextually salient words with uninformative structural tokens. In this paper, we address this limitation by integrating a learnable, temperature-scaled additive attention mechanism into the HAL representation pipeline. To mitigate the sparsity and high dimensionality of the raw co-occurrence matrices, we apply Truncated Singular Value Decomposition (SVD) to project the vectors into a dense latent space prior to the attention layer. We evaluate the proposed architecture on the IMDB sentiment analysis dataset. Empirical results demonstrate that the attention-based pooling approach achieves a test accuracy of 82.38%, yielding an absolute improvement of 6.74 percentage points over the traditional mean pooling baseline (75.64%). Furthermore, qualitative analysis of the attention weights indicates that the mechanism successfully suppresses stop-words and selectively attends to sentiment-bearing tokens, improving both classification performance and model interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。