将文本转为彩色图像,用颜色区分主题、情感和强度,让模型更懂语义。
Semimage: HSV-Based Semantic Image Encoding for Disentangled Text Representation
- 用HSV色彩空间表示单词,分通道编码主题、情感与确定性。
- 动态插入语义边界行,使段落切换在图像中更明显。
- 结合CNN实现分类,结果优于BERT等模型且可解释性强。
我们提出SemImage,一种将文本文档表示为二维语义图像的新方法,以便卷积神经网络(CNN)处理。在SemImage中,每个词对应一个像素:行代表句子,句间插入额外的边界行以标记语义转换。每个像素不是传统RGB值,而是解耦的HSV色彩空间向量,其中色相H_cos与H_sin编码主题(考虑循环特性),饱和度编码情感,明度编码强度或确定性。通过多任务学习框架实现解耦:ColorMapper网络将词嵌入映射至HSV空间,并对色相和饱和度通道施加辅助监督,分别预测主题和情感标签,同时完成主任务。在句间动态计算并插入边界行,使得语义不同时形成清晰视觉分界,有效突出段落分割。将SemImage与标准2D CNN(如ResNet)结合用于文档分类。在包含主题与情感标注的多标签数据集及单标签基准上,实验表明SemImage性能可媲美甚至超越强基线(包括BERT与层次注意力网络),同时提升可解释性。消融实验验证了多通道HSV表示与动态边界行的重要性。最后,我们展示的SemImage可视化清晰呈现主题转移与情感变化模式,表明该表示能让语言特征对人和机器都可见。
原文摘要 · Abstract (English)
We propose SemImage, a novel method for representing a text document as a two-dimensional semantic image to be processed by convolutional neural networks (CNNs). In a SemImage, each word is represented as a pixel in a 2D image: rows correspond to sentences and an additional boundary row is inserted between sentences to mark semantic transitions. Each pixel is not a typical RGB value but a vector in a disentangled HSV color space, encoding different linguistic features: the Hue with two components H_cos and H_sin to account for circularity encodes the topic, Saturation encodes the sentiment, and Value encodes intensity or certainty. We enforce this disentanglement via a multi-task learning framework: a ColorMapper network maps each word embedding to the HSV space, and auxiliary supervision is applied to the Hue and Saturation channels to predict topic and sentiment labels, alongside the main task objective. The insertion of dynamically computed boundary rows between sentences yields sharp visual boundaries in the image when consecutive sentences are semantically dissimilar, effectively making paragraph breaks salient. We integrate SemImage with standard 2D CNNs (e.g., ResNet) for document classification. Experiments on multi-label datasets (with both topic and sentiment annotations) and single-label benchmarks demonstrate that SemImage can achieve competitive or better accuracy than strong text classification baselines (including BERT and hierarchical attention networks) while offering enhanced interpretability. An ablation study confirms the importance of the multi-channel HSV representation and the dynamic boundary rows. Finally, we present visualizations of SemImage that qualitatively reveal clear patterns corresponding to topic shifts and sentiment changes in the generated image, suggesting that our representation makes these linguistic features visible to both humans and machines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。