用方向感知门控提升罕见毒言分类，小模型效果超BERT。
CoGate-LSTM: Prototype-Guided Feature-Space Gating for Mitigating Gradient Dilution in Imbalanced Toxic Comment Classification
- 通过余弦相似度门控机制，动态调整词向量方向以增强少数类特征。
- 在Jigsaw数据集上达0.881宏F1，少数类提升最高达71%。
- 参数仅730万，推理快48毫秒，适合部署在资源受限场景。
极端类别不平衡下，威胁、严重毒言等高风险标签常被传统模型漏检。本文提出CoGate-LSTM，一种基于余弦相似度特征门控的轻量级循环架构，通过学习毒性原型，自适应地根据词向量方向相似性重标定嵌入表示。模型融合冻结的多源嵌入（GloVe、FastText、BERT-CLS）、字符级BiLSTM、嵌入空间SMOTE及加权焦点损失。在Jigsaw毒言分类基准上，该模型实现0.881宏F1（95%置信区间：[0.873, 0.889]）与96.0%准确率，较微调BERT提升6.9点宏F1（p < 0.001），较XGBoost提升4.7点，仅需730万参数（约BERT的1/15）和48毫秒CPU推理延迟。对少数类提升显著：severe_toxic F1提升71%，threat提升33%，identity_hate提升28%。消融实验表明，余弦门控是性能主因（移除后宏F1下降4.8点），字符融合与多头注意力也贡献明显。模型具备一定跨数据集迁移能力，零样本在Contextual Abuse Dataset上达0.71宏F1，经轻量阈值适配可达0.73。结果表明，方向感知门控为不平衡毒言分类提供高效替代方案。
原文摘要 · Abstract (English)
Toxic text classification for online moderation remains challenging under extreme class imbalance, where rare but high-risk labels such as threat and severe_toxic are consistently underdetected by conventional models. We propose CoGate-LSTM, a parameter-efficient recurrent architecture built around a novel cosine-similarity feature gating mechanism that adaptively rescales token embeddings by their directional similarity to a learned toxicity prototype. Unlike token-position attention, the gate emphasizes feature directions most informative for minority toxic classes. The model combines frozen multi-source embeddings (GloVe, FastText, and BERT-CLS), a character-level BiLSTM, embedding-space SMOTE, and weighted focal loss. On the Jigsaw Toxic Comment benchmark, CoGate-LSTM achieves 0.881 macro-F1 (95% CI: [0.873, 0.889]) and 96.0% accuracy, outperforming fine-tuned BERT by 6.9 macro-F1 points (p < 0.001) and XGBoost by 4.7, while using only 7.3M parameters (about 15$\times$ fewer than BERT) and 48 ms CPU inference latency. Gains are strongest on minority labels, with F1 improvements of +71% for severe_toxic, +33% for threat, and +28% for identity_hate relative to fine-tuned BERT. Ablations identify cosine gating as the primary driver of performance (-4.8 macro-F1 when removed), with additional benefits from character-level fusion (-2.4) and multi-head attention (-2.9). CoGate-LSTM also transfers reasonably across datasets, reaching a 0.71 macro-F1 zero-shot on the Contextual Abuse Dataset and 0.73 with lightweight threshold adaptation. These results show that direction-aware feature gating offers an effective and efficient alternative to large, fully fine-tuned transformers for classifying imbalanced toxic comments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。