AI改写文本反而让情感分类更准，但讽刺句让模型自信心下降。
When AI Rewrites, Classifiers Relax: Uncertainty-Aware Sentiment Analysis on Sarcastic and AI-Paraphrased Social Text
- 模型对讽刺文本的置信度显著降低，说明能感知自身不确定性。
- 用AI重写后的评论，分类准确率反而提升5.8个百分点。
- 只需放弃低置信度的14%输入，就能让准确率提高6.7个百分点。
情感分类器在处理讽刺或AI生成的社交媒体内容时表现异常，而标准评估难以提供指导。我们通过三部分实证研究发现：首先，讽刺文本的置信度显著低于非讽刺文本（Mann–Whitney p = 2 × 10⁻⁶），表明模型即使无显式不确定性建模也能感知讽刺带来的困惑；其次，反直觉地发现，基于RoBERTa的分类器在AI重写评论上的准确率高于原始人类撰写文本（Qwen3.5-4B重写提升5.8个百分点，Gemma4-E4B提升3.7个百分点），源于AI重写消除了分布噪声，使文本更具典型性；第三，采用轻量级弃权机制——剔除置信度低于0.6的14%样本，可将保留集准确率从82.2%提升至88.9%（+6.7个百分点）。对比语义熵与MC-Dropout分歧作为不确定性信号，二者在讽刺文本上的AUROC分别为0.650和0.646，几乎等效，表明短文本场景下两者可互换。结果推动高风险场景（如心理健康预警、内容审核）向不确定性感知的弃权机制转变。
原文摘要 · Abstract (English)
Sentiment classifiers are increasingly applied to social media content that is either sarcastic or AI-generated --- two distributional regimes where standard evaluations offer little guidance. We present a three-part empirical study of sentiment classifier behaviour under these conditions. First, we find that confidence scores on sarcastic text are significantly lower than on non-sarcastic text (Mann--Whitney $p = 2 \times 10^{-6}$), confirming that classifiers sense their own uncertainty on ironic content even without explicit uncertainty modelling. Second, and counterintuitively, we show that sentiment classifiers achieve higher accuracy on AI-paraphrased reviews than on the original human-authored text (RoBERTa: $+5.8$ pp for Qwen3.5-4B paraphrases, $+3.7$ pp for Gemma4-E4B), revealing a cross-domain stylistic alignment effect: AI paraphrases remove distributional noise that confounds Twitter-trained classifiers, producing cleaner, more prototypical sentiment text. Third, we demonstrate that a lightweight abstention wrapper --- flagging the $14\%$ of inputs with confidence below $0.6$ --- improves accuracy from 82.2\% to 88.9\% ($+6.7$ pp) on the retained set. We further compare Semantic Entropy and MC-Dropout-style disagreement as uncertainty signals and find near-identical AUROC ($0.650$ vs.\ $0.646$) on sarcastic text, suggesting that for short social media inputs, both methods are interchangeable. Our results motivate a shift from confident single-label prediction to uncertainty-aware abstention in high-stakes sentiment applications such as mental health flagging and content moderation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。