用教师模型指导小模型，让少样本下识别讽刺更准
Teaching Sarcasm: Few-Shot Multimodal Sarcasm Detection via Distillation to a Parameter-Efficient Student
- 用大模型做老师，通过知识蒸馏提升小模型性能
- 在两个数据集上，少样本效果超越大模型和传统方法
- 适合资源有限但需高精度讽刺检测的场景
多模态讽刺检测在低资源环境下尤为困难，因图像与文本间的微妙矛盾难以从稀疏标注数据中学习，导致模型性能受限。参数高效微调（PEFT）方法如适配器、LoRA 和提示调优虽能缓解过拟合，但在少样本条件下仍难达到最优表现。本文提出 PEKD 框架，通过在大规模讽刺数据上训练的专家模型作为教师，对参数高效方法进行知识蒸馏。为降低教师不可靠信号的影响，引入基于熵的门控机制，动态调节蒸馏强度。在两个公开数据集上的实验表明，该框架使 PEFT 方法在少样本场景下超越先前参数高效方法及大型多模态模型，实现优异性能。框架模块化，可适配多种多模态模型与任务。
原文摘要 · Abstract (English)
Multimodal sarcasm detection is challenging, especially in low-resource settings where subtle image-text contradictions are hard to learn due to scarce annotated data, which hinders the model's performance. Parameter-efficient fine-tuning (PEFT) methods like adapters, LoRA, and prompt tuning reduce overfitting but struggle to reach optimal performance due to limited supervision from few-shot data. We propose PEKD, a unified framework that enhances PEFT methods via distillation from an expert model trained on large-scale sarcasm data, which acts as the teacher. To mitigate unreliable signals from the teacher, we introduce an entropy-aware gating mechanism that dynamically adjusts the distillation strength based on teacher confidence. Experiments on two public datasets demonstrate that our PEKD framework enables PEFT methods to outperform both prior parameter-efficient approaches and large multimodal models, achieving strong results in the few-shot scenario. The framework is modular and adaptable to a wide range of multimodal models and tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。