用视觉听觉信号动态生成卷积核，精准调节文本特征提取。
DyKen-Hyena: Dynamic Kernel Generation via Cross-Modal Attention for Multimodal Intent Recognition
- 将多模态信号转为每词对应的动态卷积核，精细调节文本表示。
- 在MIntRec和MIntRec2.0上达到当前最优，跨域检测F1提升10.46%。
- 适合需要高鲁棒性意图识别的多模态应用，如智能客服、人机交互。
尽管多模态意图识别（MIR）通过融合语言、视频和音频等多源信息展现出良好效果，但各模态间存在的无关或冲突信息可能阻碍性能进一步提升。现有模型多采用多头注意力机制对单模态特征序列进行融合，并将结果叠加回原始表征，这容易使噪声或无关的非语言信号污染主要语言特征，且未能捕捉非语言线索在细粒度词级别上应实现的调制作用而非简单增强。为此，我们提出DyKen-Hyena，将问题从特征融合重构为调制处理：将音视频线索转化为针对每个词的动态卷积核，直接调控文本特征提取过程。该细粒度方法在MIntRec和MIntRec2.0基准上取得当前最优表现，尤其在跨域检测中实现+10.46%的F1分数提升，验证了所提方法能构建更鲁棒的意图表征。
原文摘要 · Abstract (English)
Though Multimodal Intent Recognition (MIR) proves effective by utilizing rich information from multiple sources (e.g., language, video, and audio), the potential for intent-irrelevant and conflicting information across modalities may hinder performance from being further improved. Most current models attempt to fuse modalities by applying mechanisms like multi-head attention to unimodal feature sequences and then adding the result back to the original representation. This process risks corrupting the primary linguistic features with noisy or irrelevant non-verbal signals, as it often fails to capture the fine-grained, token-level influence where non-verbal cues should modulate, not just augment, textual meaning. To address this, we introduce DyKen-Hyena, which reframes the problem from feature fusion to processing modulation. Our model translates audio-visual cues into dynamic, per-token convolutional kernels that directly modulate textual feature extraction. This fine-grained approach achieves state-of-the-art results on the MIntRec and MIntRec2.0 benchmarks. Notably, it yields a +10.46% F1-score improvement in out-of-scope detection, validating that our method creates a fundamentally more robust intent representation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。