arXiv:2608.20384cs.AI2026-08

用树模型融合多模态数据,既准又可解释。

Interpretable Multimodal Classification with Linear Discriminant Tree Ensembles

论文配图:Interpretable Multimodal Classification with Linear Discriminant Tree Ensembles
图 1 · 摘自论文原文
  • 用线性判别树集成处理多模态特征,提升可解释性。
  • 在IEMOCAP和CMU-MOSI上比Transformer高4.3% F1-mod。
  • 新重要性度量更准识别关键情绪线索,适合医疗教育场景。

融合文本、音频和视觉流的多模态情感与行为分类器需同时具备高准确率和人类可理解的决策解释——当前高性能模型如Transformer仅部分满足此需求。尽管其预测能力强,但分布式表示和深层非线性导致难以赋予各模态特征有意义的重要性权重,限制了其在临床情感监测、教育评估等可信敏感场景的应用。为此,本文提出基于树集成的框架:将各模态编码为令牌,提取并聚类概念以降维,通过树集成分类器融合多模态信息,并使用一种改进的特征重要性度量解析趋势。该度量降低了负类对二分类任务的影响,提升了标志物检测能力。提出的线性判别树(LDT)、线性判别森林(LDF)和线性判别AdaBoost(LDAB)在多模态分类上相较Multimodal Transformer提升4.3% F1-mod,较基线模型IMR提升3.0%准确率。新特征重要性在IEMOCAP上的人工标注一致率达62.2%(默认为43.2%),在CMU-MOSI上达46.7%(默认为32.1%),显著优于传统方法。

原文摘要 · Abstract (English)

Multimodal affect and behaviour classifiers that fuse heterogeneous text, audio, and visual streams must simultaneously achieve competitive accuracy and produce human-understandable explanations of the cues driving their decisions -- a dual objective that current high-capacity models, notably Transformers, only partially address. While Transformers attain strong predictive performance, their distributed representations and deep nonlinearity make it difficult to assign meaningful importance weights to individual multimodal features, limiting their use in trust-sensitive applications such as clinical affect monitoring and educational assessment. We address this gap by developing a framework based on tree-based ensembles that balances accuracy and interpretability. The framework encodes each modality into tokens, extracts and clusters concepts to reduce dimensionality, routes the fused modalities through tree-based ensemble classifiers, and interprets trends using a novel modified feature importance metric. The modified importance reduces the influence of the negative class in binary classification tasks, thereby improving indicator or marker detection. The proposed tree-based ensembles -- Linear Discriminant Tree (LDT), Linear Discriminant Forest (LDF), and Linear Discriminant AdaBoost (LDAB) -- achieve F1-mod gains of 4.3\% over the Multimodal Transformer and accuracy gains of 3.0\% over the primary interpretable multimodal baseline, Interpretable Multimodal Routing (IMR). The proposed multimodal feature importance extracts salient inter-modal concepts with substantially higher human-annotator agreement scores than default feature importance (62.2\% vs.\ 43.2\% on IEMOCAP; 46.7\% vs.\ 32.1\% on CMU-MOSI).

多模态可解释性树模型情感分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。