arXiv:2603.24358cs.HCcs.LG2026-03

用可解释的神经符号模型融合眼动与脑血流数据,检测疲劳状态。

A Neuro-Symbolic System for Interpretable Multimodal Physiological Signals Integration in Human Fatigue Detection

  • 通过注意力编码提取四种生理概念,结合可微推理规则
  • 在18人数据上达72.1%准确率,且能展示概念激活与规则触发
  • 适合需要安全可解释性的驾驶、医疗等疲劳监测场景

我们提出一种神经符号架构,从眼动追踪和功能性近红外光谱(fNIRS)数据中学习四种可解释的生理概念:眼动动力学、注视稳定性、前额叶血流动力学及多模态特征。该模型使用基于注意力的编码器提取特征,并通过可微分近似推理规则与学习权重和软阈值进行组合,以解决传统手工规则僵化及个体水平对齐诊断缺失的问题。该系统用于多模态生理信号的疲劳分类任务,该领域要求模型兼具高精度与可解释性,其内部推理过程可被审查以满足安全关键应用需求。在18名参与者(共560个样本)的留一被试者外评估中,方法达到72.1% ± 12.3%的准确率,与调优基线相当,同时揭示了概念激活与规则触发强度。消融实验表明,个体化校准带来+5.2个百分点的提升,缺少fNIRS概念导致-1.2个百分点下降,而使用Lukasiewicz算子比乘积算子略优(+0.9个百分点)。我们还引入概念保真度这一离线个体审计指标,基于保留标签计算,与个体准确率高度相关(r=0.843,p<0.0001)。

原文摘要 · Abstract (English)

We propose a neuro-symbolic architecture that learns four interpretable physiological concepts, oculomotor dynamics, gaze stability, prefrontal hemodynamics, and multimodal, from eye-tracking and neural hemodynamics, functional near-infrared spectroscopy, (fNIRS) windows using attention-based encoders, and combines them with differentiable approximate reasoning rules using learned weights and soft thresholds, to address both rigid hand-crafted rules and the lack of subject-level alignment diagnostics. We apply this system to fatigue classification from multimodal physiological signals, a domain that requires models that are accurate and interpretable, with internal reasoning that can be inspected for safety-critical use. In leave-one-subject-out evaluation on 18 participants (560 samples), the method achieves 72.1% +/- 12.3% accuracy, comparable to tuned baselines while exposing concept activations and rule firing strengths. Ablations indicate gains from participant-specific calibration (+5.2 pp), a modest drop without the fNIRS concept (-1.2 pp), and slightly better performance with Lukasiewicz operators than product (+0.9 pp). We also introduce concept fidelity, an offline per-subject audit metric from held-out labels, which correlates strongly with per-subject accuracy (r=0.843, p < 0.0001).

疲劳检测可解释性多模态融合神经符号

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。