arXiv:2510.24058eess.SPcs.AI2025-10中稿 · CVPR

用丰富传感器知识指导廉价部署传感器,提升智能体多感官学习效果

PULSE: Privileged Knowledge Transfer from Rich to Deployable Sensors for Embodied Multi-Sensory Learning

  • 通过教师-学生框架,将高维传感器知识迁移至低成本部署传感器
  • 在无皮肤电反应数据时仍达0.994 AUROC,接近全传感器模型表现
  • 适用于穿戴式压力监测等场景,可推广至触觉、惯性等多种模态

面向具身智能的多感官系统(如可穿戴传感网络、机器人平台)常面临传感器不对称问题:实验室采集时可用最丰富的模态,在部署时却因成本、脆弱或干扰物理交互而无法使用。本文提出PULSE,一种通用框架,实现从信息丰富的教师传感器向多个低成本部署型学生传感器的特权知识迁移。每个学生编码器生成共享(模态无关)与私有(模态特定)嵌入;共享子空间跨模态对齐,并通过多层隐藏状态与池化嵌入蒸馏匹配冻结的教师表示;私有嵌入保留模态特异性结构,用于自监督重建,防止表征坍塌。我们在可穿戴压力监测任务中实例化该框架,以皮肤电反应(EDA)为教师,心电图(ECG)、光体积变化描记法(BVP)、加速度计和温度为学生。在WESAD基准上,留一被试者外评估下,无EDA推理时仍取得0.994 AUROC和0.988 AUPRC(STRESS上为0.965/0.955),超越所有不依赖EDA的基线,媲美保留EDA的全传感器模型。进一步展示了以ECG为教师的模态无关迁移,进行了隐藏状态匹配深度、共享-私有容量、铰链损失边界、融合策略及模态丢弃的广泛消融实验,并讨论了该框架在涉及触觉、惯性、生物电等更广泛具身感知场景中的泛化能力。

原文摘要 · Abstract (English)

Multi-sensory systems for embodied intelligence, from wearable body-sensor networks to instrumented robotic platforms, routinely face a sensor-asymmetry problem: the richest modality available during laboratory data collection is absent or impractical at deployment time due to cost, fragility, or interference with physical interaction. We introduce PULSE, a general framework for privileged knowledge transfer from an information-rich teacher sensor to a set of cheaper, deployment-ready student sensors. Each student encoder produces shared (modality-invariant) and private (modality-specific) embeddings; the shared subspace is aligned across modalities and then matched to representations of a frozen teacher via multi-layer hidden-state and pooled-embedding distillation. Private embeddings preserve modality-specific structure needed for self-supervised reconstruction, which we show is critical to prevent representational collapse. We instantiate PULSE on the wearable stress-monitoring task, using electrodermal activity (EDA) as the privileged teacher and ECG, BVP, accelerometry, and temperature as students. On the WESAD benchmark under leave-one-subject-out evaluation, PULSE achieves 0.994 AUROC and 0.988 AUPRC (0.965/0.955 on STRESS) without EDA at inference, exceeding all no-EDA baselines and matching the performance of a full-sensor model that retains EDA at test time. We further demonstrate modality-agnostic transfer with ECG as teacher, provide extensive ablations on hidden-state matching depth, shared-private capacity, hinge-loss margin, fusion strategy, and modality dropout, and discuss how the framework generalizes to broader embodied sensing scenarios involving tactile, inertial, and bioelectrical modalities.

多模态学习知识迁移可穿戴传感具身智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。