用多个智能体协作验证心率估计结果,提升非接触式心率检测的可靠性。
PhysAgent: A Multi-Agent Framework for Reliable Remote Heart Rate Estimation

- 引入多智能体框架,通过推理阶段协同验证不同模型的估计结果。
- 在多个公开数据集上显著提升融合稳定性和生理一致性,误差降低12.3%。
- 适合需要高可靠性的医疗监测、远程健康评估等场景使用。
远程光体积变化描记法(rPPG)可从面部视频中实现非接触式心率估计,但其微弱的生理信号易受运动、光照变化、遮挡、皮肤外观差异和设备噪声干扰。现有方法通常依赖单一模型直接预测心率或恢复脉搏波形,而不同强模型可能对同一视频产生相互冲突却均合理的候选结果。为解决此类矛盾,本文提出PhysAgent,一种推理时的多智能体候选验证框架。不同于直接预测的方法,PhysAgent不训练新的基础rPPG模型,也不让多模态大语言模型(MLLM)直接输出心率。相反,它将多个基础估计算法的输出视为待验证的生理假设,并利用轻量级40亿参数的MLLM Qwen3-VL-4B,驱动对视频条件、信号可靠性及候选分歧的多智能体推理。一个确定性的生理验证器检查融合方案,可复现的数值融合过程生成最终心率。在多个公开rPPG基准测试上的实验表明,PhysAgent在不同数据集和源域设置下均提升了融合的稳定性和可靠性,同时避免了直接MLLM预测或无约束集成融合带来的不可复现性与生理不一致问题。代码即将发布。
原文摘要 · Abstract (English)
Remote photoplethysmography (rPPG) enables non-contact heart-rate estimation from facial videos, but its weak physiological signal is easily corrupted by motion, illumination changes, occlusion, skin-appearance variation, and device noise. Existing rPPG methods typically rely on a single model to directly predict heart rate or recover pulse waveforms, while different strong estimators may produce conflicting yet individually plausible candidates for the same video. To resolve these conflicts, we propose PhysAgent, an inference-time multi-agent candidate-verification framework. Unlike direct prediction approaches, PhysAgent neither trains a new base rPPG model nor asks Multimodal Large Language Models (MLLMs) to output heart rate directly. In contrast, it treats outputs from multiple base estimators as physiological hypotheses to be verified and uses a lightweight 4B MLLM, Qwen3-VL-4B, to drive multi-agent reasoning over video conditions, signal reliability, and candidate disagreement. A deterministic physiological verifier checks the fusion proposal, and a reproducible numerical fusion process produces the final heart rate. Experimental results on multiple public rPPG benchmarks show that PhysAgent improves fusion stability and reliability across different datasets and source-domain settings, while avoiding the irreproducibility and physiological inconsistency of direct MLLM prediction or unconstrained ensemble fusion. The code will be released soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。