arXiv:2601.04297cs.LGcs.CV2026-01

用绘画的视觉和动态行为数据,自动分析心理状态。

ArtCognition: A Multimodal AI Framework for Affective State Sensing from Visual and Kinematic Drawing Cues

  • 融合画作图像与绘制过程的动作数据,实现多模态分析。
  • 多模态特征与心理量表相关性显著,优于单一模态。
  • 基于心理知识增强生成,提升结果可解释性,适合临床辅助。

人类情感与心理状态的客观评估面临挑战,尤其通过非语言渠道。本文将数字绘画作为丰富且未充分探索的情感感知模态,提出名为ArtCognition的新型多模态框架,用于自动化分析广泛使用的屋-树-人(HTP)心理测试。该框架独特融合两类数据流:由计算机视觉模型捕获的最终作品静态视觉特征,以及绘制过程中提取的动态行为轨迹,如笔触速度、停顿与流畅度。为弥合低级特征与高级心理解读之间的鸿沟,采用检索增强生成(RAG)架构,将分析锚定在既有心理学知识上,提升可解释性并降低模型幻觉风险。实验表明,视觉与行为动力学特征的融合能提供比单一模态更细致的评估。我们验证了提取的多模态特征与标准化心理量表间的显著相关性,证实该框架具备作为可扩展工具支持临床工作的潜力。本研究贡献了一种非侵入式情感状态评估新方法,并为技术辅助心理健康医疗开辟新路径。

原文摘要 · Abstract (English)

The objective assessment of human affective and psychological states presents a significant challenge, particularly through non-verbal channels. This paper introduces digital drawing as a rich and underexplored modality for affective sensing. We present a novel multimodal framework, named ArtCognition, for the automated analysis of the House-Tree-Person (HTP) test, a widely used psychological instrument. ArtCognition uniquely fuses two distinct data streams: static visual features from the final artwork, captured by computer vision models, and dynamic behavioral kinematic cues derived from the drawing process itself, such as stroke speed, pauses, and smoothness. To bridge the gap between low-level features and high-level psychological interpretation, we employ a Retrieval-Augmented Generation (RAG) architecture. This grounds the analysis in established psychological knowledge, enhancing explainability and reducing the potential for model hallucination. Our results demonstrate that the fusion of visual and behavioral kinematic cues provides a more nuanced assessment than either modality alone. We show significant correlations between the extracted multimodal features and standardized psychological metrics, validating the framework's potential as a scalable tool to support clinicians. This work contributes a new methodology for non-intrusive affective state assessment and opens new avenues for technology-assisted mental healthcare.

情感计算多模态心理评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。