arXiv:2603.02072cs.HCcs.AI2026-03被引 1

用AI整合声音、生理和视线数据,帮知识工作者找回工作记忆。

Cognitive Prosthetic: An AI-Enabled Multimodal System for Episodic Recall in Knowledge Work

  • 将语音、生理信号和视线数据同步为时间对齐的可检索记忆记录。
  • 支持自然语言查询过去的工作经历,可按内容、时间或注意力状态检索。
  • 模块化设计且本地处理,适合企业部署的隐私保护记忆增强系统。

现代知识型工作环境日益加重人类情景记忆负担,个体需应对碎片化注意力、重叠会议及多模态信息流。现有工具虽通过笔记或分析提供部分支持,但很少将认知、生理与注意力上下文整合进可检索的记忆表征中。本文提出认知假体多模态系统(CPMS)——一个基于AI的原型系统,旨在通过结构化的情景记录与自然语言检索,支持知识工作中的情景回忆。CPMS将语音转录、生理信号与注视行为同步为时间对齐的JSON格式情景记录,并在本地处理以保障隐私。系统包含基于Web的检索界面,用户可通过自然语言查询过往工作体验,参考语义内容、时间、注意力焦点或生理状态。本研究展示了将异构传感器数据转化为可查询情景记忆的技术可行性。系统设计为模块化,支持部分传感器配置运行,并集成隐私保护机制,适用于职场部署。该工作贡献了一个端到端、隐私友好的AI增强记忆架构。

原文摘要 · Abstract (English)

Modern knowledge workplaces increasingly strain human episodic memory as individuals navigate fragmented attention, overlapping meetings, and multimodal information streams. Existing workplace tools provide partial support through note-taking or analytics but rarely integrate cognitive, physiological, and attentional context into retrievable memory representations. This paper presents the Cognitive Prosthetic Multimodal System (CPMS) --an AI-enabled proof-of-concept designed to support episodic recall in knowledge work through structured episodic capture and natural language retrieval. CPMS synchronizes speech transcripts, physiological signals, and gaze behavior into temporally aligned, JSON-based episodic records processed locally for privacy. Beyond data logging, the system includes a web-based retrieval interface that allows users to query past workplace experiences using natural language, referencing semantic content, time, attentional focus, or physiological state. We present CPMS as a functional proof-of-concept demonstrating the technical feasibility of transforming heterogeneous sensor data into queryable episodic memories. The system is designed to be modular, supporting operation with partial sensor configurations, and incorporates privacy safeguards for workplace deployment. This work contributes an end-to-end, privacy-aware architecture for AI-enabled memory augmentation in workplace settings.

记忆增强多模态隐私计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。