人机协作重建脑海中的面孔,准确率超60%。
HAIFAI: Human-AI Interaction for Mental Face Reconstruction
- 用户逐轮打分,系统提取特征生成初始画像
- 手动滑块微调,重建准确率达60.6%
- 无需大量训练数据,适合心理图像重建场景
我们提出HAIFAI——一种新型两阶段人机交互系统,用于重建仅存在于人们脑海中的面部形象。第一阶段,用户对系统生成的图像进行迭代排序,系统据此提取相关特征,融合为统一向量,并用生成模型生成初始重建结果。第二阶段,利用现有人脸编辑方法,用户通过简易滑块界面手动调整面部形状以进一步优化。为避免耗时的人类数据收集,我们引入计算型用户排序行为模型,并基于275名参与者在线众包研究构建了小型人脸排序数据集。在12名参与者的用户研究中,HAIFAI在重建质量、易用性、感知工作量和速度上均优于先前最先进方法。后续18名参与者验证实验显示,其识别准确率达60.6%,创历史新高。该成果推动了可高效可靠还原用户心智图像的交互智能系统发展。
原文摘要 · Abstract (English)
We present HAIFAI - a novel two-stage system where humans and AI interact to tackle the challenging task of reconstructing a visual representation of a face that exists only in a person's mind. In the first stage, users iteratively rank images our reconstruction system presents based on their resemblance to a mental image. These rankings, in turn, allow the system to extract relevant image features, fuse them into a unified feature vector, and use a generative model to produce an initial reconstruction of the mental image. The second stage leverages an existing face editing method, allowing users to manually refine and further improve this reconstruction using an easy-to-use slider interface for face shape manipulation. To avoid the need for tedious human data collection for training the reconstruction system, we introduce a computational user model of human ranking behaviour. For this, we collected a small face ranking dataset through an online crowd-sourcing study containing data from 275 participants. We evaluate HAIFAI and an ablated version in a 12-participant user study and demonstrate that our approach outperforms the previous state of the art regarding reconstruction quality, usability, perceived workload, and reconstruction speed. We further validate the reconstructions in a subsequent face ranking study with 18 participants and show that HAIFAI achieves a new state-of-the-art identification rate of 60.6%. These findings represent a significant advancement towards developing new interactive intelligent systems capable of reliably and effortlessly reconstructing a user's mental image.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。