让虚拟博物馆的展品实时回应不同游客,实现个性化全息互动。
Holo-Artisan: A Personalized Multi-User Holographic Experience for Virtual Museums on the Edge Intelligence
- 边缘计算处理多人姿态、表情、语音数据,驱动数字艺术实时响应。
- 通过联邦学习本地更新模型,仅上传参数以保护隐私并持续优化体验。
- 支持无镜片全息显示,多人共享场景下实现低延迟个性化交互。
我们提出Holo-Artisan,一种新型系统架构,通过真正的全息显示与个性化边缘智能,实现虚拟博物馆中的沉浸式多用户互动体验。本地边缘计算节点同时处理多位参观者的实时数据——包括姿态、面部表情和语音,生成式AI模型则驱动数字艺术品(如立体版蒙娜丽莎)对每位观众做出独特响应。例如,蒙娜丽莎可对一人微笑,同时与另一人进行语音问答,全程实时。云辅助协作平台使用通用场景描述构建共享场景,并通过光线追踪渲染高保真、个性化的视图,直接输出至无镜片全息显示器。为保障用户隐私并持续提升个性化,系统集成联邦学习:边缘设备本地微调模型,仅上传模型更新进行聚合。该边缘主导方案显著降低延迟与带宽消耗,确保多人同步且个性化的沉浸式体验。通过Holo-Artisan,静态展陈变为动态活体艺术,开启文化遗产交互新范式。
原文摘要 · Abstract (English)
We present Holo-Artisan, a novel system architecture enabling immersive multi-user experiences in virtual museums through true holographic displays and personalized edge intelligence. In our design, local edge computing nodes process real-time user data -- including pose, facial expression, and voice -- for multiple visitors concurrently. Generative AI models then drive digital artworks (e.g., a volumetric Mona Lisa) to respond uniquely to each viewer. For instance, the Mona Lisa can return a smile to one visitor while engaging in a spoken Q\&A with another, all in real time. A cloud-assisted collaboration platform composes these interactions in a shared scene using a universal scene description, and employs ray tracing to render high-fidelity, personalized views with a direct pipeline to glasses-free holographic displays. To preserve user privacy and continuously improve personalization, we integrate federated learning (FL) -- edge devices locally fine-tune AI models and share only model updates for aggregation. This edge-centric approach minimizes latency and bandwidth usage, ensuring a synchronized shared experience with individual customization. Through Holo-Artisan, static museum exhibits are transformed into dynamic, living artworks that engage each visitor in a personal dialogue, heralding a new paradigm of cultural heritage interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。