用AI+AR实现实时3D建模，小白也能轻松操作。
Transcending Dimensions using Generative AI: Real-Time 3D Model Generation in Augmented Reality
- 结合生成式AI与AR，直接在增强现实中实时生成3D模型。
- 35人测试显示系统可用性评分69.64，高频用户达80.71。
- 适合游戏、教育、AR电商等无专业背景用户场景。
传统3D建模需专业技术、专用软件和长时间投入，对多数用户不友好。本研究通过将生成式AI与增强现实（AR）融合，构建一体化系统，使用户能在AR环境中轻松实现3D模型的实时生成、操控与交互。利用Shap-E等前沿AI模型，解决2D图像向AR环境中的3D表示转换的复杂挑战。通过Mask R-CNN等先进目标检测方法，应对对象隔离、复杂背景处理及无缝用户交互等关键问题。对35名参与者评估显示，整体系统可用性量表（SUS）得分为69.64；频繁使用AR/VR的用户评分显著更高，达80.71。该研究在游戏、教育及基于AR的电子商务中具有重要应用价值，为无专业技能用户提供了直观的建模体验。
原文摘要 · Abstract (English)
Traditional 3D modeling requires technical expertise, specialized software, and time-intensive processes, making it inaccessible for many users. Our research aims to lower these barriers by combining generative AI and augmented reality (AR) into a cohesive system that allows users to easily generate, manipulate, and interact with 3D models in real time, directly within AR environments. Utilizing cutting-edge AI models like Shap-E, we address the complex challenges of transforming 2D images into 3D representations in AR environments. Key challenges such as object isolation, handling intricate backgrounds, and achieving seamless user interaction are tackled through advanced object detection methods, such as Mask R-CNN. Evaluation results from 35 participants reveal an overall System Usability Scale (SUS) score of 69.64, with participants who engaged with AR/VR technologies more frequently rating the system significantly higher, at 80.71. This research is particularly relevant for applications in gaming, education, and AR-based e-commerce, offering intuitive, model creation for users without specialized skills.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。