AR系统通过识别零件与手部追踪，动态更新装配指引。
Augmented Assembly: Object Recognition and Hand Tracking for Adaptive Assembly Instructions in Augmented Reality
- 结合物体识别与手部追踪，实时构建工作区数字孪生。
- 用框选提示零件位置，检测装配偏差并动态调整步骤。
- 适合需要灵活装配指导的制造、教育场景。
近年来，增强现实（AR）技术推动了交互式物理装配辅助系统的兴起。本文提出一种AR辅助装配流程，利用物体识别和手部追踪实现：(1) 识别自定义组件，(2) 显示分步指令，(3) 检测装配偏差，(4) 根据用户实际操作动态更新指引。通过物体识别，系统实时检测并定位组件，构建工作区的数字孪生。每一步装配中，系统在AR中叠加边界框，指示相关组件的当前位置与目标位置；手部追踪数据则验证用户是否操作正确部件。系统不强制固定流程，而是突出潜在错误，并将用户偏离视为迭代与创造性探索的机会。以乐高积木和定制3D打印组件为案例，证明系统能将数字指引与物理装配无缝衔接，无需手动查找、分类或标记零件。
原文摘要 · Abstract (English)
Recent advances in augmented reality (AR) have enabled interactive systems that assist users in physical assembly tasks. In this paper, we present an AR-assisted assembly workflow that leverages object recognition and hand tracking to (1) identify custom components, (2) display step-by-step instructions, (3) detect assembly deviations, and (4) dynamically update the instructions based on users' hands-on interactions with physical parts. Using object recognition, the system detects and localizes components in real time to create a digital twin of the workspace. For each assembly step, it overlays bounding boxes in AR to indicate both the current position and the target placement of relevant components, while hand-tracking data verifies whether the user interacts with the correct part. Rather than enforcing a fixed sequence, the system highlights potential assembly errors and interprets user deviations as opportunities for iteration and creative exploration. A case study with LEGO blocks and custom 3D-printed components demonstrates how the system links digital instructions to physical assembly, eliminating the need for manual searching, sorting, or labeling of parts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。