用自然语言指挥叉车在户外自动搬运托盘，支持复杂环境灵活作业。
Lang2Lift: A Language-Guided Autonomous Forklift System for Outdoor Industrial Pallet Handling
- 通过语言指令定位目标托盘，结合视觉感知与6D位姿估计实现精准识别
- 在真实户外场景中成功完成多类托盘的自主取放，姿态误差满足插入要求
- 适合工业自动化、无人物流等需人机协同的户外作业场景
在非结构化户外物流与建筑环境中实现托盘搬运自动化仍具挑战，主要受限于场景杂乱、托盘配置多样及环境变化。本文提出Lang2Lift，一种端到端的语言引导式自主叉车系统，可在真实户外环境下实现托盘抓取操作。操作员可通过自然语言指令从多个具有不同负载和空间布局的托盘中选择目标。系统集成基于基础模型的感知模块与运动规划控制，在闭环自治流程中实现语言引导的视觉感知，识别并分割目标托盘，进而进行6D位姿估计与几何优化，生成可操作的插入位姿。该位姿直接接入叉车规划与控制模块，执行完全自主的托盘抓取动作。我们在ADAPT自主户外叉车平台上部署并评估了该系统，覆盖多种真实场景，包括杂乱环境、光照变化及不同载荷配置。基于容差的位姿评估表明其精度足以保证叉齿顺利插入。时序与失败分析揭示了实际部署中的权衡与限制，为语言引导感知在工业自动化中的集成提供了关键洞察。视频演示见 https://eric-nguyen1402.github.io/lang2lift.github.io/
原文摘要 · Abstract (English)
Automating pallet handling in outdoor logistics and construction environments remains challenging due to unstructured scenes, variable pallet configurations, and changing environmental conditions. In this paper, we present Lang2Lift, an end-to-end language-guided autonomous forklift system designed to support practical pallet pick-up operations in real-world outdoor settings. The system enables operators to specify target pallets using natural language instructions, allowing flexible selection among multiple pallets with different loads and spatial arrangements. Lang2Lift integrates foundation-model-based perception modules with motion planning and control in a closed-loop autonomy pipeline. Language-grounded visual perception is used to identify and segment target pallets, followed by 6D pose estimation and geometric refinement to generate manipulation-feasible insertion poses. The resulting pose estimates are directly coupled with the forklift planning and control modules to execute fully autonomous pallet pick-up maneuvers. We deploy and evaluate the proposed system on the ADAPT autonomous outdoor forklift platform across diverse real-world scenarios, including cluttered scenes, variable lighting, and different payload configurations. Tolerance-based pose evaluation further indicates accuracy sufficient for successful fork insertion. Timing and failure analyses highlight key deployment trade-offs and practical limitations, providing insights into integrating language-guided perception within industrial automation systems. Video demonstrations are available at https://eric-nguyen1402.github.io/lang2lift.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。