构建100万样本机器人设计数据集，助力AI自动设计与理解
RoboDesign1M: A Large-scale Dataset for Robot Design Understanding
- 基于科学文献的半自动化采集，融合多模态设计数据
- 在图像生成、视觉问答、检索等任务上验证数据集有效性
- 适合研究机器人设计自动化与AI辅助设计的学者使用
机器人设计过程复杂且耗时，需专业知识。深入理解设计数据可推动自动化设计生成、文本检索示例设计及开发AI设计助手。尽管基础模型进展迅速，但该领域受限于缺乏大规模设计数据集。本文提出RoboDesign1M，一个包含100万样本的大型数据集，涵盖多领域机器人设计的多模态数据，源自科学文献。我们设计了半自动化数据收集流程，实现高效多样采集。通过在设计图像生成、设计视觉问答和设计图像检索等任务上的广泛实验，结果表明该数据集是设计理解任务的挑战性新基准，具备推动该领域研究的潜力。RoboDesign1M将公开发布，以支持人工智能驱动的机器人设计自动化发展。项目详情见https://airvlab.github.io/robotdesign1m/
原文摘要 · Abstract (English)
Robot design is a complex and time-consuming process that requires specialized expertise. Gaining a deeper understanding of robot design data can enable various applications, including automated design generation, retrieving example designs from text, and developing AI-powered design assistants. While recent advancements in foundation models present promising approaches to addressing these challenges, progress in this field is hindered by the lack of large-scale design datasets. In this paper, we introduce RoboDesign1M, a large-scale dataset comprising 1 million samples. Our dataset features multimodal data collected from scientific literature, covering various robotics domains. We propose a semi-automated data collection pipeline, enabling efficient and diverse data acquisition. To assess the effectiveness of RoboDesign1M, we conduct extensive experiments across multiple tasks, including design image generation, visual question answering about designs, and design image retrieval. The results demonstrate that our dataset serves as a challenging new benchmark for design understanding tasks and has the potential to advance research in this field. RoboDesign1M will be released to support further developments in AI-driven robotic design automation. Our project is available at https://airvlab.github.io/robotdesign1m/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。