将考古图像转化为可训练数据集的可复现流程
A Reproducible Workflow for Scraping, Structuring, and Segmenting Legacy Archaeological Artifact Images
- 开发爬虫与处理管道，自动提取网页图像与元数据
- 生成带掩码和边框的标注文件，兼容COCO格式
- 保护原始数据，仅分享衍生成果，适合数字考古研究
本文提出一种可复现的工作流程，将遗产考古图像集转换为结构化且可用于分割的数据集。以考古数据服务（ADS）保管的旧石器时代手斧和两面器图像集为例，该数据集包含数千张标准化照片，但缺乏批量下载和自动化处理机制。为此，开发了两个开源工具：一个网络爬虫脚本，可按协议获取记录页面，提取元数据并下载图像；一个图像处理流水线，使用UUID重命名文件，通过传统计算机视觉生成二值掩码和边界框，并将所有衍生信息存储在包含考古元数据的COCO兼容JSON文件中。原始图像不重新分发，仅共享掩码、轮廓和标注等衍生产品。该方案提供轻量级、可重复使用的方法，将基于网页的考古图像集转化为机器学习友好格式，促进下游分析，推动数字考古研究的可复现性。
原文摘要 · Abstract (English)
This technical note presents a reproducible workflow for converting a legacy archaeological image collection into a structured and segmentation ready dataset. The case study focuses on the Lower Palaeolithic hand axe and biface collection curated by the Archaeology Data Service (ADS), a dataset that provides thousands of standardised photographs but no mechanism for bulk download or automated processing. To address this, two open source tools were developed: a web scraping script that retrieves all record pages, extracts associated metadata, and downloads the available images while respecting ADS Terms of Use and ethical scraping guidelines; and an image processing pipeline that renames files using UUIDs, generates binary masks and bounding boxes through classical computer vision, and stores all derived information in a COCO compatible Json file enriched with archaeological metadata. The original images are not redistributed, and only derived products such as masks, outlines, and annotations are shared. Together, these components provide a lightweight and reusable approach for transforming web based archaeological image collections into machine learning friendly formats, facilitating downstream analysis and contributing to more reproducible research practices in digital archaeology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。