arXiv:2609.08105cs.AIcs.DL2026-09

用AI自动抓取网络文化数据,提升印尼文化遗产库的覆盖与深度。

Artificial Intelligence-Assisted Digital Inventory of Cultural Heritage & Traditional Knowledge: Case for Indonesian Open Digital Library of Culture

  • 构建五阶段经济漏斗框架,实现多语言文化数据自动采集与整合。
  • 通过贝叶斯证据融合与唯一发布机制,确保数据真实且不重复。
  • 保留人类把关角色,兼顾自动化效率与文化敏感性,适合数字人文项目参考。

印尼文化数字图书馆(PDBI)自2007年起通过公众参与收集了数万条南洋文化遗产条目。手动贡献面临三大结构性障碍:覆盖不足(知识分散于多语言和多平台)、完整性差(公开源混杂真实与噪声信息)、完整度低(记录浅层)。本文提出一种基于AI的自主知识采集方法论框架,旨在扩大语料覆盖并深化每条记录的数据深度。该框架采用五阶段经济漏斗设计:定向爬取、多语言提取与标准化、向量编码与阻断、代理决策、幂等式发布,遵循确定性编排与代理决策原则。各阶段均形式化:漏斗经济学与最优过滤排序;爬取前沿动态作为亚临界分支过程,解释需反复重置种子的必要性;事实级新颖性通过包含度量衡量;贝叶斯多源证据融合,对神圣类别设置更高发布阈值;通过幂等插入与事务型消息队列实现精确一次效果;滑动窗口推理预算与预留协议;统计质量审计;种子选择建模为子模覆盖最大化。框架保留四项高价值人工角色:方向制定者、升级审批人、质量审计员、意义守门人,逐步提升机器自主性。伦理、法律与文化敏感性问题亦被讨论,架构上保证机器永不覆盖人类贡献。

原文摘要 · Abstract (English)

The Indonesian Digital Library of Culture (Perpustakaan Digital Budaya Indonesia, PDBI; budaya-indonesia.org) is a participatory platform that has collected tens of thousands of entries on Nusantara cultural heritage through public contribution since 2007. Manual contribution faces three structural barriers: coverage (knowledge is scattered across languages and sites), integrity (open sources mix authentic documentation with noise), and completeness (subjects are recorded but their data remain shallow). This paper presents a methodological framework for autonomous, AI-based harvesting of cultural knowledge from the open web, designed to expand corpus coverage while intensifying per-entry data depth. The methodology is organised as a five-stage economic funnel: focused crawling, multilingual extraction and canonicalisation, vector encoding with blocking, agentic decision-making, and idempotent publication, under the principle of deterministic orchestration, agentic decisions. Each stage is formalised: funnel economics and optimal filter ordering; crawl-frontier dynamics as a subcritical branching process that explains the necessity of recurrent re-seeding; fact-level novelty via a containment measure; Bayesian multi-source evidence fusion with elevated publication thresholds for sacred categories; exactly-once effects via idempotent upserts and the transactional outbox; sliding-window inference budgeting with a reservation protocol; statistical quality auditing; and seed selection as submodular coverage maximisation. The framework retains four high-value human roles: curator of direction, escalation approver, quality auditor, and guardian of meaning, while machine autonomy is raised in stages. Ethical, legal, and cultural-sensitivity implications are discussed, including the architectural guarantee that the machine never overwrites human contributions.

文化遗产AI采集数字人文多语言处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。