arXiv:2508.07286cs.CLcs.IR2025-08

用大模型生成解释性数据,让小模型更准地识别建筑文本中的专业信息。

Arce: Augmented Roberta with Contextualized Elucidations for Ner in Automated Rule Checking

  • 用大模型生成任务相关的解释数据,给小模型做增量预训练。
  • 在建筑领域数据集上达到77.20%的宏平均F1,超越小模型和微调大模型。
  • 简单直接的解释比复杂角色推理更有效,避免语义噪声。

从专业文本中准确提取信息是建筑、工程与施工(AEC)领域自动化规则检查(ARC)的关键挑战。尽管大语言模型(LLMs)具备强大推理能力,但在资源受限的AEC环境中部署常不现实;而标准高效模型又面临显著领域差距。虽然可通过在大规模人工标注语料上预训练缓解此问题,但此类方法成本高昂且耗时。为此,我们提出ARCE(增强型RoBERTa带上下文阐明),一种新颖的知识蒸馏框架,利用LLMs合成面向任务的语料Cote,用于小模型的增量预训练。ARCE系统性探索了知识迁移的最优策略。大量实验表明,ARCE在基准AEC数据集上达到新最佳性能,宏平均F1达77.20%,优于领域专用基线及微调的LLMs。关键发现为‘少即是多’原则:在命名实体识别任务中,简单直接的解释显著优于复杂的角色化推理,后者易引入语义噪声。代码将在接受后公开。

原文摘要 · Abstract (English)

Accurate information extraction from specialized texts is a critical challenge for automated rule checking (ARC) in the architecture, engineering, and construction (AEC) domain. While large language models (LLMs) possess strong reasoning capabilities, their deployment in resource-constrained AEC environments is often impractical. Conversely, standard efficient models struggle with the significant domain gap. Although this gap can be mitigated by pre-training on large, humancurated corpora, such approaches are labor-intensive and costly. To address this, we propose ARCE (Augmented RoBERTa with Contextualized Elucidations), a novel knowledge distillation framework that leverages LLMs to synthesize a task-oriented corpus, termed Cote, for incrementally pre-training smaller models. ARCE systematically explores the optimal strategy for knowledge transfer. Our extensive experiments demonstrate that ARCE establishes a new state-of-the-art on a benchmark AEC dataset, achieving a Macro-F1 score of 77.20% and outperforming both domain-specific baselines and fine-tuned LLMs. Crucially, our study reveals a less is more principle: simple, direct explanations prove significantly more effective for domain adaptation than complex, role-based rationales in the NER task, which tend to introduce semantic noise. The source code will be made publicly available upon acceptance.

命名实体识别知识蒸馏建筑AI小模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。