BERT处理四种论元结构时呈现层级表征，早期即显构造特征。
A Hierarchical and Attentional Analysis of Argument Structure Constructions in BERT Using Naturalistic Corpora
- 结合多种分析方法，解析BERT对论元结构的表征机制。
- 中层特征聚类分离度最高，体现构造特异性。
- 适合研究NLP模型语义表征与注意力机制的读者。
本研究探究双向编码器表示模型（BERT）如何处理四种基本论元结构构造。采用多维度分析框架，整合了多维尺度分析（MDS）、t-SNE降维、广义区分值（GDV）作为聚类分离度量、费雪判别比（FDR）作为线性诊断探测，以及注意力机制分析。结果揭示出一种层级化表征结构：构造特异性信息在早期层中出现，在中层形成最大可分聚类，并在后续处理阶段得以保持。
原文摘要 · Abstract (English)
This study investigates how the Bidirectional Encoder Representations from Transformers model processes four fundamental Argument Structure Constructions. We employ a multi-dimensional analytical framework, which integrates MDS, t-SNE as dimensionality reduction, Generalized Discrimination Value (GDV) as cluster separation metrics, Fisher Discriminant Ratio (FDR) as linear diagnostic probing, and attention mechanism analysis. Our results reveal a hierarchical representational structure. Construction-specific information emerges in early layers, forms maximally separable clusters in middle layers, and is maintained through later processing stages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。