分析BERT如何编码意大利语名词-介词-名词结构的语法与语义信息
'Layer su Layer': Identifying and Disambiguating the Italian NPN Construction in BERT's family
- 用分层探针分类器逐层分析BERT的上下文向量
- 发现模型深层更准确捕捉构造形式与语义关系
- 为构式语法与神经语言模型的对话提供实证支持
可解释性研究强调需将预训练语言模型（PLMs）及其上下文嵌入与明确的语言理论对比，以判断其编码的语法信息。本研究聚焦意大利语的名词-介词-名词（NPN）构造家族，挑战先前实验设计的理论与方法假设，并将此类研究拓展至较少被关注的语言。从BERT中提取上下文向量，作为输入送入分层探针分类器，系统评估模型各内部层所编码的信息。结果揭示了构造形式与语义在上下文嵌入中的反映程度，为构式理论与神经语言建模之间的对话提供了实证依据。
原文摘要 · Abstract (English)
Interpretability research has highlighted the importance of evaluating Pretrained Language Models (PLMs) and in particular contextual embeddings against explicit linguistic theories to determine what linguistic information they encode. This study focuses on the Italian NPN (noun-preposition-noun) constructional family, challenging some of the theoretical and methodological assumptions underlying previous experimental designs and extending this type of research to a lesser-investigated language. Contextual vector representations are extracted from BERT and used as input to layer-wise probing classifiers, systematically evaluating information encoded across the model's internal layers. The results shed light on the extent to which constructional form and meaning are reflected in contextual embeddings, contributing empirical evidence to the dialogue between constructionist theory and neural language modelling
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。