用大模型识别非裔英语语法特征，发现其有偏见但潜力可观。
Analysis of LLM as a grammatical feature tagger for African American English
- 对比规则、Transformer与大模型在非裔英语语法特征识别上的表现
- 大模型在零样本和少样本下均优于基线，但准确率受文本形式影响
- 适合研究语言多样性、公平性或语音识别的学者参考
非裔英语（AAE）在自然语言处理中面临独特挑战。本研究系统比较了现有NLP模型——基于规则、基于Transformer及大语言模型（LLMs）——在识别AAE关键语法特征（习惯性be和多重否定）方面的性能。这些特征因语法复杂性和高频出现而被选中。评估采用句级二分类任务，涵盖零样本与少样本策略。分析表明，尽管大模型表现优于基线，但仍受时效性偏差及文本正式度等无关特征干扰。研究强调需改进模型训练与架构以更好适配AAE的独特语言特征。数据与代码已公开。
原文摘要 · Abstract (English)
African American English (AAE) presents unique challenges in natural language processing (NLP). This research systematically compares the performance of available NLP models--rule-based, transformer-based, and large language models (LLMs)--capable of identifying key grammatical features of AAE, namely Habitual Be and Multiple Negation. These features were selected for their distinct grammatical complexity and frequency of occurrence. The evaluation involved sentence-level binary classification tasks, using both zero-shot and few-shot strategies. The analysis reveals that while LLMs show promise compared to the baseline, they are influenced by biases such as recency and unrelated features in the text such as formality. This study highlights the necessity for improved model training and architectural adjustments to better accommodate AAE's unique linguistic characteristics. Data and code are available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。