研究大模型如何恰当生成非裔美国英语，发现用户更希望自主决定使用场景。
My LLM might Mimic AAE -- But When Should it?
- 通过问卷和标注实验，探索黑人用户对LLM生成非裔美式英语的真实感评价。
- 在正式场合偏好主流美式英语，非正式场景中对非裔美式英语接受度更高。
- 提供上下文示例和恰当提示后，生成内容真实感可媲美真实黑人说话记录。
我们研究了大型语言模型（LLMs）对非裔美国英语（AAE）的表征，探讨（a）非裔美国人对这些技术生成真实AAE的有效性感知，以及（b）他们在何种情境下认为这种生成是可取的。通过对104名非裔美国人进行调查，并由228名非裔美国人标注LLM生成的AAE，研究发现，非裔美国人更倾向于在输出中拥有选择权和自主权，以决定何时应使用AAE。他们普遍认为，在正式场合中，LLM应默认使用主流美式英语；而在非正式场景中，对生成AAE的兴趣更大。当模型获得适当提示并提供上下文示例时，参与者认为其输出的AAE真实性与真实黑人对话转录相当。项目代码与数据详见：https://github.com/smelliecat/AAEMime.git。
原文摘要 · Abstract (English)
We examine the representation of African American English (AAE) in large language models (LLMs), exploring (a) the perceptions Black Americans have of how effective these technologies are at producing authentic AAE, and (b) in what contexts Black Americans find this desirable. Through both a survey of Black Americans ($n=$ 104) and annotation of LLM-produced AAE by Black Americans ($n=$ 228), we find that Black Americans favor choice and autonomy in determining when AAE is appropriate in LLM output. They tend to prefer that LLMs default to communicating in Mainstream U.S. English in formal settings, with greater interest in AAE production in less formal settings. When LLMs were appropriately prompted and provided in context examples, our participants found their outputs to have a level of AAE authenticity on par with transcripts of Black American speech. Select code and data for our project can be found here: https://github.com/smelliecat/AAEMime.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。