梳理大模型在软件工程中的信任问题,揭示当前研究与开发者认知的差距。
Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectives
- 通过88篇论文系统综述,厘清大模型在软件工程中的信任概念
- 发现过度信任易引发安全漏洞,信任不足则阻碍创新
- 面向25位专家调研,揭示学术研究与实践认知的断层
大型语言模型(LLMs)在软件工程(SE)任务中的应用正迅速增长。随着这些模型日益融入关键流程,其可靠性与可信性变得至关重要。然而,当前对信任相关概念的理解仍不清晰,包括信任、不信任和可信性等术语缺乏明确定义。为厘清研究现状并识别未来方向,本研究系统回顾了18篇聚焦于LLMs在SE中的论文,并分析了70篇更广泛的信任领域文献。此外,还对25位领域专家进行了调查,以了解从业者对信任的理解,并识别现有文献与开发者实际认知之间的差距。分析结果形成了一张涵盖信任相关概念的路线图,指出了未来可探索的关键方向。
原文摘要 · Abstract (English)
Applications of Large Language Models (LLMs) are rapidly growing in industry and academia for various software engineering (SE) tasks. As these models become more integral to critical processes, ensuring their reliability and trustworthiness becomes essential. Consequently, the concept of trust in these systems is becoming increasingly critical. Well-calibrated trust is important, as excessive trust can lead to security vulnerabilities, and risks, while insufficient trust can hinder innovation. However, the landscape of trust-related concepts in LLMs in SE is relatively unclear, with concepts such as trust, distrust, and trustworthiness lacking clear conceptualizations in the SE community. To bring clarity to the current research status and identify opportunities for future work, we conducted a comprehensive review of $88$ papers: a systematic literature review of $18$ papers focused on LLMs in SE, complemented by an analysis of 70 papers from broader trust literature. Additionally, we conducted a survey study with 25 domain experts to gain insights into practitioners' understanding of trust and identify gaps between existing literature and developers' perceptions. The result of our analysis serves as a roadmap that covers trust-related concepts in LLMs in SE and highlights areas for future exploration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。