用语义嵌入匹配任务与智能体，92.2%准确率下实现快速推荐。
AgentRec: Agent Recommendation Using Sentence Embeddings Aligned to Human Feedback
- 基于SBERT扩展模型，将自然语言任务转为语义向量进行匹配
- 在测试集上达到92.2%的顶级准确率，单次分类耗时<300毫秒
- 支持新类别自适应、可解释性好，适合需要可控推荐的场景
多智能体系统需判断哪个智能体最适合执行给定任务。本文提出一种新型架构，通过扩展Sentence-BERT（SBERT）编码器模型，根据自然语言提示从多个LLM智能体中推荐最合适者。在测试数据上，模型实现92.2%的top-1准确率，每次分类耗时低于300毫秒。相比传统分类方法，该架构计算成本低、可适应新类别、具备可解释性，并可通过人类反馈强化学习实现任意指标的可控优化。通过将自然语言提示编码为语义嵌入，模型捕捉与智能体推荐相关的语义信息；通过微调最小化同一智能体对应嵌入间的距离，并借助人类反馈强化学习对齐人类价值观。最终基于嵌入的余弦相似度，通过最近邻匹配完成任务分类。本研究基于自动生成的合成数据集构建，相关代码与数据已开源至https://github.com/joshprk/agentrec。
原文摘要 · Abstract (English)
Multi-agent systems must decide which agent is the most appropriate for a given task. We propose a novel architecture for recommending which LLM agent out of many should perform a task given a natural language prompt by extending the Sentence-BERT (SBERT) encoder model. On test data, we are able to achieve a top-1 accuracy of 92.2% with each classification taking less than 300 milliseconds. In contrast to traditional classification methods, our architecture is computationally cheap, adaptive to new classes, interpretable, and controllable with arbitrary metrics through reinforcement learning. By encoding natural language prompts into sentence embeddings, our model captures the semantic content relevant to recommending an agent. The distance between sentence embeddings that belong to the same agent is then minimized through fine-tuning and aligned to human values through reinforcement learning from human feedback. This allows the classification of natural language prompts based on their nearest neighbors by measuring the cosine similarity between embeddings. This work is made possible through the generation of a synthetic dataset for agent recommendation, which we have open-sourced to the public along with the code for AgentRec recommendation system at https://github.com/joshprk/agentrec.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。