用生物启发的阈值控制推理，降低能耗42%且精度几乎不变。
Green MLOps: Closed-Loop, Energy-Aware Inference with NVIDIA Triton, FastAPI, and Bio-Inspired Thresholding
- 模仿蛋白质折叠能量图，动态调节请求准入阈值。
- 在A100测试集上处理时间缩短至0.29秒，比开放环快42%。
- 适合追求绿色推理、需可审计能效控制的生产系统。
能源效率是AI部署的首要问题，长期推理的累计碳足迹甚至超过训练阶段。本文提出一种生物启发框架，将蛋白质折叠的能量洼地映射为推理成本景观，并通过衰减的闭环阈值控制执行。仅当预期效用与能耗权衡有利时（高置信度、低边际能耗和拥堵）才接纳请求，促使系统偏向首个可接受的局部最优解，而非追求代价高昂的全局最优。我们在RTX 4000 Ada GPU上评估了通过FastAPI与ONNX Runtime、NVIDIA Triton部署的DistilBERT和ResNet-18模型。消融实验显示，该生物控制器相比标准开环执行，处理时间减少42%（0.50秒降至0.29秒），精度下降不足0.5%。此外，我们确立了轻量本地服务（ORT）与管理批处理（Triton）间的能效边界。结果将生物物理能量模型与绿色MLOps结合，为生产环境中的闭环能效感知推理提供实用且可审计的基础。
原文摘要 · Abstract (English)
Energy efficiency is a first-order concern in AI deployment, as long-running inference can exceed training in cumulative carbon impact. We propose a bio-inspired framework that maps protein-folding energy basins to inference cost landscapes and controls execution via a decaying, closed-loop threshold. A request is admitted only when the expected utility-to-energy trade-off is favorable (high confidence/utility at low marginal energy and congestion), biasing operation toward the first acceptable local basin rather than pursuing costly global minima. We evaluate DistilBERT and ResNet-18 served through FastAPI with ONNX Runtime and NVIDIA Triton on an RTX 4000 Ada GPU. Our ablation study reveals that the bio-controller reduces processing time by 42% compared to standard open-loop execution (0.50s vs 0.29s on A100 test set), with a minimal accuracy degradation (<0.5%). Furthermore, we establish the efficiency boundaries between lightweight local serving (ORT) and managed batching (Triton). The results connect biophysical energy models to Green MLOps and offer a practical, auditable basis for closed-loop energy-aware inference in production.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。