Function-as-a-Service (FaaS) is one form of the serverless cloud computing paradigm and is defined through FaaS platforms (e.g., AWS Lambda) executing event-triggered code snippets (i.e., functions). Many studies that empirically evaluate the performance of such FaaS platforms have started to appear but we are currently lacking a comprehensive understanding of the overall domain. In our work, we survey existing research on FaaS performance evaluation and present results from a multivocal literature review (MLR) covering 112 studies from academic (51) and grey literature (61). We find that existing work heavily focuses on AWS Lambda and prevalently evaluates micro-benchmarks using simple functions to measure CPU speed and FaaS platform overhead (i.e., container cold starts). Further, we identify a mismatch between academic and industrial sources on tested platform configurations, conclude that function triggers remain insufficiently studied, and list HTTP API gateways and cloud storages as the most used external service integrations. Following existing guidelines on experimentation in cloud systems, we discover many flaws threatening the reproducibility of experiments presented in the surveyed studies. We conclude with a discussion of gaps in literature and highlight methodological suggestions that may serve to improve future FaaS performance evaluation studies.

中文翻译：

---

功能即服务性能评估：多声道文献综述

功能即服务 (FaaS) 是无服务器云计算范式的一种形式，通过 FaaS 平台（例如 AWS Lambda）执行事件触发的代码片段（即函数）来定义。许多实证评估此类 FaaS 平台性能的研究已经开始出现，但我们目前缺乏对整个领域的全面了解。在我们的工作中，我们调查了关于 FaaS 绩效评估的现有研究，并展示了多语文献综述 (MLR) 的结果，涵盖了来自学术 (51) 和灰色文献 (61) 的 112 项研究。我们发现现有的工作主要集中在 AWS Lambda 上，并且普遍使用简单的函数来评估微基准测试，以测量 CPU 速度和 FaaS 平台开销（即容器冷启动）。更多，我们在测试的平台配置上发现学术和工业来源之间的不匹配，得出结论认为函数触发器仍未得到充分研究，并将 HTTP API 网关和云存储列为最常用的外部服务集成。遵循现有的云系统实验指南，我们发现了许多威胁调查研究中提出的实验可重复性的缺陷。我们最后讨论了文献中的差距，并强调了可能有助于改进未来 FaaS 性能评估研究的方法论建议。我们发现了许多威胁调查研究中提出的实验可重复性的缺陷。我们最后讨论了文献中的差距，并强调了可能有助于改进未来 FaaS 性能评估研究的方法论建议。我们发现了许多威胁调查研究中提出的实验可重复性的缺陷。我们最后讨论了文献中的差距，并强调了可能有助于改进未来 FaaS 性能评估研究的方法论建议。