Zeroth-order (ZO) optimization is a subset of gradient-free optimization that emerges in many signal processing and machine learning (ML) applications. It is used for solving optimization problems similarly to gradient-based methods. However, it does not require the gradient, using only function evaluations. Specifically, ZO optimization iteratively performs three major steps: gradient estimation, descent direction computation, and the solution update. In this article, we provide a comprehensive review of ZO optimization, with an emphasis on showing the underlying intuition, optimization principles, and recent advances in convergence analysis. Moreover, we demonstrate promising applications of ZO optimization, such as evaluating robustness and generating explanations from black-box deep learning (DL) models and efficient online sensor management.

中文翻译：

---

信号处理和机器学习中的零阶优化入门：原理、最新进展和应用

零阶 (ZO) 优化是无梯度优化的一个子集，出现在许多信号处理和机器学习 (ML) 应用程序中。它用于解决类似于基于梯度的方法的优化问题。但是，它不需要梯度，仅使用函数评估。具体来说，ZO 优化迭代执行三个主要步骤：梯度估计、下降方向计算和解更新。在本文中，我们全面回顾了 ZO 优化，重点展示了收敛分析的潜在直觉、优化原则和最新进展。此外，我们展示了 ZO 优化的有前景的应用，例如评估鲁棒性并从黑盒深度学习 (DL) 模型和高效的在线传感器管理中生成解释。