We characterize proximity operators, that is to say functions that map a vector to a solution of a penalized least-squares optimization problem. Proximity operators of convex penalties have been widely studied and fully characterized by Moreau. They are also widely used in practice with nonconvex penalties such as the \(\ell ^0\) pseudo-norm, yet the extension of Moreau’s characterization to this setting seemed to be a missing element of the literature. We characterize proximity operators of (convex or nonconvex) penalties as functions that are the subdifferential of some convex potential. This is proved as a consequence of a more general characterization of the so-called Bregman proximity operators of possibly nonconvex penalties in terms of certain convex potentials. As a side effect of our analysis, we obtain a test to verify whether a given function is the proximity operator of some penalty, or not. Many well-known shrinkage operators are indeed confirmed to be proximity operators. However, we prove that windowed Group-LASSO and persistent empirical Wiener shrinkage—two forms of a so-called social sparsity shrinkage—are generally not the proximity operator of any penalty; the exception is when they are simply weighted versions of group-sparse shrinkage with non-overlapping groups.

中文翻译：

---

邻近算子的刻画

我们表征邻近算子，即将向量映射到惩罚最小二乘优化问题的解的函数。Moreau已广泛研究了凸惩罚的接近算子并对其进行了充分表征。它们在实践中也广泛用于非凸罚分，例如\（\ ell ^ 0 \）伪规范，但将Moreau的表征扩展到此设置似乎是文献中缺少的元素。我们将（凸或非凸）罚分的邻近算子表征为某些凸势的次微分函数。事实证明，这是所谓的Bregman接近算子在某些凸势上可能具有非凸罚分的更普遍特征的结果。作为我们分析的副作用，我们获得了一个测试，以验证给定的函数是否是某个惩罚的接近算子。实际上，已确认许多著名的收缩算子是接近算子。但是，我们证明了窗口化的LASSO组和持久的经验式维纳收缩（所谓的社会稀疏性收缩的两种形式）通常不邻近运营商将受到任何处罚；唯一的例外是当它们只是具有非重叠组的组稀疏收缩的加权版本时。