Machine learning algorithms typically rely on optimization subroutines and are well known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to train machine learning methods to automatically improve the performance of optimization and signal processing algorithms. As a proof of concept, we use our approach to improve two popular data processing subroutines in data science: stochastic gradient descent and greedy methods in compressed sensing. We provide experimental results that demonstrate the answer is “yes”, machine learning algorithms do lead to more effective outcomes for optimization problems, and show the future potential for this research direction. In addition to our experimental work, we prove relevant Probably Approximately Correct (PAC) learning theorems for our problems of interest. More precisely, we show that there exists a learning algorithm that, with high probability, will select the algorithm that optimizes the average performance on an input set of problem instances with a given distribution.

中文翻译：

---

优化和信号处理中的数据驱动算法选择和调整

机器学习算法通常依赖于优化子程序，并且众所周知可以为许多类型的问题提供非常有效的结果。在这里，我们翻转依赖并提出相反的问题：机器学习算法能否为优化问题带来更有效的结果？我们的目标是训练机器学习方法以自动提高优化和信号处理算法的性能。作为概念证明，我们使用我们的方法来改进数据科学中两个流行的数据处理子程序：随机梯度下降和压缩感知中的贪婪方法。我们提供的实验结果证明答案是肯定的，机器学习算法确实为优化问题带来了更有效的结果，并展示了该研究方向的未来潜力。除了我们的实验工作之外，我们还为我们感兴趣的问题证明了相关的可能近似正确 (PAC) 学习定理。更准确地说，我们表明存在一种学习算法，该算法以高概率选择优化具有给定分布的一组输入问题实例的平均性能的算法。