Brief paperDistributed consensus-based solver for semi-definite programming: An optimization viewpoint☆
Introduction
Distributed optimization has attracted intensive attention in the past decade, and its ideas have been applied to solving linear algebraic equations (Liu et al., 2019, Mou et al., 2015, Yuan et al., 2021) and matrix equations (Zeng et al., 2019) over multi-agent networks.
One of the most important problems related to matrix computation is semi-definite programming (SDP), which arises in a variety of applications. For instance, matrix rank minimization (Fazel, 2002), sensor network localization (Simonetto & Leus, 2014), and optimal power flow (Dall’Anese et al., 2013) can be cast into SDP. Due to its great importance, centralized algorithms such as the ellipsoid algorithm (Shor, 2012) and the interior point method (Vandenberghe & Boyd, 1996) have been developed. More recently, exploring distributed algorithms for SDP has received much research interest. Most of existing results were about SDP with sparsity, whose approaches generally consisted of two steps: transform the problem into a coupled one by exploiting its structure (Fukuda et al., 2001), and design distributed algorithms including first-order splitting methods (Dall’Anese et al., 2013, Simonetto and Leus, 2014, Sun et al., 2014, Zheng et al., 2020) and the primal–dual interior point method (Pakazad et al., 2014, Pakazad et al., 2017). In Sun et al. (2014), a first-order splitting algorithm was proposed for SDP with chordal sparsity. In Dall’Anese et al. (2013), a distributed alternating direction method of multipliers (ADMM) was explored and applied to state estimation for smart grids. In Simonetto and Leus (2014), the distributed ADMM was designed with great scalability after edge-based decomposition. In Zheng et al. (2020), an ADMM-based algorithm was designed for both primal and dual SDP problems with identifying infeasible problems. To reduce the computational complexity and improve the convergence rate, distributed primal–dual interior point methods were developed. In Pakazad et al. (2014), a proximal splitting method was designed for the primal–dual directions, but it required many iterations for accurate enough results. In Pakazad et al. (2017), a message passing algorithm was adopted to compute search directions and stepsizes with a fast convergence rate, but an inherent tree structure was required and agents must update sequentially following a tree. Therefore, it is worthwhile to explore effective distributed algorithms for the sparse SDP. In addition, the non-sparse SDP problems are also important with broad applications (Vandenberghe & Boyd, 1996).
In this paper, we investigate the distributed computation for both non-sparse and sparse SDP problems with the help of consensus-based methods. Main contributions are summarized as follows: (i) By fully exploiting their structures and introducing consensus constraints, we transform the two problems into distributed optimization problems. (ii) We propose two distributed primal–dual algorithms with projection operators and derivative feedbacks for positive semi-definite constraints and linear objective functions. (iii) We provide convergence and convergence rate analysis for the projected dynamics by Lyapunov approaches.
Notation: Let be the set of -by- real matrices, be -by- matrices with all entries being , and be the -by- identity matrix. Denote , as the set of -by- symmetric matrices, and positive (semi-) definite matrices, respectively. Denote as . Let be the -th entry of , be the transpose of , and be the Frobenius norm of . Define the Frobenius inner product (denoted by ) as . For , , we denote as the matrix by stacking together in columns. Let be a set of positive integers , and be the number of elements in . Define as the - matrix obtained from with rows indexed by . For , contains rows and columns of indexed by .
Section snippets
Preliminary and formulation
In this section, we introduce some preliminary knowledge, and formulate the SDP problems.
A set is convex if for any and . A function is convex if is a convex set, and for any and . Moreover, is strictly convex if the strict inequality holds whenever and .
Let be a subset of . For , the projection operator is defined by . The distance of to is
Non-sparse SDP
In the non-sparse case, we transform (2) into a distributed optimization problem, and design a distributed algorithm. The multi-agent network, described by graph , consists of agents. The following assumption is well-known.
Assumption 2 Graph is undirected and connected.
Under Assumption 2, by replacing the variable in (2) with for , we formulate a new problem as where is the -th entry in the adjacency matrix of . Under Assumption 2, means (Zeng et al., 2019
Sparse SDP
In the sparse SDP case, as discussed in Pakazad et al. (2017), we consider where , , and were defined in Section 1. Define . Then the objective function and linear equality constraints of (2) are completely determined by (partial entries of ). Remaining entries only affect whether is positive semi-definite. In the following, we focus on the positive semi-definite matrix decomposition.
In fact, a partial
Numerical examples
Here, two examples are given for illustration.
Example 2 Consider a multi-agent network of agents for (3) to verify (6). , and are randomly generated. Furthermore, under Assumption 2, the adjacency matrix of the multi-agent network is also randomly given. Fig. 2(a) shows trajectories of
, and it verifies the convergence of (6). Moreover, the result implies that the solutions are consensus. Fig. 2(b) shows the trajectory of , where is defined in (8) and is the
Weijian Li received the B.S. degree in mechanical engineering from Wuhan University of Technology, Wuhan, China, in 2016. He is currently pursuing the Ph.D. degree in operations research and cybernetics from the University of Science and Technology of China, Hefei, China. He is also a joint Ph. D. student in Key Laboratory of Systems and Control, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China.
His research interests include distributed optimization,
References (28)
- et al.
Stability of primal–dual gradient dynamics and applications to network optimization
Automatica
(2010) - et al.
A fixed-point method for approximate projection onto the positive semidefinite cone
Linear Algebra and Its Applications
(2017) - et al.
Positive definite completions of partial hermitian matrices
Linear Algebra and its Applications
(1984) - et al.
An Arrow–Hurwicz–Uzawa type flow as least squares solver for network linear equations
Automatica
(2019) - et al.
Distributed interior-point method for loosely coupled problems
IFAC Proceedings Volumes
(2014) - et al.
Projected primal–dual gradient flow of augmented Lagrangian with application to distributed maximization of the algebraic connectivity of a network
Automatica
(2018) - et al.
The role of convexity in saddle-point dynamics: Lyapunov function and robustness
IEEE Transactions on Automatic Control
(2017) - et al.
Distributed optimal power flow for smart microgrids
IEEE Transactions on Smart Grid
(2013) Matrix rank minimization with applications
(2002)- et al.
Exploiting sparsity in semidefinite programming via matrix completion I: General framework
SIAM Journal on Optimization
(2001)
Nonlinear Systems
Exploiting sparsity in linear and nonlinear matrix inequalities via positive semidefinite matrix completion
Mathematical Programming
Distributed nonsmooth optimization with coupled inequality constraints via modified Lagrangian function
IEEE Transactions on Automatic Control
A second-order multi-agent network for bound-constrained distributed optimization
IEEE Transactions on Automatic Control
Cited by (0)
Weijian Li received the B.S. degree in mechanical engineering from Wuhan University of Technology, Wuhan, China, in 2016. He is currently pursuing the Ph.D. degree in operations research and cybernetics from the University of Science and Technology of China, Hefei, China. He is also a joint Ph. D. student in Key Laboratory of Systems and Control, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China.
His research interests include distributed optimization, distributed computation of network systems, and Bayesian signal processing.
Xianlin Zeng received the B.S. and M.S. degrees in control science and engineering from the Harbin Institute of Technology, Harbin, China, in 2009 and 2011, respectively, and the Ph.D. degree in mechanical engineering from Texas Tech University, Lubbock, TX, USA, in 2015. He is currently an Associate Professor with the Key Laboratory of Intelligent Control and Decision of Complex Systems, School of Automation, Beijing Institute of Technology, Beijing, China.
His current research interests include distributed optimization, distributed control, and distributed computation of network systems.
Yiguang Hong received his B.S. and M.S. degrees from Peking University, China, and the Ph.D. degree from the Chinese Academy of Sciences (CAS), China. He is currently a professor of Shanghai Institute of Intelligent Science and Technology, Tongji University from Oct 2020, and was a professor of Academy of Mathematics and Systems Science, CAS.
His current research interests include nonlinear control, multi-agent systems, distributed optimization/game, machine learning, and social networks.
Prof. Hong serves as Editor-in-Chief of Control Theory and Technology. He also serves or served as Associate Editors for many journals, including the IEEE Transactions on Automatic Control, IEEE Transactions on Control of Network Systems, and IEEE Control Systems Magazine.
He is a recipient of the Guan Zhaozhi Award at the Chinese Control Conference, Young Author Prize of the IFAC World Congress, Young Scientist Award of CAS, the Youth Award for Science and Technology of China, and the National Natural Science Prize of China. He is also a Fellow of IEEE, a Fellow of Chinese Association for Artificial Intelligence, and a Fellow of Chinese Association of Automation (CAA). Additionally, he is the chair of Technical Committee of Control Theory of CAA and was a board of governor of IEEE Control Systems Society.
Haibo Ji was born in Anhui, China, in 1964. He received the B.Eng. degree and Ph.D. degree in Mechanical Engineering from Zhejiang University and Beijing University, in 1984 and 1990, respectively. He is currently a Professor in Department of Automation, University of Science and Technology of China, Hefei, China.
His research interests include nonlinear control and optimization, and their applications in robots and UAVs.
- ☆
This work was supported by Shanghai Municipal Science and Technology Major Project (No. 2021SHZDZX0100), the National Natural Science Foundation of China (Nos. 61733018, 61903027) and the Open Project Fund of National Defense Key Laboratory of Space Intelligent Control Technology (No. 6142208200312). The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Julien M. Hendrickx under the direction of Editor Christos G. Cassandras.