Distributed LCMV beamformer design by randomly permuted ADMM
Introduction
In traditional beamforming techniques, a dedicated device (the “fusion center”) is assumed to gather all the sensor observations for further processing. This approach is often referred to as the centralized beamforming and it requires large communication bandwidths and significant computing power at the fusion center. However, for many applications, the availability of a fusion center could be a major problem. Therefore, the distributed computing where each node has its own processing unit to update data independently and exchange compressed signal with other nodes, has been widely studied in the last few years [1], [2], [3], [4], [5], [6], [7], [8], [9].
In more recent works [10], [11], [8], [12], [13], a family of distributed beamformer design methods have been investigated based on convex optimization with different structures. For example, in [10], a distributed minimum variance (DMV) algorithm was proposed for robust linearly constrained minimum variance (LCMV) beamforming by casting it as a distributed convex optimization problem. In [11], a sparse distributed beamformer that trades off SNR performance against reduced inter-node power consumption was designed by using the bi-alternating direction method of multipliers (Bi-ADMM). In [8], a distributed MVDR beamforming technique was developed based on the primal-dual method of multipliers (PDMM) [12]. A stochastic ADMM approach for coordinated multicell beamforming (CMBF) was developed for a smart grid powered coordinated multicell downlink system in [13]. It is noted that most of the distributed approaches for speech enhancement are based on blockwise optimization techniques. However, the theoretical convergence results on ADMM-like algorithms (Bi-ADMM, stochastic ADMM) for such multi-block optimization problems have remained unclear for a very long time, except for the two-block optimization model.
The important proof of convergence for extension of ADMM beyond two-block convex minimization problems were unclear until 2016. In [14] it was shown that ADMM may actually fail to converge for blocks greater than or equal to three via a simple three block counterexample. To resolve the issue, a popular method called the randomized block-coordinate descent (RBCD) method was first developed for a class of unconstrained composite minimization problems [15], [16]. Subsequently, a variety of randomized strategies have been investigated to tackle multi-block optimization problems. More recent attempts can be found in [17], [18], [19], [20]. At the same time, many blockwise techniques and splitting methods for solving the constrained composite optimization problem have also been developed for separable optimization problems with applications in signal and imaging processing, machine learning, statistics, and engineering [21], [14], [22], [23], [24], [25], [26], [27], [28]. Indeed, numerous experiments have demonstrated that they are powerful for solving large-scale optimization problems arising in machine learning [15], [16], [29]. Among different randomized techniques, the randomly permuted alternating direction method (RP-ADMM) of multipliers developed in [30], [31] is noticeably important, which is convergent in expectation for cases with blocks greater than or equal to three for nonseparable quadratic programming problems, under the condition that the block diagonal matrix related to the coefficient matrices of objective and constraint is positive definite.
In this paper, we propose a new distributed LCMV beamformer design method based on augmented Lagrangian method together with the blockwise technique. The LCMV beamformer design is formulated into a blockwise optimization problem with each block related to a specific node. Then a simple but efficient alternative iteration algorithm is proposed to solve the problem by employing the RP-ADMM. This method transforms the original optimization problem into a class of sequential subproblems, and then distributes the computational workload to each node of the sensor array. It obtains a great reduction in the computational complexity and makes parallel computing achievable. Another contribution of this paper is to show that the proposed method is applicable to cases with blocks bigger than 3. In establishing the expected convergence of the proposed algorithm, we prove that the necessary condition for convergence is satisfied without further assumption. Finally, the complexity of the proposed distributed LCMV is discussed and compared with the centralized LCMV via both theoretical and numerical analysis.
The outline of this paper is as follows. In Section 2, we describe the LCMV beamformer design problem and introduce an iteration algorithm based on augmented Lagrangian method. In Section 3, we present the blockwise framework for the D-LCMV beamformer design, as well as the algorithm and the convergence analysis. In Section 4, we describe the application of the proposed algorithm for wireless acoustic sensor network and provide experimental results for evaluation. Conclusions are drawn in Section 5.
Section snippets
Problem statement
Consider a wireless acoustic sensor network of N nodes equipped with a microphone array consisting of M microphones in forming a wide-band acoustic beamformer. Our goal is to enhance the desired speech signal from a point source that is contaminated by noise and other interfering sources. The desired and the interference-plus-noise components are assumed to be uncorrelated. In the sequel, we will denote by , , the received signal at the i-th microphone, and by , , the
Distributed LCMV beamformer design
In this section, we develop an implementation of the design for distributed beamformer. We establish the optimization problem for distributed LCMV beamformer design, and a numerical algorithm is proposed for solving the problem, as well as convergence analysis in a concise form.
Suppose each node has its own processing unit and ability to exchange signals with others, and the n-th node is equipped with microphones, and total number is (see Fig. 1).
Recall the LCMV beamformer design
Numerical experiments
In this section, we develop a multi-source scenario to evaluate the performance of the proposed D-LCMV beamformer design method, and compare with the C-LCMV beamformer design method.
Conclusion
In this paper, we have developed a novel multi-block framework for the distributed LCMV beamformer design together with a randomly permuted alternating direction algorithm, which implements a blockwise iterative strategy. It is based on the augmented Lagrangian technique which is popular in solving large scale optimization problems. The expected convergence of this algorithm has also been established, as well as discussion of the computational complexity. We have provided simulation results to
CRediT authorship contribution statement
Zhibao Li: Conceptualization, Methodology, Software, Investigation, Writing - original draft preparation, Validation. Cedric Yiu: Supervision, Conceptualization, Validation, Writing - review & editing, Funding acquisition. Yu-hong Dai: Supervision, Validation, Funding acquisition. Sven Nordholm: Supervision, Validation, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
The first author is supported by the Natural Science Foundation of China (Nos. 11701575, 11801161), the Natural Science Foundation of Hunan Province (No. 2018JJ3624), and the research startup foundations of Central South University. The second author is supported by RGC Grant PolyU (152245/18E), PolyU grant 4-ZZGS and G-UAHF. The third author is supported by the Natural Science Foundation of China (Nos. 11631013, 71331001, 11331012) and the National 973 Program of China (No. 2015CB856002).
Zhibao Li received the B.S. degree from the Anhui Normal University, Wuhu, China, in 2007, the M.S. degree from the Nanjing Normal University, Nanjing, China, in 2010, and the Ph.D degree from The Hong Kong Polytechnic University, Hong Kong, China, in 2014. From 2014 to 2016, he was a Post-Doctoral Researcher with the Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China.
He is
References (47)
- et al.
On sparse beamformer design with reverberation
Appl. Math. Model.
(2018) - et al.
Optimization and evaluation of sigmoid function with a priori SNR estimate for real-time speech enhancement
Speech Commun.
(2013) - et al.
Collaborative beamforming for distributed wireless ad hoc sensor networks
IEEE Trans. Signal Process.
(2005) - et al.
Distributed beamforming and power allocation for cooperative networks
IEEE Trans. Wirel. Commun.
(2008) - et al.
Distributed downlink beamforming with cooperative base stations
IEEE Trans. Inf. Theory
(2008) - et al.
Distributed transmit beamforming: challenges and recent progress
IEEE Commun. Mag.
(2009) - et al.
Distributed adaptive node-specific signal estimation in fully connected sensor networks-part I: sequential node updating
IEEE Trans. Signal Process.
(2010) - et al.
Distributed delay and sum beamformer for speech enhancement via randomized gossip
IEEE Trans. Audio Speech Lang. Process.
(2014) - et al.
Multi-task wireless sensor network for joint distributed node-specific signal enhancement, LCMV beamforming and DOA estimation
IEEE J. Sel. Top. Signal Process.
(2017) - et al.
Ad hoc microphone array beamforming using the primal-dual method of multipliers
Meeting recognition with asynchronous distributed microphone array using block-wise refinement of mask-based MVDR beamformer
A distributed algorithm for robust LCMV beamforming
Distributed sparse MVDR beamforming using the bi-alternating direction method of multipliers
Distributed optimization using the primal-dual method of multipliers
IEEE Trans. Signal Inf. Process. Netw.
A stochastic ADMM approach to distributed coordinated multicell beamforming for renewables powered wireless cellular networks
IEEE Trans. Veh. Technol.
The direct extension of ADMM for multi-block convexminimization problems is not necessarily convergent
Math. Program.
Coordinate descent method for large-scale -loss linear support vector machines
J. Mach. Learn. Res.
A dual coordinate descent method for large-scale linear SVM
Randomized methods for linear constraints: convergence rates and conditioning
Math. Oper. Res.
Efficiency of coordinate descent methods on huge-scale optimization problems
SIAM J. Optim.
Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
Math. Program.
On the complexity analysis of randomized block-coordinate descent methods
Math. Program.
On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators
Math. Program.
Cited by (2)
Sensor placement optimization of blind source separation in a wireless acoustic sensor network via hybrid descent methods
2022, Applied AcousticsCitation Excerpt :Evidently, the proposed optimized BSS system outperforms the unoptimized case by at least 10 dB across the RTs. It is interesting to note that the performance improvement is fairly consistent even up the case of RT = 500 ms. To further compare the quality improvement of the enhanced speech, Figs. 4b, 5b and 6b plot the perceptual evaluation of speech quality (PESQ) [35] measure for the received and separated speech (enhanced speech). Again, the results demonstrate that the PESQ score of the proposed system is greatly improved especially for lower values of RTs.
Distributed Kalman Filtering for Speech Dereverberation and Noise Reduction in Acoustic Sensor Networks
2023, IEEE Sensors Journal
Zhibao Li received the B.S. degree from the Anhui Normal University, Wuhu, China, in 2007, the M.S. degree from the Nanjing Normal University, Nanjing, China, in 2010, and the Ph.D degree from The Hong Kong Polytechnic University, Hong Kong, China, in 2014. From 2014 to 2016, he was a Post-Doctoral Researcher with the Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China.
He is currently an Associate Professor with the School of Mathematics and Statistics, Central South University, Changsha, China. His research interests include the optimization methods for beamformer design, numerical methods for tensor equations and hemivariational inequalities.
Yu-Hong Dai received the B.Sc. degree in applied mathematics from the Beijing Institute of Technology, Beijing, China, in 1992. He then studied nonlinear optimization in the Institute of Computational Mathematics and Scientific/Engineering Computing, Chinese Academy of Sciences, and received the Ph.D. degree in nonlinear programming in 1997. After his graduation, he worked in the Academy of Mathematics and Systems Science (AMSS), Chinese Academy of Sciences, Beijing, China, and became a Full Professor in 2006. He is currently Director of the Center for Optimization and Applications (COA) of AMSS. His research interests include nonlinear optimization, integer programming and various optimization applications.
Dr. Dai has held editorial positions for several journals, including International Transactions in Operational Research, Journal of Global Optimization, Science China: Mathematics. He received the Second Prize of the National Natural Science of China in 2006 (rank 2), and the tenth Science and Technology Award for Chinese Youth in 2007. He also won the China National Funds for Distinguished Young Scientists in 2011.
Sven Nordholm (Senior Member of IEEE) received his PhD in Signal Processing in 1992, Licentiate of engineering 1989 and MscEE (Civilingenjör) 1983 all from Lund University, Sweden. Since 1999, Nordholm is Professor, Signal Processing, School of Electrical and Computer Engineering, Curtin University. He is a co-founder of two start-up companies; Sensear, providing voice communication in extreme noise conditions and Nuheara a hearables company. He was a lead editor for a special issue on assistive listing techniques in IEEE Signal Processing Magazine and several other EURASIP special issues. He is a former Associate editor for Eurasip Advances in Signal Processing and Journal of Franklin Institute and at the current time an Associate Editor IEEE/ACM Transactions on Audio, Speech and Language Processing.
His primary research has encompassed the fields of Speech Enhancement, Adaptive and Optimum Microphone Arrays, Audio Signal Processing and WirelessCommunication. He has written more than 200 papers in refereed journals and conference proceedings. He frequently contributes to book chapters and encyclopaedia articles. He is holding seven patents in the area of speech enhancement and microphone arrays.
Cedric Yiu received his M.Sc. from University of Dundee and University of London, and D.Phil. from University of Oxford. He had worked closely with the industry on different projects in University of Oxford and University College of London. He started his lecturing career in the University of Hong Kong. He is currently working in the Hong Kong Polytechnic University. He has served on the program committee and the organizing committee of a number of conferences and has organized a number of special sessions in conferences. He has published over 100 journal publications and given over 30 conference presentations. He holds two U.S. patents in signal processing. He received the third prize of Chongqing Natural Science Foundation Award in 2014. He also received the Donald Julius Groen Prize back in 2002. He is currently working on several research projects related to optimization and data analysis, signal processing system, risk management and high frequency trading. His current research interests include optimization and optimal control, signal processing and financial risk management.