Sparse representations and compressive sampling approaches in engineering mechanics: A review of theoretical concepts and diverse applications

Abstract

A review of theoretical concepts and diverse applications of sparse representations and compressive sampling (CS) approaches in engineering mechanics problems is provided from a broad perspective. First, following a presentation of well-established CS concepts and optimization algorithms, attention is directed to currently emerging tools and techniques for enhancing solution sparsity and for exploiting additional information in the data. These include alternative to ${\ell }_1$-norm minimization formulations and iterative re-weighting solution schemes, Bayesian approaches, as well as structured sparsity and dictionary learning strategies. Next, CS-based research work of relevance to engineering mechanics problems is categorized and discussed under three distinct application areas: a) inverse problems in structural health monitoring, b) uncertainty modeling and simulation, and c) computationally efficient uncertainty propagation. Notably, the vast majority of problems in all three areas share the challenge of “incomplete data”, addressed by the versatile CS framework. In this regard, incomplete data may manifest themselves in various different forms and can correspond to missing or compressed data, or even refer generally to insufficiently few function evaluations. The primary objective of this review paper relates to identifying and presenting significant contributions in each of the above three application areas in engineering mechanics, with the goal of expediting additional research and development efforts. To this aim, an extensive list of 248 references is provided, composed almost exclusively of books and archival papers, which can be readily available to a potential reader.

Introduction

The problem of determining the current and predicting the future states of a system based on knowledge of a limited number of data points has been a persistent challenge in a wide range of scientific fields. Advancements in this direction have led to various significant theoretical results, which have unequivocally revolutionized modern science. One of the most characteristic examples relates to the development of representations based on Fourier series [1]. This trigonometric series expansion of periodic functions has served as the starting point for various efficient expansion and representation schemes (e.g., [2]). During the past fifteen years, research efforts have focused on identifying and exploiting low-dimensional representations of high-dimensional data, as well as on establishing conditions guaranteeing unique representation in the low-dimensional space. This has triggered the birth of the currently expanding field of compressive sampling (CS) (e.g., [3], [4]), as well as the rejuvenation of the more general field of sparse representations (e.g., [5], [6]).

From a historical perspective, there have been several examples and early observations suggesting that signal reconstruction is possible by utilizing a smaller number of samples than the minimum dictated by the Shannon–Nyquist (SN) theorem (e.g., [2], [7]). Indicatively, Carathéodory showed in [8], [9] that a signal expressed as a sum of any $k$ sinusoids can be recovered based on knowledge of its values at zero time and at any other $2k$ time points. Further, Beurling [10] discussed the possibility of extrapolating in a nonlinear manner and determining the complete Fourier transform of a signal assuming that only part of the Fourier transform is known. Dorfman [11] studied the combinatorial group testing problem and provided one of the first sparse signal recovery problem formulations. Also, Logan [12] showed that it is possible to reconstruct a band-limited corrupted signal by an ${\ell }_1$-norm minimization approach. These early, seemingly paradoxical, results were further supported by relevant studies in the field of geophysics [13], [14], [15] (see also [16]) pertaining to the analysis of seismic signals of spike train form due to the layered structure of geological formations. It was shown that these sparse spike trains can be recovered accurately based on incomplete and noisy measurements.

Nevertheless, it can hardly be disputed that sparse representations theory and tools have been revitalized in recent years due to the pioneering work in [17], [18], [19], which provided bounds on the number of measurements required for the recovery of high-dimensional data under the condition that the latter possess a low-dimensional representation in a transformed domain. The aforementioned theoretical results, coupled with potent numerical algorithms from the well-established field of convex optimization, have led to numerous impactful contributions in a wide range of application areas. In this regard, CS-related theoretical advancements and diverse applications associated with the fields of signal and image processing, biomedicine, communication systems and sensor networks, information security and pattern recognition have been well-documented in dedicated books (e.g., [3], [4], [20], [21], [22], [23], [24], [25], [26]), special issues (e.g., [27]) and review papers (e.g., [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43]).

More recently, the field of engineering mechanics has also benefited from the advent of sparse representations and CS approaches in conjunction with uncertainty quantification and health monitoring of diverse systems and structures. However, to the best of the authors’ knowledge, there are currently no review papers providing a comprehensive discussion and a broad perspective on the aforementioned developments in engineering mechanics. In fact, although there have been a couple of relevant efforts reported previously, these focus either on specific and relatively narrow application domains, or are cross-disciplinary in nature and lack any focus on a specific research field. Indicatively, the authors in [44] focus exclusively on reviewing polynomial chaos expansions coupled with CS approaches as applied in stochastic mechanics problems, whereas reference [45] discusses the problem of CS-based governing dynamics modeling of complex systems with applications in interdisciplinary science and engineering.

In this regard, in an effort to address this gap in the literature and to complement some of the previous works by incorporating more recent developments, this review paper focuses on sparse representations and CS approaches in the field of engineering mechanics. Specifically, in Section 2, following a presentation of well-established CS concepts and optimization algorithms, attention is directed to currently emerging tools and techniques for enhancing solution sparsity and for exploiting additional information in the data. These include alternative to ${\ell }_1$-norm minimization formulations and iterative re-weighting solution schemes, Bayesian approaches, as well as structured sparsity and dictionary learning strategies. Next, in Section 3, a rather broad perspective is provided on CS-related contributions to engineering mechanics, and relevant research work is categorized under three distinct application areas: (a) inverse problems in structural health monitoring, (b) uncertainty modeling and simulation, and (c) computationally efficient uncertainty propagation. Notably, the vast majority of problems in all three areas share the challenge of “incomplete data”, addressed by the versatile CS framework. In this regard, incomplete data may manifest themselves in various different forms and can correspond to missing or compressed data, or even refer generally to insufficiently few function evaluations. Further, concluding remarks are presented in Section 4. It is noted that the primary objective of this review paper relates to identifying and discussing significant contributions in each of the above three application areas in engineering mechanics, with the goal of expediting additional research and development efforts. To this aim, an extensive list of $248$ references is provided, composed almost exclusively of books and archival papers, which can be readily available to a potential reader.