Our goal in the paper is to address the following problem: From an unknown matrix, we are given inner products of that matrix with a set of prescribed matrices, and wish to find the unknown matrix. We shall consider this problem by using the notion of minimal norm interpolation. A fixed-point proximity algorithm for solving this problem will be developed.

In this paper, we study the spectral property of a class of self-affine measures μR,D on R2 generated by the iterated function system {ϕd(⋅)=R−1(⋅+d)}d∈D associated with the real expanding matrix R=(b100b2) and the digit set D={(00),(10),(01)}. We show that μR,D is a spectral measure if and only if 3|bi, i=1,2. This extends the result of Deng and Lau [J. Funct. Anal., 2015], where they considered the case b1=b2. And we also give a tree structure for any spectrum of μR,D by providing a decomposition property on it.

We consider the class of Rudin–Shapiro-like polynomials, whose L4 norms on the complex unit circle were studied by Borwein and Mossinghoff. The polynomial f(z)=f0+f1z+⋯+fdzd is identified with the sequence (f0,f1,…,fd) of its coefficients. From the L4 norm of a polynomial, one can easily calculate the autocorrelation merit factor of its associated sequence, and conversely. In this paper, we study the crosscorrelation properties of pairs of sequences associated to Rudin–Shapiro-like polynomials. We find an explicit formula for the crosscorrelation merit factor. A computer search is then used to find pairs of Rudin–Shapiro-like polynomials whose autocorrelation and crosscorrelation merit factors are simultaneously high. Pursley and Sarwate proved a bound that limits how good this combined autocorrelation and crosscorrelation performance can be. We find infinite families of polynomials whose performance approaches quite close to this fundamental limit.

In this paper, we introduce the prolate spheroidal wave functions (PSWFs) of real order α>−1 on the unit ball in arbitrary dimension, termed as ball PSWFs. They are eigenfunctions of both an integral operator, and a Sturm–Liouville differential operator. Different from existing works on multi-dimensional PSWFs, the ball PSWFs are defined as a generalization of orthogonal ball polynomials in primitive variables with a tuning parameter c>0, through a “perturbation” of the Sturm–Liouville equation of the ball polynomials. From this perspective, we can explore some interesting intrinsic connections between the ball PSWFs and the finite Fourier and Hankel transforms. We provide an efficient and accurate algorithm for computing the ball PSWFs and the associated eigenvalues, and present various numerical results to illustrate the efficiency of the method. Under this uniform framework, we can recover the existing PSWFs by suitable variable substitutions.

This paper studies sensor calibration in spectral estimation where the true frequencies are located on a continuous domain. We consider a uniform array of sensors that collects measurements whose spectrum is composed of a finite number of frequencies, where each sensor has an unknown calibration parameter. Our goal is to recover the spectrum and the calibration parameters simultaneously from multiple snapshots of the measurements. In the noiseless case with an infinite number of snapshots, we prove uniqueness of this problem up to certain trivial, inevitable ambiguities based on an algebraic method, as long as there are more sensors than frequencies. We then analyze the sensitivity of this algebraic technique with respect to the number of snapshots and noise. We next propose an optimization approach that makes full use of the measurements by minimizing a non-convex objective which is non-negative and continuously differentiable over all calibration parameters and Toeplitz matrices. We prove that, in the case of infinite snapshots and noiseless measurements, the objective vanishes only at equivalent solutions to the true calibration parameters and the measurement covariance matrix. The objective is minimized using Wirtinger gradient descent which is proven to converge to a critical point. We show empirically that this critical point provides a good approximation of the true calibration parameters and the underlying frequencies.

Starting from measured data, we develop a method to compute the fine structure of the spectrum of the Koopman operator with rigorous convergence guarantees. The method is based on the observation that, in the measure-preserving ergodic setting, the moments of the spectral measure associated to a given observable are computable from a single trajectory of this observable. Having finitely many moments available, we use the classical Christoffel–Darboux kernel to separate the atomic and absolutely continuous parts of the spectrum, supported by convergence guarantees as the number of moments tends to infinity. In addition, we propose a technique to detect the singular continuous part of the spectrum as well as two methods to approximate the spectral measure with guaranteed convergence in the weak topology, irrespective of whether the singular continuous part is present or not. The proposed method is simple to implement and readily applicable to large-scale systems since the computational complexity is dominated by inverting an N×N Hermitian positive-definite Toeplitz matrix, where N is the number of moments, for which efficient and numerically stable algorithms exist; in particular, the complexity of the approach is independent of the dimension of the underlying state-space. We also show how to compute, from measured data, the spectral projection on a given segment of the unit circle, allowing us to obtain a finite approximation of the operator that explicitly takes into account the point and continuous parts of the spectrum. Finally, we describe a relationship between the proposed method and the so-called Hankel Dynamic Mode Decomposition, providing new insights into the behavior of the eigenvalues of the Hankel DMD operator. A number of numerical examples illustrate the approach, including a study of the spectrum of the lid-driven two-dimensional cavity flow.

This paper is concerned with the problem of reconstructing an infinite-dimensional signal from a limited number of linear measurements. In particular, we show that for binary measurements (modelled with Walsh functions and Hadamard matrices) and wavelet reconstruction the stable sampling rate is linear. This implies that binary measurements are as efficient as Fourier samples when using wavelets as the reconstruction space. Powerful techniques for reconstructions include generalized sampling and its compressed versions, as well as recent methods based on data assimilation. Common to these methods is that the reconstruction quality depends highly on the subspace angle between the sampling and the reconstruction space, which is dictated by the stable sampling rate. As a result of the theory provided in this paper, these methods can now easily use binary measurements and wavelet reconstruction bases.

Last decade saw the creation of a number of directional representation dictionaries that desire to address the weaknesses of the classical wavelet transform that arise due to its limited capacity for the analysis of edge-like features of two-dimensional signals. Salient features of these dictionaries are directional selectivity and anisotropic treatment of the axes, achieved through the parabolic scaling law. In this paper we will examine the adequacy of such dictionaries for the analysis of edge- and corner-like features of 2D regions through a comprehensive framework for directional parabolic dictionaries, called the continuous parabolic molecules. This work builds on a family of earlier studies and aims to give a broader perspective through the level of generality.

In a recent paper, Flandrin [16] proposed filtering based on the zeros of a spectrogram with Gaussian window. His results are based on empirical observations on the distribution of the zeros of the spectrogram of white Gaussian noise. These zeros tend to be uniformly spread over the time–frequency plane, and not to clutter. Our contributions are threefold: we rigorously define the zeros of the spectrogram of continuous white Gaussian noise, we explicitly characterize their statistical distribution, and we investigate the computational and statistical underpinnings of the practical implementation of signal detection based on the statistics of the zeros of the spectrogram. The crux of our analysis is that the zeros of the spectrogram of white Gaussian noise correspond to the zeros of a Gaussian analytic function, a topic of recent independent mathematical interest [24].

It was recently shown that on a large class of important Banach spaces there exist no linear methods which are able to approximate the Hilbert transform from samples of the given function. This implies that there is no linear algorithm for calculating the Hilbert transform which can be implemented on a digital computer and which converges for all functions from the corresponding Banach spaces. The present paper develops a much more general framework which also includes non-linear approximation methods. All algorithms within this framework have only to satisfy an axiom which guarantees the computability of the algorithm based on given samples of the function. The paper investigates whether there exists an algorithm within this general framework which converges to the Hilbert transform for all functions in these Banach spaces. It is shown that non-linear methods give actually no improvement over linear methods. Moreover, the paper discusses some consequences regarding the Turing computability of the Hilbert transform and the existence of computational bases in Banach spaces.

Parallel acquisition systems are employed successfully in a variety of different sensing applications when a single sensor cannot provide enough measurements for a high-quality reconstruction. In this paper, we consider compressed sensing (CS) for parallel acquisition systems when the individual sensors use subgaussian random sampling. Our main results are a series of uniform recovery guarantees which relate the number of measurements required to the basis in which the solution is sparse and certain characteristics of the multi-sensor system, known as sensor profile matrices. In particular, we derive sufficient conditions for optimal recovery, in the sense that the number of measurements required per sensor decreases linearly with the total number of sensors, and demonstrate explicit examples of multi-sensor systems for which this holds. We establish these results by proving the so-called Asymmetric Restricted Isometry Property (ARIP) for the sensing system and use this to derive both nonuniversal and universal recovery guarantees. Compared to existing work, our results not only lead to better stability and robustness estimates but also provide simpler and sharper constants in the measurement conditions. Finally, we show how the problem of CS with block-diagonal sensing matrices can be viewed as a particular case of our multi-sensor framework. Specializing our results to this setting leads to a recovery guarantee that is at least as good as existing results.

In this article, we investigate the privacy issues that arise from a new frame-based kernel analysis approach to reconstruct from frame coefficient erasures. We show that while an erasure recovery matrix is needed in addition to a decoding frame for a receiver to recover the erasures, the erasure recovery matrix can be designed in such a way that it protects the encoding frame. The set of such erasure recovery matrices is shown to be an open and dense subset of a certain matrix space. We present algorithms to construct concrete examples of encoding frame and erasure recovery matrix pairs for which the erasure reconstruction process is robust to additive channel noise. Using the Restricted Isometry Property, we also provide quantitative bounds on the amplification of sparse additive channel noise. Numerical experiments are presented on the amplification of additive normally distributed random channel noise. In both cases, the amplification factors are demonstrated to be quite small.

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. Deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation or generalization ability of deep learning methods generated by the network architectures such as deep convolutional neural networks. Here we show that a deep convolutional neural network (CNN) is universal, meaning that it can be used to approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough. This answers an open question in learning theory. Our quantitative estimate, given tightly in terms of the number of free parameters to be computed, verifies the efficiency of deep CNNs in dealing with large dimensional data. Our study also demonstrates the role of convolutions in deep CNNs.

In recent years, correntropy and its applications in machine learning have been drawing continuous attention owing to its merits in dealing with non-Gaussian noise and outliers. However, theoretical understanding of correntropy, especially in the learning theory context, is still limited. In this study, we investigate correntropy based regression in the presence of non-Gaussian noise or outliers within the statistical learning framework. Motivated by the practical way of generating non-Gaussian noise or outliers, we introduce mixture of symmetric stable noise, which include Gaussian noise, Cauchy noise, and their mixture as special cases, to model non-Gaussian noise or outliers. We demonstrate that under the mixture of symmetric stable noise assumption, correntropy based regression can learn the conditional mean function or the conditional median function well without resorting to the finite-variance or even the finite first-order moment condition on the noise. In particular, for the above two cases, we establish asymptotic optimal learning rates for correntropy based regression estimators that are asymptotically of type O(n−1). These results justify the effectiveness of the correntropy based regression estimators in dealing with outliers as well as non-Gaussian noise. We believe that the present study makes a step forward towards understanding correntropy based regression from a statistical learning viewpoint, and may also shed some light on robust statistical learning for regression.

This paper proposes the multiresolution mode decomposition (MMD) as a novel model for adaptive time series analysis. The main conceptual innovation is the introduction of the multiresolution intrinsic mode function (MIMF) of the form∑n=−N/2N/2−1ancos⁡(2πnϕ(t))scn(2πNϕ(t))+∑n=−N/2N/2−1bnsin⁡(2πnϕ(t))ssn(2πNϕ(t)) to model nonlinear and non-stationary data with time-dependent amplitudes, frequencies, and waveforms. The multiresolution expansion coefficients {an}, {bn}, and the shape function series {scn(t)} and {ssn(t)} provide innovative features for adaptive time series analysis. For complex signals that are a superposition of several MIMFs with well-differentiated phase functions ϕ(t), a new recursive scheme based on Gauss-Seidel iteration and diffeomorphisms is proposed to identify these MIMFs, their multiresolution expansion coefficients, and shape function series. Numerical examples from synthetic data and natural phenomena are given to demonstrate the power of this new method.

Strictly proper kernel scores are well-known tools in probabilistic forecasting, while characteristic kernels have been extensively investigated in machine learning. We show that both notions coincide, so insights from one part of the literature can be used in the other. We show that the metric induced by a characteristic kernel cannot reliably distinguish between distributions that are far apart in total variation norm as soon as the underlying space of measures is infinite dimensional. We describe characteristic kernels in terms of eigenvalues and eigenfunctions and apply this characterization to continuous kernels on (locally) compact spaces. In the compact case, we show that characteristic kernels exist if and only if the space is metrizable. As special cases we investigate translation-invariant kernels on compact Abelian groups and isotropic kernels on spheres. The latter are of interest for forecast evaluation of probabilistic predictions on spherical domains as encountered in meteorology and climatology.

The Davenport spectrum is a modification of the classical Kolmogorov spectrum for the inertial range of turbulence that accounts for non-scaling low frequency behavior. Like the classical fractional Brownian motion vis-à-vis the Kolmogorov spectrum, tempered fractional Brownian motion (tfBm) is a new model that displays the Davenport spectrum. The autocorrelation of the increments of tfBm displays semi-long range dependence (hyperbolic and quasi-exponential decays over moderate and large scales, respectively), a phenomenon that has been observed in a wide range of applications from wind speeds to geophysics to finance. In this paper, we use wavelets to construct the first estimation method for tfBm and a simple and computationally efficient test for fBm vs tfBm alternatives. The properties of the wavelet estimator and test are mathematically and computationally established. An application of the methodology shows that tfBm is a better model than fBm for a geophysical flow data set.

In fast direct solvers for integral equations with the Laplace kernel, a hierarchical compression process needs to compute interpolative decompositions of off-diagonal block rows of the discretized integral operators. This computation can be dramatically accelerated by a technique called the proxy surface method, which is motivated by potential theory. We present a long overdue, rigorous error analysis of this acceleration technique. The analysis provides theoretical guidance for the discretization of the proxy surface used in the technique.

Dimensionality reduction is in demand to reduce the complexity of solving large-scale problems with data lying in latent low-dimensional structures in machine learning and computer vision. Motivated by such need, in this work we study the Restricted Isometry Property (RIP) of Gaussian random projections for low-dimensional subspaces in RN, and rigorously prove that the projection Frobenius norm distance between any two subspaces spanned by the projected data in Rn (n

We consider a causal FIR symmetric paraunitary(PU) matrix extension with parameters, and the constructions of symmetric orthogonal wavelets and symmetric tight framelets with an integer dilation factor M⩾2. Firstly, we propose an algorithm for factorizing a causal FIR symmetric Laurent polynomial vector into the product of order-one paraunitary matrices and constant vector. Secondly, based on the factorization algorithm, we propose a method for a causal symmetric PU extension with parameters of a Laurent polynomial vector. This method enables us to parameterize polyphase matrix whose first row is equal to the Laurent polynomial vector composed of polyphase components of given lowpass filter. And this symmetric PU extension provides a minimal factorization structure. Thirdly, we consider the constructions of symmetric orthogonal wavelets and symmetric tight framelets with integer dilation factor by the causal symmetric PU extension. Finally, several examples are provided to illustrate the construction methods proposed in this paper.

We present a numerical algorithm for an accurate and efficient computation of the convolution of the frequency domain layered media Green's function with a given density function. Instead of compressing the convolution matrix directly as in the classical fast multipole, fast direct solver, and H-matrix algorithms, this new algorithm considers a translated form of the original matrix so existing blocks from the highly optimized free-space fast multipole method can be easily adapted to the layered media Green's function. An asymptotic analysis is performed on the Sommerfeld integrals to provide an estimate of the decay rate in the new “multipole” and “local” expansions. To avoid the highly oscillatory integrand in the original integral representations when the source and target are close to each other, mathematically equivalent alternative direction integral representations are introduced. The convergence of the new expansions and quadrature rules for the original and alternative direction representations are numerically validated.

The uncertainty product of a function is a quantity that measures the trade-off between the space and the frequency localization of the function. Its boundedness from below is the content of various uncertainty principles. In the present paper, functions over the n-dimensional sphere are considered. A formula is derived that expresses the uncertainty product of a continuous function in terms of its Fourier coefficients. It is applied to a directional derivative of a zonal wavelet, and the behavior of the uncertainty product of this function is discussed.

Knowing that the convergence of a multivariate subdivision scheme with a nonnegative mask can be characterized by whether or not some finite products of row-stochastic matrices induced by this mask have a positive column. However, the number of those products is exponential with respect to the size of matrices. For nonnegative univariate subdivision, this problem is completely solved. Thus, the convergence in this case can be checked in linear time with respect to the size of a square matrix. This paper will demonstrate the necessary and sufficient conditions for the convergence of some nonnegative bivariate subdivision schemes by means of the so-called connectivity of a square matrix, which is derived by a given mask. Moreover, the connectivity can be examined in linear time with respect to the size of this matrix.

The problem of reconstructing a function from the magnitudes of its frame coefficients has recently been shown to be never uniformly stable in infinite-dimensional spaces [5]. This result also holds for frames that are possibly continuous [2]. On the other hand, in finite-dimensional settings, unique solvability of the problem implies uniform stability. A prominent example of such a phase retrieval problem is the recovery of a signal from the modulus of its Gabor transform. In this paper, we study Gabor phase retrieval and ask how the stability degrades on a natural family of finite-dimensional subspaces of the signal domain L2(R). We prove that the stability constant scales at least quadratically exponentially in the dimension of the subspaces. Our construction also shows that typical priors such as sparsity or smoothness promoting penalties do not constitute regularization terms for phase retrieval.

The aim of matrix recovery is to recover P∈M⊂Fp×q from LA(P)=(Tr(A1TP),Tr(A2TP),…,Tr(ANTP))T with Aj∈Vj⊂Fp×q, which is raised in many areas. In this paper, we build up a framework for almost everywhere matrix recovery which means LA is almost everywhere injectivity on M. We mainly focus on the following question: how many measurements are needed to recover almost all the matrices in M? For the case where both M and Vj are algebraic varieties, we use the tools from algebraic geometry to study the question and present some results to address it under many different settings.

In this work we give a sense to the notion of orientation for self-similar Gaussian fields with stationary increments, based on a Riesz analysis of these fields, with isotropic zero-mean analysis functions. We propose a structure tensor formulation and provide an intrinsic definition of the orientation vector as eigenvector of this tensor. That is, we show that the orientation vector does not depend on the analysis function, but only on the anisotropy encoded in the spectral density of the field. Then, we generalize this definition to a larger class of random fields called localizable Gaussian fields, whose orientation is derived from the orientation of their tangent fields. Two classes of Gaussian models with prescribed orientation are studied in the light of these new analysis tools.

We prove sharp upper and lower bounds for generalized Calderón's sums associated to frames on LCA groups generated by affine actions of cocompact subgroup translations and general measurable families of automorphisms. The proof makes use of techniques of analysis on metric spaces, and relies on a counting estimate of lattice points inside metric balls. We will deduce as special cases Calderón-type inequalities for families of expanding automorphisms as well as for LCA-Gabor systems.

In this paper, we investigate the active set identification technique of ISTA and provide some good properties. An active set Newton-CG method is then proposed for ℓ1 optimization. Under appropriate conditions, we show that the proposed method is globally convergent with some nonmonotone line search. The numerical comparisons with several state-of-art methods demonstrate the efficiency of the proposed method.

This note discusses an interesting matrix factorization called the CUR Decomposition. We illustrate various viewpoints of this method by comparing and contrasting them in different situations. Additionally, we offer a new characterization of CUR decompositions which synergizes these viewpoints and shows that they are indeed the same in the exact decomposition case.

Kernel-based nonlinear dimensionality reduction methods, such as Local Linear Embedding (LLE) and Laplacian Eigenmaps, rely heavily upon pairwise distances or similarity scores, with which one can construct and study a weighted graph associated with the data set. When each individual data object carries additional structural details, however, the correspondence relations between these structures provide extra information that can be leveraged for studying the data set using the graph. Based on this observation, we generalize Diffusion Maps (DM) in manifold learning and introduce the framework of Horizontal Diffusion Maps (HDM). We model a data set with pairwise structural correspondences as a fibre bundle equipped with a connection. We demonstrate the advantage of incorporating such additional information and study the asymptotic behavior of HDM on general fibre bundles. In a broader context, HDM reveals the sub-Riemannian structure of high-dimensional data sets, and provides a nonparametric learning framework for data sets with structural correspondences.

We consider the problem of non-negative super-resolution, which concerns reconstructing a non-negative signal x=∑i=1kaiδti from m samples of its convolution with a window function ϕ(s−t), of the form y(sj)=∑i=1kaiϕ(sj−ti)+δj, where δj indicates an inexactness in the sample value. We first show that x is the unique non-negative measure consistent with the samples, provided the samples are exact. Moreover, we characterise non-negative solutions xˆ consistent with the samples within the bound ∑j=1mδj2≤δ2. We show that the integrals of xˆ and x over (ti−ϵ,ti+ϵ) converge to one another as ϵ and δ approach zero and that x and xˆ are similarly close in the generalised Wasserstein distance. Lastly, we make these results precise for ϕ(s−t) Gaussian. The main innovation is that non-negativity is sufficient to localise point sources and that regularisers such as total variation are not required in the non-negative setting.

An equiangular tight frame (ETF) is a type of optimal packing of lines. They are often represented as the columns of a short, fat matrix. In certain applications we want this matrix to be flat, that is, have unimodular entries. In particular, real flat ETFs are equivalent to self-complementary binary codes that achieve the Grey-Rankin bound. Some flat ETFs are (complex) Hadamard ETFs, meaning they arise by extracting rows from a (complex) Hadamard matrix. In this paper, we give some new results about flat ETFs. We give an explicit Naimark complement for all Steiner ETFs, which in turn implies that all Kirkman ETFs are possibly-complex Hadamard ETFs. This in particular produces a new infinite family of real flat ETFs. Another result establishes an equivalence between real flat ETFs and certain types of quasi-symmetric designs, resulting in a new infinite family of such designs.

We consider a compressed sensing problem in which both the measurement and the sparsifying systems are assumed to be frames (not necessarily tight) of the underlying Hilbert space of signals, which may be finite or infinite dimensional. The main result gives explicit bounds on the number of measurements in order to achieve stable recovery, which depends on the mutual coherence of the two systems. As a simple corollary, we prove the efficiency of nonuniform sampling strategies in cases when the two systems are not incoherent, but only asymptotically incoherent, as with the recovery of wavelet coefficients from Fourier samples. This general framework finds applications to inverse problems in partial differential equations, where the standard assumptions of compressed sensing are often not satisfied. Several examples are discussed, with a special focus on electrical impedance tomography.

A family of Gaussian analytic functions (GAFs) has recently been linked to the Gabor transform of Gaussian white noises [4]. This answered pioneering work by Flandrin [10], who observed that the zeros of the Gabor transform of white noise had a regular distribution and proposed filtering algorithms based on the zeros of a spectrogram. In this paper, we study in a systematic way the link between GAFs and a class of time-frequency transforms of Gaussian white noises. Our main observation is a correspondence between pairs (transform, GAF) and generating functions for classical orthogonal polynomials. This covers some classical time-frequency transforms, such as the Gabor transform and the Daubechies-Paul wavelet transform. It also unveils new windowed discrete Fourier transforms, which map white noises to fundamental GAFs. Moreover, we discuss subtleties in defining a white noise and its transform on infinite dimensional Hilbert spaces and its finite dimensional approximations.

There are two major routes to address inverse problems in signal and image processing, such as denoising or deblurring. A first route relies on Bayesian modeling, where prior probabilities embody models of both the distribution of the unknown variables and their statistical dependence with respect to the observed data. Estimation typically relies on the minimization of an expected loss (e.g. minimum mean squared error, or MMSE). The second route designs (often convex) optimization problems involving the sum of a data fidelity term and a penalty term promoting certain types of unknowns (e.g., sparsity, using an ℓ1 penalty). Well known relations between these two approaches have led to some widely spread misconceptions. In particular, while the so-called Maximum A Posteriori (MAP) estimate with a Gaussian noise model does lead to an optimization problem with a quadratic data-fidelity term, we disprove through explicit examples the common belief that the converse would be true. It was shown [7], [9] that for denoising in the presence of additive Gaussian noise, for any prior probability on the unknowns, MMSE estimation can be expressed as a penalized least squares problem, with the apparent characteristics of a MAP estimate with Gaussian noise and a (generally) different prior on the unknowns. In other words, the second route is rich enough to build all possible MMSE estimators associated to additive Gaussian noise via a well chosen penalty. We generalize these results beyond Gaussian denoising and characterize noise models for which the same phenomenon occurs. In particular, we prove that with (a variant of) Poisson noise and any prior probability on the unknowns, MMSE estimation can again be expressed as the solution of a penalized least squares optimization problem. For additive scalar denoising the phenomenon holds if and only if the noise distribution is log-concave. In particular, Laplacian denoising can (perhaps surprisingly) be expressed as the solution of a penalized least squares problem. In the multivariate case, the same phenomenon occurs when the noise model belongs to a particular subset of the exponential family. For multivariate additive denoising, the phenomenon holds if and only if the noise is white and Gaussian.

Spherical Gauss-Laguerre (SGL) basis functions, i.e., normalized functions of the type Ln−l−1(l+1/2)(r2)rlYlm(ϑ,φ), |m|≤l

The continuous wavelet transform (CWT)-based synchrosqueezing transform (SST) is a special type of the reassignment method which not only enhances the energy concentration of CWT in the time-frequency plane, but also separates the components of multicomponent signals. The “bump wavelet” and Morlet's wavelet are commonly used continuous wavelets for SST. There is a parameter in these wavelets which controls the widths of the time-frequency localization window. In most literature on SST, this parameter is a fixed positive constant. In this paper, we consider the CWT with a time-varying parameter (called the adaptive CWT) and the corresponding SST (called the adaptive SST). We also introduce the 2nd-order adaptive SST. We analyze the separation conditions for non-stationary multicomponent signals with the local approximation of linear frequency modulation mode. We derive well-separated conditions of a multicomponent signal based on the adaptive CWT. We propose methods to select the time-varying parameter so that the corresponding adaptive SSTs of the components of a multicomponent signal have sharp representations and are well-separated. We provide comparison experimental results to demonstrate the efficiency and robustness of the proposed adaptive SST in separating components of multicomponent signals with fast varying frequencies.

We generalize the scattering transform to graphs and consequently construct a convolutional neural network on graphs. We show that under certain conditions, any feature generated by such a network is approximately invariant to permutations and stable to signal and graph manipulations. Numerical results demonstrate competitive performance on relevant datasets.

In this paper, we consider the design of wavelet filters based on the Thiran's common-factor approach proposed in [13]. This approach aims at building finite impulse response filters of a Hilbert-pair of wavelets serving as real and imaginary part of a complex wavelet. Unfortunately it is not possible to construct wavelets which are both finitely supported and analytic. The wavelet filters constructed using the common-factor approach are then approximately analytic. Thus, it is of interest to control their analyticity. The purpose of this paper is to first provide precise and explicit expressions as well as easily exploitable bounds for quantifying the analytic approximation of this complex wavelet. Then, we prove the existence of such filters enjoying the classical perfect reconstruction conditions, with arbitrarily many vanishing moments.

Single particle cryo-electron microscopy (EM) is a method for determining the 3-D structure of macromolecules from many noisy 2-D projection images of individual macromolecules whose orientations and positions are random and unknown. The problem of orientation assignment for the images motivated work on multireference alignment. The recent non-unique games framework provides a representation theoretic approach to alignment over compact groups, and offers a convex relaxation with certificates of global optimality in some cases. One of the great opportunities in cryo-EM is studying heterogeneous samples, containing two or more distinct conformations of molecules. Taking advantage of this opportunity presents an algorithmic challenge: determining both the class and orientation of each particle. We generalize multireference alignment to a problem of alignment and classification, and propose to extend non-unique games to the problem of simultaneous alignment and classification with the goal of simultaneously classifying cryo-EM images and aligning them within their classes.

In this paper, we consider the phase retrieval problem for recovering a complex signal, given a number of observations on the magnitude of linear measurements. This problem has direct applications in X-ray crystallography, diffraction imaging and microscopy. Motivated by the extensively studied theory of (tight) wavelet frame and its great success in various applications, we propose a wavelet frame based model for phase retrieval using the balanced approach. A hybrid fidelity term is designed to deal with complicated noises and a hybrid penalty term is constructed for different pursuits of sparsity and smoothness. Consequently, a proximal alternating linearization algorithm is developed and its convergence is analyzed. In particular, our proposed algorithm updates both the internal weights in the hybrid penalty term and the penalty parameter balancing the fidelity and penalty terms in a data-driven way. Extensive numerical experiments show that our method is quite competitive with other existing algorithms. On one hand, our method can reconstruct the truth successfully from a small number of measurements even if the phase retrieval problem is ill-posed. On the other hand, our algorithm is very robust to different types of noise, including Gaussian noise, Poisson noise and their mixtures.

In Group Synchronization, one attempts to recover a collection of unknown group elements from noisy measurements of their pairwise differences. Several important problems in vision and data analysis reduce to group synchronization over various compact groups. Spectral Group Synchronization is a commonly used, robust algorithm for solving group synchronization problems, which relies on diagonalization of a block matrix whose blocks are matrix representations of the measured pairwise differences. Assuming uniformly distributed measurement errors, we present a rigorous analysis of the accuracy and noise sensitivity of spectral group synchronization algorithms over any compact group. We identify a noise threshold above which the performance of the algorithm completely breaks down. Below the threshold, we calculate an asymptotically exact formula for the accuracy, up to the rounding error, as a function of the noise level. We also provide a consistent risk estimate, allowing practitioners to estimate the method's accuracy from available measurements.

The robustness of manifold learning methods is often predicated on the stability of the Neumann Laplacian eigenfunctions under deformations of the assumed underlying domain. Indeed, many manifold learning methods are based on approximating the Neumann Laplacian eigenfunctions on a manifold that is assumed to underlie data, which is viewed through a source of distortion. In this paper, we study the stability of the first Neumann Laplacian eigenfunction with respect to deformations of a domain by a diffeomorphism. In particular, we are interested in the stability of the first eigenfunction on tall thin domains where, intuitively, the first Neumann Laplacian eigenfunction should only depend on the length along the domain. We prove a rigorous version of this statement and apply it to a machine learning problem in geophysical interpretation.

We introduce the prime coset sum method for constructing tight wavelet frames, which allows one to construct nonseparable multivariate tight wavelet frames with prime dilation, using a univariate lowpass mask with this same prime dilation as input. This method relies on the idea of finding a sum of hermitian squares representation for a nonnegative trigonometric polynomial related to the sub-QMF condition for the lowpass mask. We prove the existence of these representations under some conditions on the input lowpass mask, utilizing the special structure of the recently introduced prime coset sum method, which is used to generate the lowpass masks in our construction. We also prove guarantees on the vanishing moments of the wavelets arising from this method, some of which hold more generally.

We study (homogeneous and inhomogeneous) anisotropic Besov spaces associated to expansive dilation matrices A∈GL(d,R), with the goal of clarifying when two such matrices induce the same scale of Besov spaces. For this purpose, we first establish that anisotropic Besov spaces have an alternative description as decomposition spaces. This result allows to relate properties of function spaces to combinatorial properties of the underlying coverings. This principle is applied to the question of classifying dilation matrices. It turns out that the scales of homogeneous and inhomogeneous Besov spaces differ in the way they depend on the dilation matrix: Two matrices A,B that induce the same scale of homogeneous Besov spaces also induce the same scale of inhomogeneous spaces, but the converse of this statement is generally false. We give a complete characterization of the different types of equivalence in terms of the Jordan normal forms of A,B.

In this paper, we consider the recovery of group sparse signals corrupted by impulsive noise. In some recent literature, researchers have utilized stable data fitting models, like l1-norm, Huber penalty function and Lorentzian-norm, to substitute the l2-norm data fidelity model to obtain more robust performance. In this paper, a stable model is developed, which exploits the generalized lp-norm as the measure for the error for sparse reconstruction. In order to address this model, we propose an efficient alternative direction method of multipliers, which includes the proximity operator of lp-norm functions to the framework of Lagrangian methods. Besides, to guarantee the convergence of the algorithm in the case of 0≤p<1 (nonconvex case), we took advantage of a smoothing strategy. For both 0≤p<1 (nonconvex case) and 1≤p≤2 (convex case), we have derived the conditions of the convergence for the proposed algorithm. Moreover, under the block restricted isometry property with constant δτk0<τ/(4−τ) for 0<τ<4/3 and δτk0<(τ−1)/τ for τ≥4/3, a sharp sufficient condition for group sparse recovery in the presence of impulsive noise and its associated error upper bound estimation are established. Numerical results based on the synthetic block sparse signals and the real-world FECG signals demonstrate the effectiveness and robustness of new algorithm in highly impulsive noise.

Over the last decade, the sparse representation model has led to remarkable results in numerous signal and image processing applications. To incorporate the inherent structure of the data and account for the fact that not all support patterns are equally likely, this model was enriched by enforcing various structural sparsity patterns. One plausible such extension of classic sparse coding, instigated by the emergence of graph signal processing, is graph regularized sparse coding. This model explicitly considers the intrinsic geometrical structure of the data domain, and has been successfully employed in various applications. However, emphasis was given to developing algorithmic solutions, and to date, the theoretical foundations to this problem have been lagging behind. In this work, we fill this gap and present a novel theoretical analysis of the graph regularized sparse coding problem, providing worst-case guarantees for the stability of the obtained solution, as well as for the success of several pursuit techniques. Furthermore, we formulate the conditions for which the superiority of the graph regularized sparse coding solution over the structure-agnostic sparse coding counterpart is established.

Scalings in which the graph Laplacian approaches a differential operator in the large graph limit are used to develop understanding of a number of algorithms for semi-supervised learning; in particular, the probit algorithm, level set and kriging methods. Both optimization and Bayesian approaches are considered, based around a regularizing quadratic form found from an affine transformation of the Laplacian, raised to a possibly fractional, exponent. Conditions on the parameters defining this quadratic form are identified under which well-defined limiting continuum analogues of the optimization and Bayesian semi-supervised learning problems may be found, thereby shedding light on the design of algorithms in the large graph setting. The large graph limits of the optimization formulations are tackled through Γ-convergence, using the recently introduced TLp metric. The small labeling noise limits of the Bayesian formulations are also identified, and contrasted with pre-existing harmonic function approaches to the problem.

We consider the bilinear inverse problem of recovering two vectors, x and w, in RL from their entrywise product. For the case where the vectors have known signs and belong to known subspaces, we introduce the convex program BranchHull, which is posed in the natural parameter space that does not require an approximate solution or initialization in order to be stated or solved. Under the structural assumptions that x and w are members of known K and N dimensional random subspaces, we present a recovery guarantee for the noiseless case and a noisy case. In the noiseless case, we prove that the BranchHull recovers the vectors up to the inherent scaling ambiguity with high probability when L≫2(K+N). The analysis provides a precise upper bound on the coefficient for the sample complexity. In a noisy case, we show that with high probability the BranchHull is robust to small dense noise when L=Ω(K+N).

Holographic representations of data encode information in packets of equal importance that enable progressive recovery. The quality of recovered data improves as more and more packets become available. This progressive recovery of the information is independent of the order in which packets become available. Such representations are ideally suited for distributed storage and for the transmission of data packets over networks with unpredictable delays and or erasures. Several methods for holographic representations of signals and images have been proposed over the years and multiple description information theory also deals with such representations. Surprisingly, however, these methods had not been considered in the classical framework of optimal least-squares estimation theory, until very recently. We develop a least-squares approach to the design of holographic representation for stochastic data vectors, relying on the framework widely used in modeling signals and images.

This paper considers the problem of phase retrieval, where the goal is to recover a signal z∈Cn from the observations yi=|ai⁎z|, i=1,2,⋯,m. While many algorithms have been proposed, the alternating minimization algorithm is still one of the most commonly used and the simplest methods. Existing works have proved that when the observation vectors {ai}i=1m are sampled from a complex normal distribution CN(0,I), the alternating minimization algorithm recovers the underlying signal with a good initialization when m=O(n), or with random initialization when m=O(n2), and it is conjectured that random initialization succeeds with m=O(n) [26]. This work proposes a modified alternating minimization method in a batch setting and proves that when m=O(nlog5⁡n), the proposed algorithm with random initialization recovers the underlying signal with high probability. The proof is based on the observation that after each iteration of alternating minimization, with high probability, the correlation between the direction of the estimated signal and the direction of the underlying signal increases.

This paper discusses isotropic sparse regularization for a random field on the unit sphere S2 in R3, where the field is expanded in terms of a spherical harmonic basis. A key feature is that the norm used in the regularization term, a hybrid of the ℓ1 and ℓ2-norms, is chosen so that the regularization preserves isotropy, in the sense that if the observed random field is strongly isotropic then so too is the regularized field. The Pareto efficient frontier is used to display the trade-off between the sparsity-inducing norm and the data discrepancy term, in order to help in the choice of a suitable regularization parameter. A numerical example using Cosmic Microwave Background (CMB) data is considered in detail. In particular, the numerical results explore the trade-off between regularization and discrepancy, and show that substantial sparsity can be achieved along with small L2 error.

Distributed learning based on the divide and conquer approach is a powerful tool for big data processing. We introduce a distributed kernel gradient descent algorithm for the minimum error entropy principle and analyze its convergence. We show that the L2 error decays at a minimax optimal rate under some mild conditions. As a tool we establish some concentration inequalities for U-statistics which play pivotal roles in our error analysis.

Input data is high-dimensional while the intrinsic dimension of this data maybe low. Data analysis methods aim to uncover the underlying low dimensional structure imposed by the low dimensional hidden parameters. In general, uncovering these hidden parameters is achieved by utilizing distance metrics that considers the set of attributes as a single monolithic set. However, the transformation of a low dimensional phenomena into measurement of high dimensional observations can distort the distance metric. This distortion can affect the quality of the desired estimated low dimensional geometric structure. In this paper, we propose to utilize the redundancy in the feature domain by analyzing multiple subsets of features that are called views. The proposed methods utilize the consensus between different views to extract valuable geometric information that unifies multiple views about the intrinsic relationships among several different observations. This unification enhances the information better than what a single view or a simple concatenations of views can provide.

We show that in finite-dimensional nonlinear approximations, the best r-term approximant of a function f almost always exists over C but that the same is not true over R, i.e., the infimum inff1,…,fr∈D⁡‖f−f1−…−fr‖ is almost always attainable by complex-valued functions f1,…,fr in D, a set (dictionary) of functions (atoms) with some desired structures. Our result extends to functions that possess properties like symmetry or skew-symmetry under permutations of arguments. When D is the set of separable functions, this is the best rank-r tensor approximation problem. We show that over C, any tensor almost always has a unique best rank-r approximation. This extends to other notions of ranks such as symmetric and alternating ranks, to best r-block-terms approximations, and to best approximations by tensor networks. Applied to sparse-plus-low-rank approximations, we obtain that for any given r and k, a general tensor has a unique best approximation by a sum of a rank-r tensor and a k-sparse tensor with a fixed sparsity pattern; a problem arising in covariance estimation of Gaussian model with k observed variables conditionally independent given r hidden variables. The existential (but not uniqueness) part of our result also applies to best approximations by a sum of a rank-r tensor and a k-sparse tensor with no fixed sparsity pattern, and to tensor completion problems.

In this paper, we study the Nyström type subsampling for large-scale kernel methods to reduce the computational complexities of big data. We discuss the multi-penalty regularization scheme based on Nyström type subsampling which is motivated from well-studied manifold regularization schemes. We develop a theoretical analysis of the multi-penalty least-square regularization scheme under the general source condition in vector-valued function setting, therefore the results can also be applied to multi-task learning problems. We achieve the optimal minimax convergence rates of the multi-penalty regularization using the concept of effective dimension for the appropriate subsampling size. We discuss an aggregation approach based on the linear function strategy to combine various Nyström approximants. Finally, we demonstrate the performance of the multi-penalty regularization based on Nyström type subsampling on the Caltech-101 dataset for multi-class image classification and NSL-KDD benchmark dataset for intrusion detection problem.

Construction of multivariate tight framelets is known to be a challenging problem because it is linked to the difficult problem on sum of squares of multivariate polynomials in real algebraic geometry. Multivariate dual framelets with vanishing moments generalize tight framelets and are not easy to be constructed either, since their construction is related to syzygy modules and factorization of multivariate polynomials. On the other hand, compactly supported multivariate framelets with directionality or high vanishing moments are of interest and importance in both theory and applications. In this paper we introduce the notion of a quasi-tight framelet, which is a dual framelet, but behaves almost like a tight framelet. Let ϕ∈L2(Rd) be an arbitrary compactly supported real-valued M-refinable function with a general dilation matrix M and ϕˆ(0)=1 such that its underlying real-valued low-pass filter satisfies the basic sum rule. We first constructively prove by a step-by-step algorithm that we can always easily derive from the arbitrary M-refinable function ϕ a directional compactly supported real-valued quasi-tight M-framelet in L2(Rd) associated with a directional quasi-tight M-framelet filter bank, each of whose high-pass filters has one vanishing moment and only two nonzero coefficients. If in addition all the coefficients of its low-pass filter are nonnegative, then such a quasi-tight M-framelet becomes a directional tight M-framelet in L2(Rd). Furthermore, we show by a constructive algorithm that we can always derive from the arbitrary M-refinable function ϕ a compactly supported quasi-tight M-framelet in L2(Rd) with the highest possible order of vanishing moments. We shall also present a result on quasi-tight framelets whose associated high-pass filters are purely differencing filters with the highest order of vanishing moments. Several examples will be provided to illustrate our main theoretical results and algorithms in this paper.

The idea that signals reside in a union of low dimensional subspaces subsumes many low dimensional models that have been used extensively in the recent decade in many fields and applications. Until recently, the vast majority of works have studied each one of these models on its own. However, a recent approach suggests providing general theory for low dimensional models using their Gaussian mean width, which serves as a measure for the intrinsic low dimensionality of the data. In this work we use this novel approach to study a generalized version of the popular compressive sampling matching pursuit (CoSaMP) algorithm, and to provide general recovery guarantees for signals from a union of low dimensional linear subspaces, under the assumption that the measurement matrix is Gaussian. We discuss the implications of our results for specific models, and use the generalized algorithm as an inspiration for a new greedy method for signal reconstruction in a combined sparse-synthesis and cosparse-analysis model. We perform experiments that demonstrate the usefulness of the proposed strategy.

Many tight frames of interest are constructed via their Gramian matrix (which determines the frame up to unitary equivalence). Given such a Gramian, it can be determined whether or not the tight frame is projective group frame, i.e., is the projective orbit of some group G (which may not be unique). On the other hand, there is complete description of the projective group frames in terms of the irreducible projective representations of G. Here we consider the inverse problem of taking the Gramian of a projective group frame for a group G, and identifying the cocycle and constructing the frame explicitly as the projective group orbit of a vector v (decomposed in terms of the irreducibles). The key idea is to recognise that the Gramian is a group matrix given by a vector f∈CG, and to take the Fourier transform of f to obtain the components of v as orthogonal projections. This requires the development of a theory of group matrices and the Fourier transform for projective representations. Of particular interest, we give a block diagonalisation of (projective) group matrices. This leads to a unique Fourier decomposition of the group matrices, and a further fine-scale decomposition into low rank group matrices.

In this paper, we consider an infinite-dimensional phase retrieval problem to reconstruct real-valued signals living in a shift-invariant space from their phaseless samples taken either on the whole line or on a discrete set with finite sampling density. We characterize all phase retrievable signals in a real-valued shift-invariant space using their nonseparability. For nonseparable signals generated by some function with support length L, we show that they can be well approximated, up to a sign, from their noisy phaseless samples taken on a discrete set with sampling density 2L−1. In this paper, we also propose an algorithm with linear computational complexity to reconstruct nonseparable signals in a shift-invariant space from their phaseless samples corrupted by bounded noises.