In this paper, we generalize finite depth wavelet scattering transforms, which we formulate as Lq(Rn) norms of a cascade of continuous wavelet transforms (or dyadic wavelet transforms) and contractive nonlinearities. We then provide norms for these operators, prove that these operators are well-defined, and are Lipschitz continuous to the action of C2 diffeomorphisms in specific cases. Lastly, we extend

Given only a finite collection of points sampled from a Riemannian manifold embedded in a Euclidean space, in this paper we propose a new method to numerically solve elliptic and parabolic partial differential equations (PDEs) supplemented with boundary conditions. Since the construction of triangulations on unknown manifolds can be both difficult and expensive, both in terms of computational and data

We introduce new frames, called metaplectic Gabor frames, as natural generalizations of Gabor frames in the framework of metaplectic Wigner distributions, cf. [7], [8], [5], [17], [27], [28]. Namely, we develop the theory of metaplectic atoms in a full-general setting and prove an inversion formula for metaplectic Wigner distributions on Rd. Its discretization provides metaplectic Gabor frames. Next

In deep learning, it is common to use more network parameters than training points. In such scenario of over-parameterization, there are usually multiple networks that achieve zero training error so that the training algorithm induces an implicit bias on the computed solution. In practice, (stochastic) gradient descent tends to prefer solutions which generalize well, which provides a possible explanation

Multivariate time-series have become abundant in recent years, as many data-acquisition systems record information through multiple sensors simultaneously. In this paper, we assume the variables pertain to some geometry and present an operator-based approach for spatiotemporal analysis. Our approach combines three components that are often considered separately: (i) manifold learning for building operators

The aim of noisy phase retrieval is to estimate a signal x0∈Cd from m noisy intensity measurements bj=|〈aj,x0〉|2+ηj,j=1,…,m, where aj∈Cd are known measurement vectors and η=(η1,…,ηm)⊤∈Rm is a noise vector. A commonly used estimator for x0 is to minimize the intensity-based loss function, i.e., xˆ:=argminx∈Cd∑j=1m(|〈aj,x〉|2−bj)2. Although many algorithms for solving the intensity-based estimator have

It is frequently observed that overparameterized neural networks generalize well. Regarding such phenomena, existing theoretical work mainly devotes to linear settings or fully-connected neural networks. This paper studies the learning ability of an important family of deep neural networks, deep convolutional neural networks (DCNNs), under both underparameterized and overparameterized settings. We

Is it possible to detect if the sample paths of a stochastic process almost surely admit a finite expansion with respect to some/any basis? The determination is to be made on the basis of a finite collection of discretely/noisily observed sample paths. We show that it is indeed possible to construct a hypothesis testing scheme that is almost surely guaranteed to make only finite many incorrect decisions

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function spaces lacks stability. In this paper, we investigate an alternative representation of CPWL functions that relies on local hat basis functions and

We give a proof of that harmonic oscillator propagators and fractional Fourier transforms are essentially the same. We deduce continuity properties and fix time estimates for such operators on modulation spaces, and apply the results to prove Strichartz estimates for such propagators when acting on Pilipović and modulation spaces. Especially we extend some results by Balhara, Cordero, Nicola, Rodino

Signal processing is an important research topic. This paper aims to provide a general framework for signal processing on arbitrary dynamic graphs. We propose a new graph transformation by defining a temporal-attention product. This product transforms the sequence of graph time slices with arbitrary topology and number of nodes into a static graph, effectively capturing graph signals' spatio-temporal

This note considers the problem of approximating the locations of dominant spikes for a probability measure from noisy spectrum measurements under the condition of residue signal, significant noise level, and no minimum spectrum separation. We show that the simple procedure of thresholding the smoothed inverse Fourier transform allows for approximating the spike locations rather accurately.

We consider the problem of reconstructing a measurable function over a Locally Compact Abelian group G from random measurements. The results presented herein are partially inspired by the concept of alias-free sampling. Here, the sampling and interpolation operation is modelled as an approximate convolution operator with respect to a stochastic integral defined with an appropriately chosen random measure

In this paper, we introduce a sketching algorithm for constructing a tensor train representation of a probability density from its samples. Our method deviates from the standard recursive SVD-based procedure for constructing a tensor train. Instead, we formulate and solve a sequence of small linear systems for the individual tensor train cores. This approach can avoid the curse of dimensionality that

We study sharp frame bounds of Gabor systems over rectangular lattices for different windows and integer oversampling rate. In some cases we obtain optimality results for the square lattice, while in other cases the lattices optimizing the frame bounds and the condition number are rectangular lattices which are different for the respective quantities. Also, in some cases optimal lattices do not exist

This note is to point out a serious typo in [1] and clarify a notation in [2].

Abstract not available

This note is to study the proximity operator of hp=‖⋅‖1p, the power function of the ℓ1 norm. For general p, computing the proximity operator requires solving a system of potentially highly nonlinear inclusions. For p=1, the proximity operator of h1 is the well known soft-thresholding operator. For p=2, the function h2 serves as a penalty function that promotes structured solutions to optimization problems

The paper presents a versatile library of analytic and quasi-analytic complex-valued wavelet packets (WPs) which originate from discrete splines of arbitrary orders. The real parts of the quasi-analytic WPs are the regular spline-based orthonormal WPs designed in [4]. The imaginary parts are the so-called complementary orthonormal WPs, which, unlike the symmetric regular WPs, are antisymmetric. Tensor

By making a seminal use of the maximum modulus principle of holomorphic functions we prove existence of n-best kernel approximation for a wide class of reproducing kernel Hilbert spaces of holomorphic functions in the unit disc, and for the corresponding class of Bochner type spaces of stochastic processes. This study thus generalizes the classical result of n-best rational approximation for the Hardy

We consider the problem of recovering an unknown low-rank matrix X⋆ with (possibly) non-orthogonal, effectively sparse rank-1 decomposition from measurements y gathered in a linear measurement process A. We propose a variational formulation that lends itself to alternating minimization and whose global minimizers provably approximate X⋆ up to noise level. Working with a variant of robust injectivity

This article provides convergence analysis of online stochastic gradient descent algorithms for functional linear models. Adopting the characterizations of the slope function regularity, the kernel space capacity, and the capacity of the sampling process covariance operator, significant improvement on the convergence rates is achieved. Both prediction problems and estimation problems are studied, where

Nowadays we often meet with a learning problem when data are distributed on different machines connected via a network, instead of stored centrally. Here we consider decentralized supervised learning in a reproducing kernel Hilbert space. We note that standard gradient descent in a reproducing kernel Hilbert space is difficult to implement with multiple communications between worker machines. On the

The problem of synchronization over a group G aims to estimate a collection of group elements G1⁎,…,Gn⁎∈G based on noisy observations of a subset of all pairwise ratios of the form Gi⁎Gj⁎−1. Such a problem has gained much attention recently and finds many applications across a wide range of scientific and engineering areas. In this paper, we consider the class of synchronization problems in which the

We study a class of orbit recovery problems in which we observe independent copies of an unknown element of Rp, each linearly acted upon by a random element of some group (such as Z/p or SO(3)) and then corrupted by additive Gaussian noise. We prove matching upper and lower bounds on the number of samples required to approximately recover the group orbit of this unknown element with high probability

The fractional Hilbert transform was introduced by Zayed [30, Zayed, 1998] and has been widely used in signal processing. In view of its connection with the fractional Fourier transform, Chen, the first, second and fourth authors of this paper in [6, Chen et al., 2021] studied the fractional Hilbert transform and other fractional multiplier operators on the real line. The present paper is concerned

We introduce an efficient scheme for the construction of quadrature rules for bandlimited functions. While the scheme is predominantly based on well-known facts about prolate spheroidal wave functions of order zero, it has the asymptotic CPU time estimate O(nlog⁡n) to construct an n-point quadrature rule. Moreover, the size of the “nlog⁡n” term in the CPU time estimate is small, so for all practical

Recovery of sparse vectors and low-rank matrices from a small number of linear measurements is well-known to be possible under various model assumptions on the measurements. The key requirement on the measurement matrices is typically the restricted isometry property, that is, approximate orthonormality when acting on the subspace to be recovered. Among the most widely used random matrix measurement

We consider the finite alphabet phase retrieval problem: recovering a signal whose entries lie in a small alphabet of possible values from its Fourier magnitudes. This problem arises in the celebrated technology of X-ray crystallography to determine the atomic structure of biological molecules. Our main result states that for generic values of the alphabet, two signals have the same Fourier magnitudes

Given multiple point clouds, how to find the rigid transform (rotation, reflection, and shifting) such that these point clouds are well aligned? This problem, known as the generalized orthogonal Procrustes problem (GOPP), has found numerous applications in statistics, computer vision, and imaging science. While one commonly-used method is finding the least squares estimator, it is generally an NP-hard

In this short note, we propose a new method for quantizing the weights of a fully trained neural network. A simple deterministic pre-processing step allows us to quantize network layers via memoryless scalar quantization while preserving the network performance on given training data. On one hand, the computational complexity of this pre-processing slightly exceeds that of state-of-the-art algorithms

Frames formed by orbits of vectors through the iteration of a bounded operator have recently attracted considerable attention, in particular due to its applications to dynamical sampling. In this article, we consider two commuting bounded operators acting on some separable Hilbert space H. We completely characterize operators T and L with TL=LT and sets Φ⊂H such that the collection {TkLjϕ:k∈Z,j∈J,ϕ∈Φ}

Classical wavelet, wavelet packets and time-frequency dictionaries have been generalized to the graph setting, the main goal being to obtain atoms which are jointly localized both in the vertex and the graph spectral domain. We present a new method to generate a whole dictionary of frames of wavelet packets defined in the graph spectral domain to represent signals on weighted graphs. We will give some

This paper provides algebraic and analytic, as well as numerical arguments why and how double preconditioning of the Gabor frame operator yields an efficient method to compute approximate dual (respectively tight) Gabor atoms for a given time-frequency lattice. We extend the definition of the approach to the continuous setting, making use of the so-called Banach Gelfand Triple, based on the Segal algebra

This paper proposes a novel kernel-based optimization scheme to handle tasks in the analysis, e.g., signal spectral estimation and single-channel source separation of 1D non-stationary oscillatory data. The key insight of our optimization scheme for reconstructing the time-frequency information is that when a nonparametric regression is applied on some input values, the output regressed points would

We introduce the broad subclass of algebraic compressed sensing problems, where structured signals are modeled either explicitly or implicitly via polynomials. This includes, for instance, low-rank matrix and tensor recovery. We employ powerful techniques from algebraic geometry to study well-posedness of sufficiently general compressed sensing problems, including existence, local recoverability, global

One of the key problems in tensor completion is the number of uniformly random sample entries required for recovery guarantee. The main aim of this paper is to study n1×n2×n3 third-order tensor completion based on transformed tensor singular value decomposition, and provide a bound on the number of required sample entries. Our approach is to make use of the multi-rank of the underlying tensor instead

In this paper, we consider the problem of recovery of a burst-like forcing term in an initial value problem (IVP) in the framework of dynamical sampling. We introduce an idea of using two particular classes of samplers that allow one to predict the solution of the IVP over a time interval without a burst. This leads to two different algorithms that stably and accurately approximate the burst-like forcing

This paper extends the existing theory of perfect reconstruction two-channel filter banks from bipartite graphs to non-bipartite graphs. By generalizing the concept of downsampling/upsampling we establish the frame of two-channel filter bank on arbitrary connected, undirected and weighted graphs. Then the equations for perfect reconstruction of the filter banks are presented and solved under proper

This paper studies the approximation capacity of ReLU neural networks with norm constraint on the weights. We prove upper and lower bounds on the approximation error of these networks for smooth function classes. The lower bound is derived through the Rademacher complexity of neural networks, which may be of independent interest. We apply these approximation bounds to analyze the convergences of regression

We introduce an extension to local principal component analysis for learning symmetric manifolds. In particular, we use a spectral method to approximate the Lie algebra corresponding to the symmetry group of the underlying manifold. We derive the sample complexity of our method for various manifolds before applying it to various data sets for improved density estimation.

We study the problem of super-resolution of a linear combination of Dirac distributions and their derivatives on a one-dimensional circle from noisy Fourier measurements. Following numerous recent works on the subject, we consider the geometric setting of “partial clustering”, when some Diracs can be separated much below the Rayleigh limit. Under this assumption, we prove sharp asymptotic bounds for

It is well known that second order linear ordinary differential equations with slowly varying coefficients admit slowly varying phase functions. This observation is the basis of the Liouville-Green method and many other techniques for the asymptotic approximation of the solutions of such equations. More recently, it was exploited by the author to develop a highly efficient solver for second order linear

Let M be a smooth submanifold of Rn equipped with the Euclidean (chordal) metric. This note considers the smallest dimension m for which there exists a bi-Lipschitz function f:M↦Rm with bi-Lipschitz constants close to one. The main result bounds the best achievable embedding dimension m below in terms of the Lipschitz constants of f as well as the reach, volume, diameter, and dimension of M. This new

In this paper we present new constructive methods, random and deterministic, for the efficient subsampling of finite frames in Cm. Based on a suitable random subsampling strategy, we are able to extract from any given frame with bounds 0

This paper deals with the modeling of non-stationary signals, from the point of view of signal synthesis. A class of random, non-stationary signals, generated by synthesis from a random time-scale representation, is introduced and studied. Non-stationarity is implemented in the time-scale representation through a prior distribution which models the action of time warping on a stationary signal. A main

Factorization of matrices of Laurent polynomials plays an important role in mathematics and engineering such as wavelet frame construction and filter bank design. Wavelet frames (a.k.a. framelets) are useful in applications such as signal and image processing. Motivated by the recent development of quasi-tight framelets, we study and characterize generalized spectral factorizations with symmetry for

Expressing a matrix as the sum of a low-rank matrix plus a sparse matrix is a flexible model capturing global and local features in data. This model is the foundation of robust principle component analysis [1], [2], and popularized by dynamic-foreground/static-background separation [3]. Compressed sensing, matrix completion, and their variants [4], [5] have established that data satisfying low complexity

An equi-isoclinic tight fusion frame (EITFF) is a type of Grassmannian code, being a sequence of subspaces of a finite-dimensional Hilbert space of a given dimension with the property that the smallest spectral distance between any pair of them is as large as possible. EITFFs arise in compressed sensing, yielding dictionaries with minimal block coherence. Their existence remains poorly characterized

The paper is concerned with the sparse approximation of functions having hybrid regularity borrowed from the theory of solutions to electronic Schrödinger equations due to Yserentant (2004) [42]. We use hyperbolic wavelets to introduce corresponding new spaces of Besov- and Triebel-Lizorkin-type to particularly cover the energy norm approximation of functions with dominating mixed smoothness. Explicit

Particle tracking in a live cell environment is concerned with reconstructing the trajectories, locations, or velocities of the targeting particles, which holds the promise of revealing important new biological insights. The standard approach of particle tracking consists of two steps: first reconstructing statically the source locations in each time step, and second applying tracking techniques to

This paper examines the frame properties of finitely and infinitely iterated dyadic filter banks. It is shown that the stability of an infinitely iterated dyadic filter bank guarantees that of any associated finitely iterated dyadic filter bank with uniform bounds. Conditions under which the stability of finitely iterated dyadic filter banks with uniform bounds implies that of the infinitely iterated

Many applications, such as system identification, classification of time series, direct and inverse problems in partial differential equations, and uncertainty quantification lead to the question of approximation of a non-linear operator between metric spaces X and Y. We study the problem of determining the degree of approximation of such operators on a compact subset KX⊂X using a finite amount of

Multi-dimensional sampling approaches are often faced with certain intricacies as we need to deal with matrices and the non-commutativity of matrices precludes straightforward extensions of the usual one-dimensional results. In this article, we reformulate the Papoulis' sampling theorem for the reconstruction of higher-dimensional signals that are band-limited in the sense of free metaplectic transformation

We study the recovery of square-integrable signals from the absolute values of their wavelet transforms, also called wavelet phase retrieval. We present a new uniqueness result for wavelet phase retrieval. To be precise, we show that any wavelet with finitely many vanishing moments allows for the unique recovery of real-valued bandlimited signals up to global sign. Additionally, we present the first

The study of rare events in molecular and atomic systems such as conformal changes and cluster rearrangements has been one of the most important research themes in chemical physics. Key challenges are associated with long waiting times rendering molecular simulations inefficient, high dimensionality impeding the use of PDE-based approaches, and the complexity or breadth of transition processes limiting

Abstract not available

For a class F of complex-valued functions on a set D, we denote by gn(F) its sampling numbers, i.e., the minimal worst-case error on F, measured in L2, that can be achieved with a recovery algorithm based on n function evaluations. We prove that there is a universal constant c∈N such that, if F is the unit ball of a separable reproducing kernel Hilbert space, thengcn(F)2≤1n∑k≥ndk(F)2, where dk(F) are

We generalize the classical universal approximation theorem for neural networks to the case of complex-valued neural networks. Precisely, we consider feedforward networks with a complex activation function σ:C→C in which each neuron performs the operation CN→C,z↦σ(b+wTz) with weights w∈CN and a bias b∈C. We completely characterize those activation functions σ for which the associated complex networks

Inspired by the linear Schrödinger operator, we consider a generalized p-Laplacian operator on discrete graphs and present new results that characterize several spectral properties of this operator with particular attention to the nodal domain count of its eigenfunctions. Just like the one-dimensional continuous p-Laplacian, we prove that the variational spectrum of the discrete generalized p-Laplacian