Skip to main content
SpringerLink
Log in

Gabor Neural Networks with Proven Approximation Properties

  • Published:
The Journal of Geometric Analysis Aims and scope Submit manuscript

Abstract

In this paper, we propose a new type of a neural network which is inspired by Gabor systems from harmonic analysis. In this regard, we construct a class of sparsely connected neural networks utilizing the concept of time–frequency shifts, and we show that their approximation error rates can be tied to the number of modulations in the corresponding Gabor frame and to the smoothness of the input function. Furthermore, we show that such networks are easily implementable and we illustrate their performance with some numerical examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436 (2015)

    Article  Google Scholar 

  2. Balan, R., Singh, M., Zou, D.: Lipschitz properties for deep convolutional networks. (2017) arXiv preprint arXiv:1701.05217

  3. Shaham, U., Cloninger, A., Coifman, R.R.: Provable approximation properties for deep neural networks. Appl. Comput. Harmon. Anal. 44, 537–557 (2016)

    Article  MathSciNet  Google Scholar 

  4. Bolcskei, H., Grohs, P., Kutyniok, G., Petersen, P.: Optimal approximation with sparsely connected deep neural networks. SIAM J. Math. Data Sci. 1, 8–45 (2019)

    Article  MathSciNet  Google Scholar 

  5. Mallat, S.: Group invariant scattering. Commun. Pure Appl. Math. 65, 1331–1398 (2012)

    Article  MathSciNet  Google Scholar 

  6. Mallat, S.: Understanding deep convolutional networks. Philos. Trans. R. Soc. A 374, 20150203 (2016)

    Article  Google Scholar 

  7. Wiatowski, T., Bölcskei, H.: A mathematical theory of deep convolutional neural networks for feature extraction. IEEE Trans. Inf. Theory 64, 1845–1866 (2017)

    Article  MathSciNet  Google Scholar 

  8. Labate, D.: A unified characterization of reproducing systems generated by a finite family. J. Geomet. Anal. 12, 469–491 (2002)

    Article  MathSciNet  Google Scholar 

  9. Hernández, E., Labate, D., Weiss, G.: A unified characterization of reproducing systems generated by a finite family, ii. J. Geomet. Anal. 12, 615–662 (2002)

    Article  MathSciNet  Google Scholar 

  10. Hernández, E., Labate, D., Weiss, G., Wilso, E.: Oversampling, quasi affine frames and wave packets. Appl. Comput. Harmon. Anal. 16, 111–147 (2004)

    Article  MathSciNet  Google Scholar 

  11. Labate, D., Weiss, G.: Continuous and discrete reproducing systems that arise from translations: theory and applications of composite wavelets. Four Short Courses on Harmonic Analysis, pp. 87–130. Berlin, Springer (2010)

    Chapter  Google Scholar 

  12. Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303–314 (1989)

    Article  MathSciNet  Google Scholar 

  13. Hornik, K.: Approximation capabilities of multilayer feedforward networks. Neural Netw. 4, 251–257 (1991)

    Article  Google Scholar 

  14. Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39, 930–945 (1993)

    Article  MathSciNet  Google Scholar 

  15. Mhaskar, H.N.: Neural networks for optimal approximation of smooth and analytic functions. Neural Comput. 8, 164–177 (1996)

    Article  Google Scholar 

  16. Chui, C.K., Mhaskar, H.N.: Deep nets for local manifold learning. arXiv preprint arXiv:1607.07110(2016)

  17. Czaja, W., Manning, B., Murphy, J.M., Stubbs, K.: Discrete directional Gabor frames. Appl. Comput. Harmon. Anal. 45, 1–21 (2016)

    Article  MathSciNet  Google Scholar 

  18. Czaja, W., Li, W.: Analysis of time–frequency scattering transforms. Appl. Comput. Harmon. Anal. 47, 149–171 (2019)

    Article  MathSciNet  Google Scholar 

  19. Czaja, W., Li, W.: Rotationally invariant time–frequency scattering transforms. J. Fourier Anal. Appl. 26, 4 (2020)

    Article  MathSciNet  Google Scholar 

  20. Grohs, P., Perekrestenko, D., Elbrächter, D., Bölcskei, H.: Deep neural network approximation theory. (2019) arXiv preprint arXiv:1901.02220

  21. Li, Y.: Feature extraction in image processing and deep learning. Ph.D. thesis, Univeristy of Maryland, College Park (2018)

  22. Benedetto, J.J.: Harmonic Analysis and Applications, vol. 23. CRC Press, Boca Raton (1996)

    MATH  Google Scholar 

  23. Gabor, D.: Theory of communication. J. Inst. Electr. Eng. III 93, 429–457 (1946)

    Google Scholar 

  24. Gröchenig, K.: Foundations of Time-Frequency Analysis. Springer, Berlin (2013)

    MATH  Google Scholar 

  25. Duffin, R.J., Schaeffer, A.C.: A class of nonharmonic Fourier series. Trans. Am. Math. Soc. 72, 341–366 (1952)

    Article  MathSciNet  Google Scholar 

  26. Walnut, D.F.: Continuity properties of the Gabor frame operator. J. Math. Anal. Appl. 165, 479–504 (1992)

    Article  MathSciNet  Google Scholar 

  27. Feichtinger, H.G., Gröchenig, K., Walnut, D.: Wilson bases and modulation spaces. Math. Nachrich. 155, 7–17 (1992)

    Article  MathSciNet  Google Scholar 

  28. Casazza, P.G., Christensen, O., Janssen, A.: Weyl–Heisenberg frames, translation invariant systems and the walnut representation. J. Funct. Anal. 180, 85–147 (2001)

    Article  MathSciNet  Google Scholar 

  29. Christensen, O., Kim, H.O., Kim, R.Y.: Regularity of dual Gabor windows. Abstract and Applied Analysis, vol. 2013, Hindawi (2013)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Maryland, College Park, MD, USA

    Wojciech Czaja & Yiran Li

Authors
  1. Wojciech Czaja
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiran Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wojciech Czaja.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Czaja, W., Li, Y. Gabor Neural Networks with Proven Approximation Properties. J Geom Anal 31, 8999–9015 (2021). https://doi.org/10.1007/s12220-020-00575-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12220-020-00575-z

Keywords

Search

Navigation