Skip to main content
SpringerLink
Log in

Research on learning mechanism designing for equilibrated bipolar spiking neural networks

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Artificial Intelligence (AI) has become very popular due to both the increasing demands from applications and the booming of computer techniques. Spiking Neural Network (SNN), as the third generation of Artificial Neural Network, receives more and more attention in the field of AI. With the high similarity to biological neural network, SNN has the potential to break through the barriers of strong AI. However, the using of SNNs on practical scenarios is rather limited, as a result of the lack of high efficient learning algorithms. Nowadays, learning methods of SNNs are designed mainly based on previous biological discoveries. The fact that there are both excitatory neurons and inhibitory neurons in the biological neural network has stimulated the motive of this research. The existence of inhibitory neurons could strengthen the self-regulation ability of neural networks and improve learning efficiency. Inspired by the ancient Chinese “Yin and Yang” Theory, we first presented our effort at constructing SNN structure with equilibrated excitatory neurons and inhibitory neurons. Then an ensemble learning optimized supervised learning method is designed and tailored for this SNN structure. Experiments are conducted using MNIST data sets, and results show that, with the designed learning mechanism, our equilibrated bipolar SNN structure could gain reasonable accuracy with much more compact structure and much more sparse synapse connections.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Beyeler M, Dutt ND, Krichmar JL (2013) Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule. Neural Netw 48:109–124

    Article  Google Scholar 

  • Bohte SM, Kok JN, Han ALP (2000) Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1):17–37

    MATH  Google Scholar 

  • Cho S, Beigné E, Zhang Z (2019) A 2048-neuron spiking neural network accelerator with neuro-inspired pruning and asynchronous network on chip in 40nm CMOS, In: IEEE custom integrated circuits conference (CICC). Austin, TX, USA, pp 1–4

  • Diehl PU, Matthew C (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci. https://doi.org/10.3389/fncom.2015.00099

    Article  Google Scholar 

  • Florian RV, Michal Z (2012) The chronotron: a neuron that learns to fire temporally precise spike patterns. PLoS ONE 7(8):e40233

    Article  Google Scholar 

  • Gewaltig MO, Diesmann M (2007) Nest (neural simulation tool). Scholarpedia 2(4):1430

    Article  Google Scholar 

  • Ghosh-Dastidar S, Adeli H (2009) A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Netw 22(10):1419–1431

    Article  Google Scholar 

  • Guo L, Wang Z, Adjouadi M (2015) A novel biologically plausible supervised learning method for spiking neurons. In: World congress in computer science, computer engineering, and applied computing, int’l conf. artificial intelligence

  • Hazan H, Saunders D, Sanghavi DT, Siegelmann H, Kozma R (2018) Unsupervised learning with self-organizing spiking neural networks. In: 2018 international joint conference on neural networks (IJCNN), Rio de Janeiro, pp 1–6. https://doi.org/10.1109/IJCNN.2018.8489673

  • Hebb DO (1949) The organization of behavior. Wiley, New York

    Google Scholar 

  • Indiveri G (2003) A low-power adaptive integrate-and-fire neuron circuit. In: International symposium on circuits and systems. IEEE

  • Iyer LR, Basu A (2017) Unsupervised learning of event-based image recordings using spike-timing-dependent plasticity. In: International joint conference on neural networks. IEEE, pp 1840–1846

  • Kasabov N, Dhoble K, Nuntalid N, Indiveri G (2013) Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. Neural Netw 41(5):188–201

    Article  Google Scholar 

  • Lecun Y, Cortes C (2010) The MNIST database of handwritten digits. Available online at: http://yann.lecun.com/exdb/mnist/

  • Legenstein R, Naeger C, Maass W (2005) What can a neuron learn with spike-timing-dependent plasticity? Neural Comput 17(11):2337

    Article  MathSciNet  Google Scholar 

  • Liu D, Yue S (2016) Visual pattern recognition using unsupervised spike timing dependent plasticity learning. In: International joint conference on neural networks. IEEE, pp 285–292

  • Maas W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671

    Article  Google Scholar 

  • Matsubara T (2017) Spike timing-dependent conduction delay learning model classifying spatio-temporal spike patterns. In: International joint conference on neural networks, pp 1831–1839

  • Mckennoch S, Liu D, Bushnell LG (2006) Fast modifications of the spikeprop algorithm. In: IEEE international joint conference on neural network. IEEE

  • Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In: Computer vision and pattern recognition, pp 427–436

  • Ponulak F (2005) Resume—new supervised learning method for spiking neural networks

  • Ponulak F, Kasiski A (2010) Supervised learning in spiking neural networks with resume: sequence learning, classification, and spike shifting. Neural Comput 22(2):467–510

    Article  MathSciNet  Google Scholar 

  • Qiao GC et al (2019) A neuromorphic-hardware oriented bio-plausible online-learning spiking neural network model. IEEE Access 7:71730–71740. https://doi.org/10.1109/ACCESS.2019.2919163

    Article  Google Scholar 

  • Ruf B, Schmitt M (1997) Learning temporally encoded patterns in networks of spiking neurons. Neural Process Lett 5(1):9–18

    Article  Google Scholar 

  • Shrestha A, Ahmed K, Wang Y et al (2017) Stable spike-timing dependent plasticity rule for multilayer unsupervised and supervised learning. In: International joint conference on neural networks. IEEE, pp 1999–2006

  • Song S, Miller KD, Abbott LF (2000) Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neurosci 3(9):919–26

    Article  Google Scholar 

  • Sporea I, Grüning A (2012) Supervised learning in multilayer spiking neural networks. Neural Comput 25(2):473–509

    Article  MathSciNet  Google Scholar 

  • Srinivasan G, Roy S, Raghunathan V, Roy K (2017) Spike timing dependent plasticity based enhanced self-learning for efficient pattern recognition in spiking neural networks. In: International joint conference on neural networks, pp 1847–1854

  • Taherkhani A, Belatreche A, Li Y et al (2014) A new biologically plausible supervised learning method for spiking neurons. In: ESANN 2014 proceedings, European symposium on artificial neural networks, computational intelligence and machine learning, Bruges (Belgium), 23–25 April 2014

  • Tang T, Xia L, Li B, Luo R (2015) Spiking neural network with RRAM: can we use it for real-world application? In: Design, automation and test in Eeurope conference and exhibition, pp 860–865

  • Thorpe S, Gautrais J (1998) Rank order coding. In: Conference on computational neuroscience: trends in research. Plenum Press, New York

  • Wade JJ, McDaid LJ, Sayers HM, Wade JJ, Mcdaid LJ, Santos JA, Santos JA et al (2010) SWAT: a spiking neural network training algorithm for classification problems. IEEE Trans Neural Netw 21(11):1817

    Article  Google Scholar 

  • Wang Z et al (2018) Experimental demonstration of ferroelectric spiking neurons for unsupervised clustering. In: 2018 IEEE international electron devices meeting (IEDM), San Francisco, CA, pp 13.3.1–13.3.4. https://doi.org/10.1109/IEDM.2018.8614586

  • Xin J, Embrechts MJ (2001) Supervised learning with spiking neural networks. In: International joint conference on neural networks. IEEE

  • Xu Y, Zeng X, Han L et al (2013) A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks. Neural Netw 43:99–113

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant No. 91846303.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China

    Xu Yang, Jiajun Lin, Wenhao Zheng, Jinfeng Zhao, Mengyao Ji, Yunlin Lei & Zenghao Chai

Authors
  1. Xu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiajun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenhao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinfeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mengyao Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yunlin Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zenghao Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xu Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Lin, J., Zheng, W. et al. Research on learning mechanism designing for equilibrated bipolar spiking neural networks. Artif Intell Rev 53, 5189–5215 (2020). https://doi.org/10.1007/s10462-020-09818-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-020-09818-5

Keywords

Search

Navigation