当前位置: X-MOL 学术Neurocomputing › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Neural networks based on vectorized neurons
Neurocomputing ( IF 5.5 ) Pub Date : 2021-09-08 , DOI: 10.1016/j.neucom.2021.09.006
Ji He 1 , Hongwei Yang 2 , Lei He 1 , Lina Zhao 1, 3
Affiliation  

As the main research content of artificial intelligence, the artificial neural network has been widely concerned because of its excellent performance in the fields such as computer vision and natural language processing since it was proposed in the 1940s. The neuron model of the traditional neural network was proposed by McCulloch and Pitts in 1943 (MP neurons), But MP neurons is too simple to representing biological neurons. Based on this, this paper studies the attention mechanism and proposes vectorized neuron and its activation function. Firstly, we propose vectorized neurons, then use the attention mechanism to dynamically generate connection weights between vectorized neurons. Nextly, we construct a new type of neural network with vectorized neurons, which we called neural functional group (NFG). Finally, we tested the proposed neural functional group model on two tasks: image classifcation and few-shot learning. The vectorized neuron can be conditionally activated through its activation function. Besides, the vectorized neuron has the potential of representing complex biological neurons, which is difficult for MP neuron. The experimental results show that it can achieve higher accuracy with fewer parameters than convolutional neural networks (CNN) and capsule networks in image classication task; it also competitive to CNN based feature extractor in few-shot learning task.



中文翻译:

基于向量化神经元的神经网络

人工神经网络作为人工智能的主要研究内容,自1940年代提出以来,因其在计算机视觉、自然语言处理等领域的优异表现而受到广泛关注。传统神经网络的神经元模型是由 McCulloch 和 Pitts 在 1943 年提出的(MP 神经元),但 MP 神经元过于简单,无法代表生物神经元。基于此,本文研究了注意力机制,提出了向量化神经元及其激活函数。首先,我们提出向量化神经元,然后使用注意力机制动态生成向量化神经元之间的连接权重。接下来,我们构建了一种带有向量化神经元的新型神经网络,我们称之为神经功能组(NFG)。最后,我们在两个任务上测试了所提出的神经功能组模型:图像分类和小样本学习。矢量化神经元可以通过其激活函数有条件地激活。此外,向量化神经元具有表示复杂生物神经元的潜力,这对于 MP 神经元来说是困难的。实验结果表明,在图像分类任务中,与卷积神经网络(CNN)和胶囊网络相比,它可以以更少的参数获得更高的准确率;它还可以在小样本学习任务中与基于 CNN 的特征提取器竞争。这对 MP 神经元来说是困难的。实验结果表明,在图像分类任务中,与卷积神经网络(CNN)和胶囊网络相比,它可以以更少的参数获得更高的准确率;它还可以在小样本学习任务中与基于 CNN 的特征提取器竞争。这对 MP 神经元来说是困难的。实验结果表明,在图像分类任务中,与卷积神经网络(CNN)和胶囊网络相比,它可以以更少的参数获得更高的准确率;它还可以在小样本学习任务中与基于 CNN 的特征提取器竞争。

更新日期:2021-09-15
down
wechat
bug