Weighted-capsule routing via a fuzzy gaussian model
Introduction
A convolutional neural network (CNN) [1] is commonly used in analyzing visual imagery and has achieved state of the art on the image classification task. Although CNNs can efficiently detect useful features, they can not explore the spatial relationships between features. The lack of relevant information and the transition invariance caused by the pooling operation affect the classification results.
Capsule network (CapsNet) is proposed by Sabour et al. [2], and extended by Hinton et al. [3] to overcome CNN’s impediments. The main idea is to learn the part-whole relationships (e.g., perspective, size, scale, orientation) between the observed objects or object parts to achieve the translation equivariance. Due to the equivariance property, capsules can recognize an object with specific variations (rotated, scaled, etc.) without the need to train a model with these variations.
The routing by agreement process is an interaction that occurs between each two adjacent capsule layers, which is an information selection mechanism. This mechanism attempts to ensure that the outputs of lower layer capsules are sent to the appropriate higher layer capsules. In [3], the Expectation Maximization algorithm has been proposed as a routing technique (EM-R), in which it assigns the output of each capsule in the lower layer to the appropriate capsule in the layer above. Despite the outperformance of CapsNet with EM-R, it still struggles from the presence of noise and backgrounds, and it may misclassify many examples from different classes into the same class. Firstly, the responsibilities used in EM-R indicate which capsule’s pose is the closest to the child capsule but not any of them is close at all. Typically, an outlying child capsule which is far from any parent capsule’s pose will have a small responsibility while it may still be assigned to the closest parent capsule. This issue may influence the classification results, especially in case of data that contains backgrounds and noise. Secondly, if the inputs have some similarities in the descriptive characteristics, CapsNet may extract similar poses of different classes, which can confuse the classifier.
To solve the issues mentioned above, a routing algorithm based on a weighted capsule fuzzy Gaussian model (WCFGM-R) and a pose loss function are proposed in this paper. We sum up the main contributions of this paper as follows.
- (a)
The activations of the child capsules are used as weights that show the importance of them. These weights play the role of precisions. Hence, each child capsule with a very small activation will be treated as noise and will have a little influence on the estimation of the means (poses) and variances of the parent capsules.
- (b)
To provide better inter-class separation characteristics, we propose a pose loss function which can be combined with the original loss function to achieve higher classification accuracies.
- (c)
Several experiments are provided to test the performance of CapsNet with WCFGM-R. The results show that the CapsNet with WCFGM-R outperforms the CapsNet with EM-R in terms of accuracy.
The rest of the paper is organized as follows. Section 2 introduces related work about the capsule network and its application in different fields. Section 3 introduces the proposed routing algorithm. Section 4 presents the proposed pose loss. Section 5 introduces a comparison study between capsule network with the proposed routing and that one with EM-R. Section 6 concludes this paper.
Section snippets
Related work
Hinton et al. [4] proposed a new concept called capsules that provides a simple way to recognize the wholes by recognizing their parts while simultaneously preserving the spatial information. The capsule’s output represents the different properties of the same entity. Sabour et al. [2] proposed the so-called dynamic routing used between every two adjacent layers for capsules assignment as the first implementation of the CapsNet.
Hinton et al. [3] extended a new version of capsules, where the
Proposed routing (WCFGM)
Fuzzy c-means (FCM) is a well known non-parametric clustering method [15]. It has been extensively applied in a variety of substantive areas [16], [17], [18], [19]. Since there are a lot of similar characteristics between FCM and the Gaussian mixture models (GMMs), many authors have strived to combine the FCM algorithm with GMMs [20], [21], [22].
In this section, a new clustering or routing algorithm is proposed for capsule networks. In what follows, capsule(s) in the lower layer is named
The pose loss
Developing an effective loss function is an interesting way to improve the ability of pattern classification. Intuitively, providing the best inter-class separation characteristics is the key. For this purpose, a pose loss is proposed as follows:where μt is the pose of the target class, and μi is the pose of the ith class. The formulation effectively characterizes the inter-class variations. Therefore, the loss function used in this paper is given by the following:
Experimental results
In this section, to compare the performance of CapsNet with WCFGM-R and that one with EM-R, the following experiments have been conducted.
Conclusion
In this work, a novel weighted capsule fuzzy gaussian model routing (WCFGM-R) and a pose loss function are introduced. The proposed WCFGM-R is employed to prohibit the effect of noise on the classification task. A pose loss function provides the best inter-class separation which improves the ability of pattern classification. Experimental analyses show that the proposed CapsNet with WCFGM-R improves the accuracy in the presence of noisy backgrounds without sacrificing the reconstruction ability
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
The authors would like to thank the Editor and the reviewers for their useful suggestions which have helped to improve the paper substantially. This work is supported by the National Natural Science Foundation of China under Grant Nos. 61976174, 11671317.
References (26)
Capsule network assisted IoT traffic classification mechanism for smart cities
IEEE Internet of Things Journal
(2019)A modified interval type-2 fuzzy c-means algorithm with application in MR image segmentation
Pattern Recognit. Lett.
(2013)- et al.
Fuzzy gaussian mixture models
Pattern Recognit.
(2012) - et al.
Type-2 fuzzy gaussian mixture models
Pattern Recognit.
(2018) - et al.
EM Algorithms for weighted-data clustering with application to audio-visual scene analysis
IEEE Trans. Pattern Anal. Mach. Intell.
(2016) - et al.
ImageNet classification with deep convolutional neural networks
Proc. 25th Int. Conf. Neural Inf. Process. Syst
(2012) - et al.
Dynamic routing between capsules
Proc. 31st Int. Conf. Neural Inf. Process. Syst
(2017) - et al.
Matrix capsules with EM routing
Proc. Int. Conf. Learn. Representations
(2018) - et al.
Transforming autoencoders
Proc. Int. Conf. Artif. Neural Netw
(2011) - et al.
Fast dynamic routing based on weighted kernel density estimation
International Symposium on Artificial Intelligence and Robotics
(2018)
MS-CapsNet: a novel multi-scale capsule network
IEEE Signal Processing Letters
Brain tumor type classification via capsule networks
25th IEEE International Conference on Image Processing (ICIP)
Cited by (4)
A data corruption detection scheme based on ciphertexts in cloud environment
2021, KSII Transactions on Internet and Information SystemsA Framework for Classification of Gabor Based Frequency Selective Bone Radiographs Using CNN
2021, Arabian Journal for Science and EngineeringAnalysis of Sports Injury Estimation Model Based on Mutation Fuzzy Neural Network
2021, Computational Intelligence and NeuroscienceFuzzy c-means clustering with conditional probability based K–L information regularization
2021, Journal of Statistical Computation and Simulation