Elsevier

Pattern Recognition Letters

Volume 138, October 2020, Pages 424-430
Pattern Recognition Letters

Weighted-capsule routing via a fuzzy gaussian model

https://doi.org/10.1016/j.patrec.2020.08.009Get rights and content

Highlights

  • A novel weighted capsule fuzzy gaussian model (WCFGM) routing algorithm is proposed to improve the original EM routing.

  • WCFGM routing prohibits atypical child capsules to contaminate the parent capsules.

  • A pose loss is introduced to provide the best inter-class separation.

  • The ability of pattern classification is improved in terms of accuracy.

Abstract

Capsule network (CapsNet) is a novel architecture that takes into account the hierarchical pose relationships between object parts, which had achieved desirable results on image classification. EM-Routing (EM-R) used in CapsNet is the process of assigning child capsules (parts) to each parent capsule (objects) based on a level of agreement, which is similar to the fuzzy clustering process. However, CapsNet still struggles with backgrounds and the presence of noise. In this paper, a new routing algorithm based on a weighted capsule fuzzy gaussian model (WCFGM-R) and a pose loss function are proposed. The proposed algorithm aims to prohibit atypical child capsules from contaminating the parent capsules by incorporating the activations of capsules in a lower layer as weights that play the role of precision. The pose loss provides the best inter-class separation and improves the ability of pattern classification. Indeed, the experimental analyses demonstrate that CapsNet with WCFGM-R outperforms the CapsNet with EM-R in which it shows excellent results on three datasets (MNIST-bg-img, MNIST-bg-rnd, and CIFAR10).

Introduction

A convolutional neural network (CNN) [1] is commonly used in analyzing visual imagery and has achieved state of the art on the image classification task. Although CNNs can efficiently detect useful features, they can not explore the spatial relationships between features. The lack of relevant information and the transition invariance caused by the pooling operation affect the classification results.

Capsule network (CapsNet) is proposed by Sabour et al. [2], and extended by Hinton et al. [3] to overcome CNN’s impediments. The main idea is to learn the part-whole relationships (e.g., perspective, size, scale, orientation) between the observed objects or object parts to achieve the translation equivariance. Due to the equivariance property, capsules can recognize an object with specific variations (rotated, scaled, etc.) without the need to train a model with these variations.

The routing by agreement process is an interaction that occurs between each two adjacent capsule layers, which is an information selection mechanism. This mechanism attempts to ensure that the outputs of lower layer capsules are sent to the appropriate higher layer capsules. In [3], the Expectation Maximization algorithm has been proposed as a routing technique (EM-R), in which it assigns the output of each capsule in the lower layer to the appropriate capsule in the layer above. Despite the outperformance of CapsNet with EM-R, it still struggles from the presence of noise and backgrounds, and it may misclassify many examples from different classes into the same class. Firstly, the responsibilities used in EM-R indicate which capsule’s pose is the closest to the child capsule but not any of them is close at all. Typically, an outlying child capsule which is far from any parent capsule’s pose will have a small responsibility while it may still be assigned to the closest parent capsule. This issue may influence the classification results, especially in case of data that contains backgrounds and noise. Secondly, if the inputs have some similarities in the descriptive characteristics, CapsNet may extract similar poses of different classes, which can confuse the classifier.

To solve the issues mentioned above, a routing algorithm based on a weighted capsule fuzzy Gaussian model (WCFGM-R) and a pose loss function are proposed in this paper. We sum up the main contributions of this paper as follows.

  • (a)

    The activations of the child capsules are used as weights that show the importance of them. These weights play the role of precisions. Hence, each child capsule with a very small activation will be treated as noise and will have a little influence on the estimation of the means (poses) and variances of the parent capsules.

  • (b)

    To provide better inter-class separation characteristics, we propose a pose loss function which can be combined with the original loss function to achieve higher classification accuracies.

  • (c)

    Several experiments are provided to test the performance of CapsNet with WCFGM-R. The results show that the CapsNet with WCFGM-R outperforms the CapsNet with EM-R in terms of accuracy.

The rest of the paper is organized as follows. Section 2 introduces related work about the capsule network and its application in different fields. Section 3 introduces the proposed routing algorithm. Section 4 presents the proposed pose loss. Section 5 introduces a comparison study between capsule network with the proposed routing and that one with EM-R. Section 6 concludes this paper.

Section snippets

Related work

Hinton et al. [4] proposed a new concept called capsules that provides a simple way to recognize the wholes by recognizing their parts while simultaneously preserving the spatial information. The capsule’s output represents the different properties of the same entity. Sabour et al. [2] proposed the so-called dynamic routing used between every two adjacent layers for capsules assignment as the first implementation of the CapsNet.

Hinton et al. [3] extended a new version of capsules, where the

Proposed routing (WCFGM)

Fuzzy c-means (FCM) is a well known non-parametric clustering method [15]. It has been extensively applied in a variety of substantive areas [16], [17], [18], [19]. Since there are a lot of similar characteristics between FCM and the Gaussian mixture models (GMMs), many authors have strived to combine the FCM algorithm with GMMs [20], [21], [22].

In this section, a new clustering or routing algorithm is proposed for capsule networks. In what follows, capsule(s) in the lower layer is named

The pose loss

Developing an effective loss function is an interesting way to improve the ability of pattern classification. Intuitively, providing the best inter-class separation characteristics is the key. For this purpose, a pose loss is proposed as follows:Lp=itμtμi2,where μt is the pose of the target class, and μi is the pose of the ith class. The formulation effectively characterizes the inter-class variations. Therefore, the loss function used in this paper is given by the following:Loss=GLξLp,

Experimental results

In this section, to compare the performance of CapsNet with WCFGM-R and that one with EM-R, the following experiments have been conducted.

Conclusion

In this work, a novel weighted capsule fuzzy gaussian model routing (WCFGM-R) and a pose loss function are introduced. The proposed WCFGM-R is employed to prohibit the effect of noise on the classification task. A pose loss function provides the best inter-class separation which improves the ability of pattern classification. Experimental analyses show that the proposed CapsNet with WCFGM-R improves the accuracy in the presence of noisy backgrounds without sacrificing the reconstruction ability

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors would like to thank the Editor and the reviewers for their useful suggestions which have helped to improve the paper substantially. This work is supported by the National Natural Science Foundation of China under Grant Nos. 61976174, 11671317.

References (26)

  • C. Xiang

    MS-CapsNet: a novel multi-scale capsule network

    IEEE Signal Processing Letters

    (2018)
  • F.D.S. Ribeiro, G. Leontidis, S. Kollias, Capsule routing via variational bayes, 2019....
  • P. Afshar et al.

    Brain tumor type classification via capsule networks

    25th IEEE International Conference on Image Processing (ICIP)

    (2018)
  • View full text