We investigate a latent variable model for multinomial classification inspired by recent capsule architectures for visual object recognition (Sabour, Frosst, & Hinton, 2017). Capsule architectures use vectors of hidden unit activities to encode the pose of visual objects in an image, and they use the lengths of these vectors to encode the probabilities that objects are present. Probabilities from different capsules can also be propagated through deep multilayer networks to model the part-whole relationships of more complex objects. Notwithstanding the promise of these networks, there still remains much to understand about capsules as primitive computing elements in their own right. In this letter, we study the problem of capsule regression—a higher-dimensional analog of logistic, probit, and softmax regression in which class probabilities are derived from vectors of competing magnitude. To start, we propose a simple capsule architecture for multinomial classification: the architecture has one capsule per class, and each capsule uses a weight matrix to compute the vector of hidden unit activities for patterns it seeks to recognize. Next, we show how to model these hidden unit activities as latent variables, and we use a squashing nonlinearity to convert their magnitudes as vectors into normalized probabilities for multinomial classification. When different capsules compete to recognize the same pattern, the squashing nonlinearity induces nongaussian terms in the posterior distribution over their latent variables. Nevertheless, we show that exact inference remains tractable and use an expectation-maximization procedure to derive least-squares updates for each capsule's weight matrix. We also present experimental results to demonstrate how these ideas work in practice.

中文翻译：

---

一种胶囊回归的 EM 算法

我们研究了一个用于多项分类的潜在变量模型，其灵感来自最近用于视觉对象识别的胶囊架构（Sabour、Frosst 和 Hinton，2017 年）。胶囊架构使用隐藏单元活动的向量来编码图像中视觉对象的姿态，并且它们使用这些向量的长度来编码对象存在的概率。来自不同胶囊的概率也可以通过深层多层网络传播，以模拟更复杂对象的部分-整体关系。尽管这些网络有前景，但关于胶囊本身作为原始计算元素的理解仍有很多。在这封信中，我们研究了胶囊回归的问题——logistic、probit、和 softmax 回归，其中类概率来自竞争幅度的向量。首先，我们提出了一种用于多项式分类的简单胶囊架构：该架构每个类别有一个胶囊，每个胶囊使用权重矩阵来计算其试图识别的模式的隐藏单元活动的向量。接下来，我们将展示如何将这些隐藏单元活动建模为潜在变量，并使用压缩非线性将它们的大小作为向量转换为用于多项式分类的归一化概率。当不同的胶囊竞争识别相同的模式时，挤压非线性会在其潜在变量的后验分布中引入非高斯项。尽管如此，我们表明精确推断仍然易于处理，并使用期望最大化程序来推导出每个胶囊的权重矩阵的最小二乘更新。我们还提供了实验结果来展示这些想法在实践中是如何运作的。