Standard deep learning models that employ the categorical cross-entropy loss are known to perform well at image classification tasks. However, many standard models thus obtained often exhibit issues like feature redundancy, low interpretability, and poor calibration. A body of recent work has emerged that has tried addressing some of these challenges by proposing the use of new regularization functions in addition to the cross-entropy loss. In this paper, we present some surprising findings that emerge from exploring the role of simple orthogonality constraints as a means of imposing physics-motivated constraints common in imaging. We propose an Orthogonal Sphere (OS) regularizer that emerges from physics-based latent-representations under simplifying assumptions. Under further simplifying assumptions, the OS constraint can be written in closed-form as a simple orthonormality term and be used along with the cross-entropy loss function. The findings indicate that orthonormality loss function results in a) rich and diverse feature representations, b) robustness to feature sub-selection, c) better semantic localization in the class activation maps, and d) reduction in model calibration error. We demonstrate the effectiveness of the proposed OS regularization by providing quantitative and qualitative results on four benchmark datasets - CIFAR10, CIFAR100, SVHN and tiny ImageNet.

中文翻译：

---

正交性约束在改善图像分类深度网络特性中的作用

众所周知，采用分类交叉熵损失的标准深度学习模型在图像分类任务中表现良好。然而，由此获得的许多标准模型往往存在特征冗余、可解释性低和校准不良等问题。最近出现的一系列工作已经尝试通过提议使用除交叉熵损失之外的新正则化函数来解决其中的一些挑战。在本文中，我们提出了一些令人惊讶的发现，这些发现来自于探索简单正交性约束作为施加成像中常见的物理驱动约束的一种手段的作用。我们提出了一个正交球 (OS) 正则化器，它在简化假设下从基于物理的潜在表示中出现。在进一步简化的假设下，OS 约束可以用封闭形式写成一个简单的正交项，并与交叉熵损失函数一起使用。研究结果表明，正交性损失函数导致 a) 丰富多样的特征表示，b) 对特征子选择的鲁棒性，c) 类激活图中更好的语义定位，以及 d) 减少模型校准误差。我们通过在四个基准数据集（CIFAR10、CIFAR100、SVHN 和 tiny ImageNet）上提供定量和定性结果来证明所提出的 OS 正则化的有效性。c) 类激活图中更好的语义定位，以及 d) 减少模型校准误差。我们通过在四个基准数据集（CIFAR10、CIFAR100、SVHN 和 tiny ImageNet）上提供定量和定性结果来证明所提出的 OS 正则化的有效性。c) 类激活图中更好的语义定位，以及 d) 减少模型校准误差。我们通过在四个基准数据集（CIFAR10、CIFAR100、SVHN 和 tiny ImageNet）上提供定量和定性结果来证明所提出的 OS 正则化的有效性。