Deep learning has emerged as a revolutionary technology for protein residue‐residue contact prediction since the 2012 CASP10 competition. Considerable advancements in the predictive power of the deep learning‐based contact predictions have been achieved since then. However, little effort has been put into interpreting the black‐box deep learning methods. Algorithms that can interpret the relationship between predicted contact maps and the internal mechanism of the deep learning architectures are needed to explore the essential components of contact inference and improve their explainability. In this study, we present an attention‐based convolutional neural network for protein contact prediction, which consists of two attention mechanism‐based modules: sequence attention and regional attention. Our benchmark results on the CASP13 free‐modeling targets demonstrate that the two attention modules added on top of existing typical deep learning models exhibit a complementary effect that contributes to prediction improvements. More importantly, the inclusion of the attention mechanism provides interpretable patterns that contain useful insights into the key fold‐determining residues in proteins. We expect the attention‐based model can provide a reliable and practically interpretable technique that helps break the current bottlenecks in explaining deep neural networks for contact prediction. The source code of our method is available at https://github.com/jianlin-cheng/InterpretContactMap.

中文翻译：

---

深度神经网络与注意力机制的结合增强了蛋白质接触预测的可解释性

自 2012 年 CASP10 竞赛以来，深度学习已成为蛋白质残基-残基接触预测的革命性技术。从那时起，基于深度学习的接触预测的预测能力取得了相当大的进步。然而，在解释黑盒深度学习方法方面付出的努力很少。需要能够解释预测接触图与深度学习架构内部机制之间关系的算法来探索接触推理的基本组成部分并提高其可解释性。在这项研究中，我们提出了一种用于蛋白质接触预测的基于注意力的卷积神经网络，它由两个基于注意力机制的模块组成：序列注意力和区域注意力。我们在 CASP13 自由建模目标上的基准测试结果表明，在现有典型深度学习模型之上添加的两个注意力模块表现出有助于预测改进的互补效应。更重要的是，注意力机制的加入提供了可解释的模式，其中包含对蛋白质中关键折叠决定残基的有用见解。我们期望基于注意力的模型可以提供一种可靠且实际可解释的技术，有助于打破当前在解释用于接触预测的深度神经网络方面的瓶颈。我们方法的源代码可在 https://github.com/jianlin-cheng/InterpretContactMap 获得。