Reverse graph self-attention for target-directed atomic importance estimation

Abstract

Estimating the importance of each atom in a molecule is one of the most appealing and challenging problems in chemistry, physics, and materials science. The most common way to estimate the atomic importance is to compute the electronic structure using density functional theory (DFT), and then to interpret it using domain knowledge of human experts. However, this conventional approach is impractical to the large molecular database because DFT calculation requires large computation, specifically, $O\left(n^4\right)$ time complexity w.r.t. the number of electronic basis functions. Furthermore, the calculation results should be manually interpreted by human experts to estimate the atomic importance in terms of the target molecular property. To tackle this problem, we first exploit the machine learning-based approach for the atomic importance estimation based on the reverse self-attention on graph neural networks and integrating it with graph-based molecular description. Our method provides an efficiently-automated and target-directed way to estimate the atomic importance without any domain knowledge of chemistry and physics.

Introduction

In molecules, each atom has its own contributions in manifesting the entire molecular properties, and estimating such atomic importance plays a key role in interpreting molecular systems. For these reasons, the atomic importance estimation has been consistently studied in the scientific communities (Pan et al., 2018, Tang et al., 2016, Yen and Winefordner, 1976). However, estimating the atomic importance is one of the most challenging tasks in chemistry and physics because the importance of each atom is comprehensively determined based on atomic properties, neighbor atoms, bonding types, target molecular property, and whole structure of the molecule.

The most common approach for estimating the atomic importance in physics and chemistry is to interpret the electronic structure using density functional theory (DFT) (Sholl & Steckel, 2009). In this approach, the atomic importance is estimated through three steps: (1) A human expert selects appropriate functional and basis sets for a given molecule to apply DFT; (2) The electronic structure of the molecule is calculated based on DFT calculation; (3) The human expert estimates the atomic importance by interpreting the calculated electronic structure in terms of target molecular property. Although some methods are proposed to estimate relative contributions of atoms in molecules, their generality is typically limited to certain molecular properties (Glendening et al., 2019, Marenich et al., 2012). For this reason, DFT has been most widely used to interpret molecular systems and to reveal the important atoms according to target molecular property because it can generate a universal description of the molecular systems (Chibani et al., 2018, Crimme et al., 2010, Geerlings et al., 2003, Lee et al., 2018b).

However, the conventional approach based on DFT has three fundamental limitations in efficiency, automation, and generality.

• Efficiency: As an example of the electronic structure computations, DFT calculation requires $O\left(n^4\right)$ time complexity to compute the electronic structure, where $n$ is the number of basis functions that describe electrons in the molecule (Jensen, 2017). In general, molecules have more electrons than atoms, and thus DFT calculation requires large computation.

• Automation: Although DFT provides details of the electronic structure that describes many properties of molecules, it does not explain all physical and chemical properties of the molecules without proper analyses (Sholl & Steckel, 2009). Thus, additional calculations that require the domain knowledge of human experts should be applied to explain some molecular properties.

• Generality: For some molecular properties, the relationship between them and the electronic distributions is not clear. Moreover, sometimes the atomic importance estimation is impossible because the relationships between molecular properties and molecular structures are not interpretable.

Due to these limitations, estimating the atomic importance remains an open problem in physics, chemistry, pharmacy, and materials science.

To overcome the limitations of the conventional approach in estimating the atomic importance, we exploit the machine learning-based approach by proposing a new concept of reverse graph self-attention (RGSA) and integrating it with the graph neural networks for the first time. The self-attention mechanism was originally designed to determine important elements within the input data to accurately predict its corresponding target or label in natural language processing (Vaswani et al., 2017). Similarly, in graph neural networks, the self-attention is used to determine important neighbor nodes within the input graph to generate a more accurate node or graph embeddings (Velickovic et al., 2018). The proposed RGSA is defined as the inverse of the self-attention to calculate how important a selected node is considered by its neighbor nodes in the graph. For a given molecule and target property, the proposed estimation method selects the atom that has the largest RGSA score as the most important atom in terms of the target property.

The proposed method estimates the target-directed atomic importance through two steps: (1) For the given molecular graphs and their corresponding target properties, a self-attention based graph neural network is trained to predict the target properties. (2) After the training, RGSA scores are calculated, and then the atomic importance is estimated based on the calculated RGSA scores. As shown in this estimation process, neither large computation nor human experts in physics and chemistry are required, and the estimation process is fully automated.

To validate the effectiveness of the proposed method for atomic importance estimation, we conducted comprehensive experiments and evaluated the estimation performance using both quantitative and qualitative analyses. The contributions of this paper are summarized as:

• This paper first proposes a machine learning-based approach to estimate the atomic importance in the molecule.

• The proposed method drastically reduced the computational cost for the atomic importance estimation from quartic time complexity to the practical time complexity of training graph neural networks.

• The proposed method provides a fully-automated and target-directed way to estimate the atomic importance.

• We comprehensively validated the effectiveness of the proposed method using both quantitative and qualitative evaluations with domain knowledge and scientific literature in physics and chemistry. However, since there is no labeled dataset for the atomic importance estimation and a systematic way to quantitatively evaluate the estimation accuracy, we devised a new quantitative evaluation method and validated the effectiveness of the proposed method using it.