MetaRisk: Semi-supervised few-shot operational risk classification in banking industry

Abstract

We study the operational risk classification problem, a critical yet challenging problem in the banking industry. In practice, banks build supervised multi-label classification models to identify the pre-defined risks using financial news sources. However, the models are often suboptimal due to the lack of labeled data and diverse combinations of risk types. To address these practical issues, we re-frame multi-label supervised operational risk classification as a semi-supervised few-shot learning problem, named MetaRisk, which can then be effectively learned using the prototypical network. We also propose a weighted scheme to help obtain accurately prototype vectors of multi-risk classes. We evaluate the proposed approach MetaRisk using a real-world operational risk classification dataset, and the results demonstrate that it outperforms a set of standard baselines. Especially, MetaRisk is capable of predicting risk types that are new to the system. We expect our work provides a direct and relevant toolkit that may assist risk officers to predict and intervene risks in the banking industry.

Introduction

In the decade since the global financial crisis, banks and regulators have become increasingly alert to operational risks. However, the banks and regulators still struggle to deal with operational risk effectively [20], [15]. It is reported that major banks global wide have suffered nearly $210 billion in operational risk losses since 2011.¹ Operational risk, according to Basel Accord II,² is defined as the risks of loss due to errors, breaches, interruptions or damages caused by people, internal processes, systems or external events. In the banking industry, one of the daily jobs of risk officers is to screen potential risks with a large number of online news outlets and to assess any news events that may expose risks to the bank’s operations. Therefore, it is of keen interest of financial organizations to develop effective machine learning methods for operational risk classification.

While this task can be easily formulated as a standard document classification problem, there are at least two challenges in designing an effective operational risk prediction system. First, labeling financial news into different risk types requires substantial domain knowledge, and thus it is impossible to build a dataset using crowd-labeling services. Moreover, in practice, different banks are exposed to and are potentially vulnerable to different types of operational risks so no public dataset is available. As a result, there are only a few labeled news articles that are manually labeled by risk officers, while a large number of news articles remain unlabeled. Second, a small portion of financial news is related to multiple risk types which makes the problem essentially a multi-label classification task. For example, Internal Fraud (such as Bribery and Corruption) and Clients and Market Practice (such as Money Laundering) are two types of operational risks. Actually, bribery and corruption are intrinsically linked with money laundering. Not only do they tend to occur together, but also the presence of one tends to reinforce the other. Therefore, a news article that has an Internal Fraud label is likely to be labeled with Clients and Market Practice labels. However, those multi-label instances consist of a small portion of the entire dataset and thus some may not even appear in the training set. As a result, a standard multi-label discriminative classification model often performs suboptimally. It is desirable that the classifier can generalize well to those rare multi-label instances and alert risk officers such “black swan” events.

To tackle the aforementioned practice problems, we re-frame multi-label supervised operational risk classification as a semi-supervised few-shot learning problem. We do so for two reasons. First, semi-supervised learning [6] leverages unlabeled data to learn better data distribution which helps the discriminative model. Second, few-shot learning [36] is expected to learn generalizable classifier and thus may accommodate new multi-label classes that are not frequently seen in the training set. These two learning paradigms are largely independently studied in prior research, with most work addressing one or the other. Recently, a few studies [35], [28] propose semi-supervised few-shot learning framework for multi-class image recognition, while some researchers focus on few-shot text classifications [37], [14]. However, these methods are not applicable for our operational risk context as we face a multi-label classification task where instances are usually associated with more than one label [50].

In this work, we propose MetaRisk, a novel multi-label semi-supervised few-shot learning model for operational risk classification. Our method is built on the prototypical network [36] but improves the prototypes of risk combinations (multi-risk classes) by adjusting the weight of each risk type for each instance. Specifically, MetaRisk first utilizes a weighted scheme to learn the prior knowledge for risk class combinations from the relevant individual risk class. It then builds and refines a prototypical network to learn the single label and multi-label prototypes. We adopt attention mechanism [4] from neural network training to calculate the weighted prototype vector for multi-label risk type combinations. Furthermore, a soft-masking mechanism is introduced to refine the prototypes using unlabeled data, which allows our model to obtain decision boundaries for better fitting underlying risk distribution. We empirically evaluate MetaRisk on a proprietary dataset collected by an international banking organization. Experiment results show that MetaRisk outperforms a set of standard baselines. In particular, it is more effective than baselines on recognizing new risk type combinations task with a small number of known labeled and a large number of unlabeled instances.

Our main contributions can be summarized as two-folds.

• First, to the best of our knowledge, we are the first to study the operational risk classification problem using semi-supervised meta-learning method. We identify two practical challenges associated with operational risk classification and frame the problem using semi-supervised few-shot learning framework with a weighted scheme. We further modify the framework so that it can be generalized to minority multi-label risk classes.

• Second, we evaluate the framework on a real-world dataset and demonstrate its effectiveness. The system prototype has been used internally by the bank’s risk management team. We hope this work provides key insight into designing the practical semi-supervised meta-learning model for important financial applications.

The rest of the paper is arranged as follows. We first review related literature and position our work in that context in Section 2. The formal problem definition, as well as the necessary background with respect to operational risk classification and meta-learning techniques, are introduced in Section 3. The details of our MetaRisk model are presented in Section 4. Comprehensive experimental results demonstrating the superiority of our model are presented in Section 5. We conclude this work and point out the future directions in Section 6.