Classification of glomerular pathological findings using deep learning and nephrologist–AI collective intelligence approach

https://doi.org/10.1016/j.ijmedinf.2020.104231Get rights and content

Highlights

  • Artificial intelligence models classified glomerular images of renal biopsy.

  • Seven major pathological findings were automatically classified by deep learning.

  • Majority decision among experts and our models can improve diagnostic performance.

Abstract

Background

Automated classification of glomerular pathological findings is potentially beneficial in establishing an efficient and objective diagnosis in renal pathology. While previous studies have verified the artificial intelligence (AI) models for the classification of global sclerosis and glomerular cell proliferation, there are several other glomerular pathological findings required for diagnosis, and the comprehensive models for the classification of these major findings have not yet been reported. Whether the cooperation between these AI models and clinicians improves diagnostic performance also remains unknown. Here, we developed AI models to classify glomerular images for major findings required for pathological diagnosis and investigated whether those models could improve the diagnostic performance of nephrologists.

Methods

We used a dataset of 283 kidney biopsy cases comprising 15,888 glomerular images that were annotated by a total of 25 nephrologists. AI models to classify seven pathological findings: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, were constructed using deep learning by fine-tuning of InceptionV3 convolutional neural network. Subsequently, we compared the agreement to truth labels between majority decision among nephrologists with or without the AI model as a voter.

Results

Our model for global sclerosis showed high performance (area under the curve: periodic acid-Schiff, 0.986; periodic acid methenamine silver, 0.983); the models for the other findings also showed performance close to those of nephrologists. By adding the AI model output to majority decision among nephrologists, out of the 14 constructed models, the results of the majority decision showed improvement in sensitivity for 10 models (four of them were statistically significant) and specificity for eight models (five significant).

Conclusion

Our study showed a proof-of-concept for the classification of multiple glomerular findings in a comprehensive method of deep learning and suggested its potential effectiveness in improving diagnostic accuracy of clinicians.

Graphical abstract

Conclusion: AI models for classifying 7 major findings of glomeruli were developed, which may improve clinicians' diagnostic accuracy.

  1. Download : Download high-res image (252KB)
  2. Download : Download full-size image

Introduction

Renal pathology is important for the diagnosis and management of patients with kidney disease. The renal survival rate tends to be better with histologic evaluation by renal biopsy than without renal biopsy [1], thus, accurate and robust diagnosis is essential for the proper management of patients with kidney disease. On the other hands, making an accurate diagnosis is a time-consuming process even for experienced pathologists. It has been expected that automated processing to support this procedure will improve the efficiency of renal pathology and contribute to a more objective and standardized diagnosis [2], especially in hospitals, areas, or countries where there are an insufficient number of nephropathologists. A field called digital pathology, which aims to diagnose and quantify disease based on image data obtained by scanning pathological tissue specimens, has rapidly been developed. With the use of current state-of-the-art techniques of deep learning (DL), the artificial intelligence (AI) approach has made a significant progress in medical image analysis of retinal fundus images [3], skin images [4], and pathology mainly on cancer [5]. Currently, the implementation of these technologies in the clinical process and their effect on healthcare workers are of great interest [6].

There are some studies trying to apply DL to renal pathology. While some studies have validated DL models analyzing the structures other than the glomeruli, such as the tubules, blood vessels, and interstitium [[7], [8], [9], [10]], many studies have focused on the glomeruli, which present various histological findings essential for diagnosis. As a first step in the automation of this diagnostic procedure, detection of a glomerulus in a whole slide image (WSI) of renal tissue specimens has been recently attempted in many studies with the use of methods to define various features [[11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24]] or using convolutional neural networks (CNNs) [25], such as InceptionV3 [26], AlexNet [27], U-Net [28], R-CNN [29,30], or DeepLab V2 ResNet [31].

On the other hands, studies trying to classify pathologic findings from the glomerular images are still very few, and the pathological findings analyzed in these studies are quite limited. Barros et al. [32] constructed a model to classify proliferative lesions. Sheehan et al. [24] quantified mesangial matrix proliferation, numbers of nuclei, and capillary openness. Ginley et al. [31] also quantified nuclei, luminal space, and periodic acid-Schiff-positive component. Kannan et al. [26] and Marsh et al. [33] reported models to distinguish between sclerotic and nonsclerotic glomeruli. The pathological findings analyzed in these studies are quite limited, and do not cover the pathological findings necessary for accurate diagnosis, and there has been no study which enables the comprehensive evaluation of the essential pathological findings necessary for the diagnosis.

In this study, we focused on seven major pathological findings required for pathological diagnosis: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, and developed AI models to classify these findings. In addition, we examined whether our AI model can cooperate with nephrologists and improve their diagnostic performance. Although many studies have compared the performance between AI and the specialists [3,4], validation of the effect of the collaboration between AI and clinicians on the diagnostic judgment is also important and clinically relevant. Assuming a situation in which a majority decision of diagnosis is taken among specialists at a case conference, we demonstrated that the diagnostic performance was improved by adding AI model as one of the specialists.

Section snippets

Data preparation

We used WSIs of 283 renal biopsy cases that were agreed to be used for research at the Kyoto University Hospital between 2012 and 2017. The renal biopsy samples, including the transplanted allografts, were obtained by needle biopsy. Specimens that were stained by periodic acid-Schiff (PAS) and periodic acid methenamine silver (PAM) were used. Details of the staining and scanning of the slides are provided in the Supplementary Material.

Patients provided written informed consent for the use of

Patients and annotation of images

The train/validation and test datasets included 218 and 65 cases, respectively. The demographics and pathological diagnoses of these cases are shown in Table 1. The median numbers of annotated images by one nephrologist were 1625 (532–1698 [minimum, maximum]). In the train/validation dataset, the cropped glomerular images comprised of 5571 images on PAS staining and 5876 images on PAM staining. After removing the images labeled as artifact (examples in Supplementary Figure S2) by at least one

Discussion

We constructed AI models to classify several pathological findings of glomerular images. To the best of our knowledge, this is the first study to verify classification models that comprehensively included as many as seven findings essential for renal pathological diagnosis. In the classification of global sclerosis, our model showed high performance, AUC greater than 0.98, and that was also close to the performances of the nephrologists. Marsh et al. [33] reported a model to distinguish between

Author statement

All persons who meet authorship criteria are listed as authors, and all authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. Furthermore, each author certifies that this material or similar material has not been and will not be submitted to or published in any other publication before its appearance in International Journal of Medical

Transparency document

.

Summary Table

What was already known

  • -

    Deep learning models for analyzing renal pathology have been developed to establish an efficient and objective diagnosis in renal pathology, mainly for detecting glomeruli in whole slide images.

  • -

    Previous studies for the classification of glomerular pathological findings have focused on a limited number of findings, and there has been no study which enables the comprehensive evaluation of findings necessary for the diagnosis.

  • -

    It also remains unknown whether these

Declaration of Competing Interest

E. Uchino and Y. Tamada were given a budget for a joint research project with Fujitsu Ltd. M. Yanagita received research grants from Astellas, Chugai, Daiichi Sankyo, Kyowa Hakko Kirin, Mitsubishi Tanabe Pharma Corporation, MSD, Baxter, Takeda Pharmaceutical, KISSEI PHARMACEUTICAL, Dainippon Sumitomo Pharma, TAISHO TOYAMA PHARM, and Torii. The other authors declare no conflicts of interest.

Acknowledgements

For annotation to the dataset and discussion, we thank the following nephrologists in Department of Nephrology, Graduate School of Medicine, Kyoto University:

Yuki Sato, MD, PhD; Akira Ishii, MD, PhD; Keita P. Mori, MD, PhD; Naohiro Toda, MD, PhD; Keisuke Osaki, MD; Sayaka Sugioka, MD; Shinya Yamamoto, MD; Keiichi Kaneko, MD; Shunsuke Kawamura, MD; Youngna Kang, MD; Takahisa Yoshikawa, MD; Yukiko Kato, MD, PhD; Makiko Kondo, MD; Shigenori Yamamoto, MD; Yuichiro Kitai, MD; Akiko Oguchi, MD;

References (43)

  • F. Cabitza et al.

    Unintended consequences of machine learning in medicine

    JAMA

    (2017)
  • Y.G. Kim et al.

    A fully automated system using a convolutional neural network to predict renal allograft rejection: extra-validation with giga-pixel immunostained slides

    Sci. Rep.

    (2019)
  • M. Hermsen et al.

    Deep learning–Based histopathologic assessment of kidney tissue

    J. Am. Soc. Nephrol.

    (2019)
  • J. Zhang et al.

    Glomerulus extraction by optimizing the fitting curve

    2008 International Symposium on Computational Intelligence and Design

    (2018)
  • J. Ma et al.

    Glomerulus extraction by using genetic algorithm for Edge patching

    2009 IEEE Congress on Evolutionary Computation

    (2009)
  • Y. Hirohashi et al.

    Automated quantitative image analysis of glomerular desmin immunostaining as a sensitive injury marker in spontaneously diabetic torii rats

    Journal of Biomedical Image Processing.

    (2014)
  • T. Kato et al.

    Segmental HOG: new descriptor for glomerulus detection in kidney microscopy image

    BMC Bioinformatics

    (2015)
  • M. Gadermayr et al.

    Do we need large annotated training data for detection applications in biomedical imaging? A case study in renal glomeruli detection

    Mach. Learn. Med. Imaging

    (2016)
  • R. Marée et al.

    An approach for detection of glomeruli in multisite digital pathology

    2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI)

    (2016)
  • Y. Zhao et al.

    Automatic glomerulus extraction in whole slide images towards computer aided diagnosis

    2016 IEEE 12th International Conference on E-Science (E-Science)

    (2016)
  • M. Ishikawa et al.

    Extraction of glomeruli in whole slide imaging of kidney biopsy specimens

    Medical Imaging 2017: Digital Pathology, Proc. of SPIE

    (2017)
  • Cited by (54)

    • Integration of artificial intelligence and multi-omics in kidney diseases

      2023, Fundamental Research
      Citation Excerpt :

      Most reported studies have focused on segmentation tasks [126,127]. This can efficiently segment kidney histopathology in periodic acid–Schiff (PAS) stained sections, and it was reported that some outputs correlate well with specific scoring systems [128,129]. Several automatic segmentation [129] and classification [130] methods have been developed using AI, mostly using a supervised deep-learning approach.

    View all citing articles on Scopus
    1

    These authors contributed equally to this work.

    View full text