Elsevier

Knowledge-Based Systems

Volume 208, 15 November 2020, 106465
Knowledge-Based Systems

Neural networks model based on an automated multi-scale method for mammogram classification

https://doi.org/10.1016/j.knosys.2020.106465Get rights and content

Abstract

Breast cancer is the most commonly diagnosed cancer among women. Convolutional neural networks (CNN)-based mammogram classification plays a vital role in early breast cancer detection. However, it pays too much attention to the lesions of mammograms and ignores the global characteristics of the breast. In the process of diagnosis, doctors not only pay attention to the features of local lesions but also combine with the comparison to the global characteristics of breasts. Mammogram images have a visible characteristic, which is that the original image is large, while the lesions are relatively small. It means that the lesions are easy to overlook. This paper proposes an automated multi-scale end-to-end deep neural networks model for mammogram classification, that only requires mammogram images and class labels (without ROI annotations). The proposed model generated three scales of feature maps that make the classifier combine global information with the local lesions for classification. Moreover, the images processed by our method contain fewer non-breast pixels and retain the small lesions information as much as possible, which is helpful for the model to focus on the small lesions. The performance of our method is verified on the INbreast dataset. Compared to other state-of-the-art mammogram classification algorithms, our model performs the best. Moreover, the multi-scale method is applied to the networks with fewer parameters that can achieve comparable performance, while saving 60% of the computing resources. It shows that the multi-scale method can work for both performance and computational efficiency.

Introduction

Nowadays, breast cancer is the most commonly diagnosed cancer and the second leading cause of cancer death among women globally [1], [2]. In China, the average age at diagnosing for breast cancer is decreasing, and the number of breast cancer patients is increasing annually [3]. Early detection and early treatment are crucial to lowering the death rates of breast cancer [4]. Fortunately, some imaging techniques, such as mammogram screening and ultrasound examinations, are beneficial to early breast cancer detection. Mammogram screening has several advantages, including easy operating, affordable price, and significantly better imaging results of calcified breast cancer, which makes diagnosis easier. For these reasons, mammogram screening has become the most popular technology to diagnose breast diseases in women over 40 years old [5].

In general, the clinicians of the radiology department scan the left and right breasts of patients from the craniocaudal (CC) and mediolateral oblique (MLO) views. Hence, mammogram screening usually contains four images, from left-CC, right-CC, left-MLO, and right-MLO views [6], respectively. Fig. 1 shows the example of mammogram images with the four views.

Mammogram images reveal lots of information about the breast, including the density, shape and size of the breast, as well as the suspected lesions such as calcifications and masses, etc. The mentioned information is beneficial to well-trained doctors for making a preliminary diagnosis. Usually, some hospitals have mammogram scanning equipment and operating clinicians but are in a lack of well-trained doctors. Therefore, the development of trained doctors-level computer-aided diagnosis (CAD) systems to diagnose mammogram images will be of great benefit [4].

Mammogram classification is the most critical part of the CAD systems. Traditional mammogram classification methods typically focus on constructing hand-crafted features to describe the characteristics of breast cancer [7], [8], [9]. These features must be carefully designed based on the region-of-interest (ROI) annotations and the doctor’s experience. The defects of these approaches can be summarized as follows. The first, ROI annotations require trained doctors to make pixel-level annotations on the image, which consumes a lot of effort. The second, the hand-crafted features are limited by medical knowledge about breast cancer and the precision of ROI annotations [10], which means that hand-crafted features may not be able to describe the characteristics of breast cancer precisely. The third, from the results reported [7], [8], [9], the classification and generalization performances of traditional methods are not good enough for clinical use.

Recently, the CNN-based method has achieved great success in computer vision and related research fields for its powerful feature extraction ability. The CNN method has high potential in mammogram image analysis, but the following challenges remain [11], [12], [13], [14], [15], [16], [17].

(1) CNN tends to achieve great performance with a large amount of data. But mammogram datasets have much fewer samples than natural image datasets. For instance, INbreast [18] is an open dataset of mammogram images and only has 410 images. In contrast, ImageNet [19] has 14,197,122 images and MSCOCO [20] has 82,783 images. Therefore, it is difficult to train a CNN model that performs well on the mammogram dataset.

(2) The dimensions of mammogram images usually are higher than those of natural images. The sizes of the images in INbreast are about 2000 × 3000 pixels, whereas those in ImageNet and MSCOCO are about 300 × 300 pixels. CNN usually resizes the input images to a fixed size, such as 299 × 299 or 224 × 224. For the raw mammogram images are too large, some information of lesions may be lost by directly resizing, which will result in that CNN cannot learn from these lesions.

(3) As can be seen in Fig. 1, there is a significant amount of redundant region in the raw mammogram images. The lesions only exist in the breast, and the breast region only occupies less than half of the whole picture. The redundant areas are not only useless for classification but also interfere with model training.

(4) Doctors combine the features of the whole breast and the suspected lesions in mammogram images for the diagnosis, while CNN usually relies on its powerful feature extraction ability to classify the natural pictures. Directly applying the CNN model to mammogram classification is difficult to achieve good performance [13], [14], [21].

(5) At present, current CNN models achieve excellent performance with ROI annotations-assisted training. However, ROI labeling is expensive and not easy to obtain [22], [23]. Moreover, the performance of a CNN that training without ROI annotations is not good enough for practical application [12], [21]. Improving the performance of classification models without ROI annotations is full of challenges.

To deal with the mentioned challenges, this paper proposes a multi-scale CNN model for mammogram classification. The model consists of the breast region segmentation (BRS) module, the feature extraction module, the multi-scale feature module, and the classifier module. The images preprocessed by the BRS module contain fewer non-breast (2) and (3) with a practical solution. To make challenge (1) less challenging, random transforms are performed on the input images firstly. Next, the dense connection mechanism of DenseNet can avoid overfitting to some degree, which is helpful for the training of the networks on the small dataset. So the pre-trained DenseNet is used for the feature extraction module. It learns from the mammogram images to generate feature maps representing the calcifications and masses. After that, the multi-scale module fuses these feature maps and generates new ones at three scales. The three scale feature maps represent the information of the whole breast and the suspected lesions, which offers challenge (4) a feasible way. Its performance is comparable to the model with ROI annotations-assisted training, and this can address challenge (5). Finally, this paper uses a fully connected layer as the classifier. The classifier collects multi-scale instead of single-scale feature maps, which helps the classifier work better.

The main contributions of this paper are summarized as follows.

(1) Based on the analysis of the doctor’s diagnostic process and the characteristics of mammogram images, an automated multi-scale CNN model for mammogram classification is proposed in this paper. This method only requires original mammogram images and the corresponding category labels (without ROI annotations) and gets the state-of-the-art classification performance on the INbreast dataset. The proposed model saves a lot of efforts of labeling and makes the model more comfortable to apply.

(2) The multi-scale module is proposed to generate feature maps at three scales, which provides the model with the information of global breast and local lesions instead of only focuses on the local lesions. Moreover, considering the finite computational ability, the DenseNet [24] is replaced with the MobileNet [25], which can save 60% of computing resources while maintaining comparable performance.

(3) The BRS module is proposed to preprocess the raw mammogram images based on the characteristics of mammogram images. The processed images contain fewer non-breast pixels and the image size is small, which helps the model to focus on the breast region for better training.

Section snippets

Related work

In recent years, neural networks have been swiftly developed in theory and application [26], [27], [28], [29], [30], especially CNN. Since AlexNet [31] won the championship in the ILSVRC2012 challenge, CNN has attracted widespread attention from researchers. Subsequently, the structures of CNN, such as VGG Net [32], Inception Net [33], ResNet [34] and DenseNet [24], were designed to be deeper, which resulted in better feature extraction capabilities. The development of these structures has

Methods

The networks structure of the proposed method is shown in Fig. 3. It consists of the following four modules.

(1) The original mammogram images are preprocessed by the BRS module. The module aims to crop out the background regions that are irrelevant for classification and improve the proportion of effective pixels in the image.

(2) Pre-trained CNN is constructed to extract mammogram features. DenseNet [24] and MobileNet [25] are taken as feature extraction modules, respectively,.

(3) To improve

Results

The proposed method is evaluated on the public dataset INbreast [18], [46], which contains multiple view images of 115 cases for a total of 410 images. Of these, 116 images containing benign or malignant masses are defined as positive samples. While the rest of them are defined as negative samples. Compared to other mammogram datasets, such as the mini-MIAS [47] and DDSM [48] dataset, INbreast’s images are original images with higher quality. The images in the dataset are full-field digital

Conclusion

Based on the analysis of the doctor’s diagnostic process and the characteristics of mammogram images, an automated multi-scale CNN model for mammogram classification is proposed in this paper. The diagnosis requires to combine features of global breast and suspected lesions. Hence, the multi-scale method is proposed to generates the multi-scale feature maps, which makes CNN not only pay attention to the features of the local lesions, but also combine with the comparison to the global features

CRediT authorship contribution statement

Lizhang Xie: Investigation, Formal analysis, Conceptualization, Ideas, Software, Methodology, Writing - original draft. Lei Zhang: Funding acquisition, Supervision, Project administration, Writing - review & editing. Ting Hu: Investigation, Data curation, Visualization. Haiying Huang: Software, Validation. Zhang Yi: Writing - review & editing, Resources.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work was supported by the National Natural Science Foundation of China under Grants No. 61772353.

References (49)

  • FernandezA. et al.

    A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets

    Fuzzy Sets and Systems

    (2008)
  • FreddieB. et al.

    Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries

    CA: A Cancer Journal for Clinicians

    (2015)
  • MossS.M. et al.

    The impact of mammographic screening on breast cancer mortality in europe: A review of trend studies

    J. Med. Screen.

    (2012)
  • WanqingC. et al.

    Cancer incidence and mortality in China, 2014

    Chin. J. Cancer Res.

    (2018)
  • GigerM.L. et al.

    Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer

    Annu. Rev. Biomed. Eng.

    (2013)
  • SicklesE.A.

    Breast cancer screening outcomes in women ages 40-49: Clinical experience with service screening using modern mammography

    J. Nat. Cancer Inst. Monogr.

    (1997)
  • Lauby-SecretanB. et al.

    Breast-cancer screeningviewpoint of the IARC working group

    N. Engl. J. Med.

    (2015)
  • BellottiR. et al.

    A completely automated CAD system for mass detection in a large mammographic database

    Med. Phys.

    (2006)
  • CampaniniR. et al.

    A novel featureless approach to mass detection in digital mammograms based on support vector machines

    Phys. Med. Biol.

    (2004)
  • BrakeG.M. et al.

    An automatic method to discriminate malignant masses from normal tissue in digital mammograms.

    Phys. Med. Biol.

    (2000)
  • BaeM.S. et al.

    Breast cancer detected with screening US: Reasons for nondetection at mammography

    Radiology

    (2014)
  • AgarwalR. et al.

    Mass detection in mammograms using pre-trained deep learning models

  • SasikalaS. et al.

    Fusion of MLO and CC view binary patterns to improve the performance of breast cancer diagnosis

    Curr. Med. Imaging Rev.

    (2018)
  • GuX. et al.

    Multi-view learning for mammogram analysis: Auto-diagnosis models for breast cancer

  • Cited by (30)

    • Mammogram classification based on a novel convolutional neural network with efficient channel attention

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      In [36], Zhu et al. proved that CNNs can still perform well without ROI annotations in training, and their accuracy on the INbreast dataset is as high as 90%. Xie et al. [37] got excellent performance without ROI annotations by an automated multi-scale end-to-end DCNN method. In addition, Albalawi et al. [38] applied wiener filter to expel the noise and background of images.

    • Evolutionary neural architecture search based on evaluation correction and functional units

      2022, Knowledge-Based Systems
      Citation Excerpt :

      In recent years, neural architecture search (NAS) has attracted the attention of many researchers [1]. It has been successfully applied to medical classification [2], target detection [3], etc. It frees researchers from the heavy burden of designing neural network architectures.

    View all citing articles on Scopus
    View full text