Neural networks model based on an automated multi-scale method for mammogram classification
Introduction
Nowadays, breast cancer is the most commonly diagnosed cancer and the second leading cause of cancer death among women globally [1], [2]. In China, the average age at diagnosing for breast cancer is decreasing, and the number of breast cancer patients is increasing annually [3]. Early detection and early treatment are crucial to lowering the death rates of breast cancer [4]. Fortunately, some imaging techniques, such as mammogram screening and ultrasound examinations, are beneficial to early breast cancer detection. Mammogram screening has several advantages, including easy operating, affordable price, and significantly better imaging results of calcified breast cancer, which makes diagnosis easier. For these reasons, mammogram screening has become the most popular technology to diagnose breast diseases in women over 40 years old [5].
In general, the clinicians of the radiology department scan the left and right breasts of patients from the craniocaudal (CC) and mediolateral oblique (MLO) views. Hence, mammogram screening usually contains four images, from left-CC, right-CC, left-MLO, and right-MLO views [6], respectively. Fig. 1 shows the example of mammogram images with the four views.
Mammogram images reveal lots of information about the breast, including the density, shape and size of the breast, as well as the suspected lesions such as calcifications and masses, etc. The mentioned information is beneficial to well-trained doctors for making a preliminary diagnosis. Usually, some hospitals have mammogram scanning equipment and operating clinicians but are in a lack of well-trained doctors. Therefore, the development of trained doctors-level computer-aided diagnosis (CAD) systems to diagnose mammogram images will be of great benefit [4].
Mammogram classification is the most critical part of the CAD systems. Traditional mammogram classification methods typically focus on constructing hand-crafted features to describe the characteristics of breast cancer [7], [8], [9]. These features must be carefully designed based on the region-of-interest (ROI) annotations and the doctor’s experience. The defects of these approaches can be summarized as follows. The first, ROI annotations require trained doctors to make pixel-level annotations on the image, which consumes a lot of effort. The second, the hand-crafted features are limited by medical knowledge about breast cancer and the precision of ROI annotations [10], which means that hand-crafted features may not be able to describe the characteristics of breast cancer precisely. The third, from the results reported [7], [8], [9], the classification and generalization performances of traditional methods are not good enough for clinical use.
Recently, the CNN-based method has achieved great success in computer vision and related research fields for its powerful feature extraction ability. The CNN method has high potential in mammogram image analysis, but the following challenges remain [11], [12], [13], [14], [15], [16], [17].
(1) CNN tends to achieve great performance with a large amount of data. But mammogram datasets have much fewer samples than natural image datasets. For instance, INbreast [18] is an open dataset of mammogram images and only has 410 images. In contrast, ImageNet [19] has 14,197,122 images and MSCOCO [20] has 82,783 images. Therefore, it is difficult to train a CNN model that performs well on the mammogram dataset.
(2) The dimensions of mammogram images usually are higher than those of natural images. The sizes of the images in INbreast are about 2000 × 3000 pixels, whereas those in ImageNet and MSCOCO are about 300 × 300 pixels. CNN usually resizes the input images to a fixed size, such as 299 × 299 or 224 × 224. For the raw mammogram images are too large, some information of lesions may be lost by directly resizing, which will result in that CNN cannot learn from these lesions.
(3) As can be seen in Fig. 1, there is a significant amount of redundant region in the raw mammogram images. The lesions only exist in the breast, and the breast region only occupies less than half of the whole picture. The redundant areas are not only useless for classification but also interfere with model training.
(4) Doctors combine the features of the whole breast and the suspected lesions in mammogram images for the diagnosis, while CNN usually relies on its powerful feature extraction ability to classify the natural pictures. Directly applying the CNN model to mammogram classification is difficult to achieve good performance [13], [14], [21].
(5) At present, current CNN models achieve excellent performance with ROI annotations-assisted training. However, ROI labeling is expensive and not easy to obtain [22], [23]. Moreover, the performance of a CNN that training without ROI annotations is not good enough for practical application [12], [21]. Improving the performance of classification models without ROI annotations is full of challenges.
To deal with the mentioned challenges, this paper proposes a multi-scale CNN model for mammogram classification. The model consists of the breast region segmentation (BRS) module, the feature extraction module, the multi-scale feature module, and the classifier module. The images preprocessed by the BRS module contain fewer non-breast (2) and (3) with a practical solution. To make challenge (1) less challenging, random transforms are performed on the input images firstly. Next, the dense connection mechanism of DenseNet can avoid overfitting to some degree, which is helpful for the training of the networks on the small dataset. So the pre-trained DenseNet is used for the feature extraction module. It learns from the mammogram images to generate feature maps representing the calcifications and masses. After that, the multi-scale module fuses these feature maps and generates new ones at three scales. The three scale feature maps represent the information of the whole breast and the suspected lesions, which offers challenge (4) a feasible way. Its performance is comparable to the model with ROI annotations-assisted training, and this can address challenge (5). Finally, this paper uses a fully connected layer as the classifier. The classifier collects multi-scale instead of single-scale feature maps, which helps the classifier work better.
The main contributions of this paper are summarized as follows.
(1) Based on the analysis of the doctor’s diagnostic process and the characteristics of mammogram images, an automated multi-scale CNN model for mammogram classification is proposed in this paper. This method only requires original mammogram images and the corresponding category labels (without ROI annotations) and gets the state-of-the-art classification performance on the INbreast dataset. The proposed model saves a lot of efforts of labeling and makes the model more comfortable to apply.
(2) The multi-scale module is proposed to generate feature maps at three scales, which provides the model with the information of global breast and local lesions instead of only focuses on the local lesions. Moreover, considering the finite computational ability, the DenseNet [24] is replaced with the MobileNet [25], which can save 60% of computing resources while maintaining comparable performance.
(3) The BRS module is proposed to preprocess the raw mammogram images based on the characteristics of mammogram images. The processed images contain fewer non-breast pixels and the image size is small, which helps the model to focus on the breast region for better training.
Section snippets
Related work
In recent years, neural networks have been swiftly developed in theory and application [26], [27], [28], [29], [30], especially CNN. Since AlexNet [31] won the championship in the ILSVRC2012 challenge, CNN has attracted widespread attention from researchers. Subsequently, the structures of CNN, such as VGG Net [32], Inception Net [33], ResNet [34] and DenseNet [24], were designed to be deeper, which resulted in better feature extraction capabilities. The development of these structures has
Methods
The networks structure of the proposed method is shown in Fig. 3. It consists of the following four modules.
(1) The original mammogram images are preprocessed by the BRS module. The module aims to crop out the background regions that are irrelevant for classification and improve the proportion of effective pixels in the image.
(2) Pre-trained CNN is constructed to extract mammogram features. DenseNet [24] and MobileNet [25] are taken as feature extraction modules, respectively,.
(3) To improve
Results
The proposed method is evaluated on the public dataset INbreast [18], [46], which contains multiple view images of 115 cases for a total of 410 images. Of these, 116 images containing benign or malignant masses are defined as positive samples. While the rest of them are defined as negative samples. Compared to other mammogram datasets, such as the mini-MIAS [47] and DDSM [48] dataset, INbreast’s images are original images with higher quality. The images in the dataset are full-field digital
Conclusion
Based on the analysis of the doctor’s diagnostic process and the characteristics of mammogram images, an automated multi-scale CNN model for mammogram classification is proposed in this paper. The diagnosis requires to combine features of global breast and suspected lesions. Hence, the multi-scale method is proposed to generates the multi-scale feature maps, which makes CNN not only pay attention to the features of the local lesions, but also combine with the comparison to the global features
CRediT authorship contribution statement
Lizhang Xie: Investigation, Formal analysis, Conceptualization, Ideas, Software, Methodology, Writing - original draft. Lei Zhang: Funding acquisition, Supervision, Project administration, Writing - review & editing. Ting Hu: Investigation, Data curation, Visualization. Haiying Huang: Software, Validation. Zhang Yi: Writing - review & editing, Resources.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
This work was supported by the National Natural Science Foundation of China under Grants No. 61772353.
References (49)
- et al.
A deep feature based framework for breast masses classification
Neurocomputing
(2016) - et al.
Inbreast: Toward a full-field digital mammographic database
Academic Radiol.
(2012) - et al.
Dynamical properties of background neural networks with uniform firing rate and background input
Chaos Solitons Fractals
(2007) - et al.
Intelligent fault diagnosis of rolling bearings based on normalized cnn considering data imbalance and variable working conditions
Knowl.-Based Syst.
(2020) - et al.
CNN-Based image recognition for topology optimization
Knowl.-Based Syst.
(2020) - et al.
Automated retinopathy of prematurity screening using deep neural networks
EBioMedicine
(2018) - et al.
Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks
Neurocomputing
(2018) - et al.
Automated diagnosis of breast ultrasonography images using deep neural networks
Med. Image Anal.
(2019) - et al.
Automatic diagnosis for thyroid nodules in ultrasound images by deep neural networks
Med. Image Anal.
(2020) - et al.
Network-based direction of movement prediction in financial markets
Eng. Appl. Artif. Intell.
(2020)
A study of the behaviour of linguistic fuzzy rule based classification systems in the framework of imbalanced data-sets
Fuzzy Sets and Systems
Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries
CA: A Cancer Journal for Clinicians
The impact of mammographic screening on breast cancer mortality in europe: A review of trend studies
J. Med. Screen.
Cancer incidence and mortality in China, 2014
Chin. J. Cancer Res.
Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer
Annu. Rev. Biomed. Eng.
Breast cancer screening outcomes in women ages 40-49: Clinical experience with service screening using modern mammography
J. Nat. Cancer Inst. Monogr.
Breast-cancer screeningviewpoint of the IARC working group
N. Engl. J. Med.
A completely automated CAD system for mass detection in a large mammographic database
Med. Phys.
A novel featureless approach to mass detection in digital mammograms based on support vector machines
Phys. Med. Biol.
An automatic method to discriminate malignant masses from normal tissue in digital mammograms.
Phys. Med. Biol.
Breast cancer detected with screening US: Reasons for nondetection at mammography
Radiology
Mass detection in mammograms using pre-trained deep learning models
Fusion of MLO and CC view binary patterns to improve the performance of breast cancer diagnosis
Curr. Med. Imaging Rev.
Multi-view learning for mammogram analysis: Auto-diagnosis models for breast cancer
Cited by (30)
A novel approach for automatic tumor detection and localization in mammography images via mixture of factor analyzers based on co-clustering
2024, Biomedical Signal Processing and ControlMsGoF: Breast lesion classification on ultrasound images by multi-scale gradational-order fusion framework
2023, Computer Methods and Programs in BiomedicineMammogram classification based on a novel convolutional neural network with efficient channel attention
2022, Computers in Biology and MedicineCitation Excerpt :In [36], Zhu et al. proved that CNNs can still perform well without ROI annotations in training, and their accuracy on the INbreast dataset is as high as 90%. Xie et al. [37] got excellent performance without ROI annotations by an automated multi-scale end-to-end DCNN method. In addition, Albalawi et al. [38] applied wiener filter to expel the noise and background of images.
Evolutionary neural architecture search based on evaluation correction and functional units
2022, Knowledge-Based SystemsCitation Excerpt :In recent years, neural architecture search (NAS) has attracted the attention of many researchers [1]. It has been successfully applied to medical classification [2], target detection [3], etc. It frees researchers from the heavy burden of designing neural network architectures.