Abstract
The Mount Wilson magnetic classification of sunspot groups is thought to be meaningful to forecast flares’ eruptions. In this paper, we adopt a deep learning method, CornerNet-Saccade, to perform the Mount Wilson magnetic classification of sunspot groups. It includes three stages, generating object locations, detecting objects, and merging detections. The key technologies consist of the backbone as Hourglass-54, the attention mechanism, and the key points’ mechanism including the top-left corners and the bottom-right corners of the object by corner pooling layers. These technologies improve the efficiency of detecting the objects without sacrificing accuracy. A dataset is built by a total of 2486 composited images which are composited with the continuum images and the corresponding magnetograms from HMI and MDI. After training the network, the sunspot groups in a composited solar full image are detected and classified in 3 seconds on average. The test results show that this method has a good performance, with the accuracy, precision, recall, and mAP as 0.94, 0.93, 0.94, and 0.90, respectively. Moreover, the flare productivities of different types of sunspot groups from 2011 to 2020 are calculated. As 1, the flare productivities of , and sunspot groups are 0.14, 0.28, 0.61, 0.71, and 0.87, respectively. As 10, the flare productivities are 0.02, 0.07, 0.27, 0.45, and 0.65, respectively. It means that the , and types are indeed very closely related to the eruption of solar flares, especially the type. Based on the reliability of this method, the sunspot groups of the HMI solar full images from 2011 to 2020 are detected and classified, and the detailed data are shared on the website (https://61.166.157.71/MWMCSG.html).
1. Introduction
Sunspots are the typical manifestation of the strong magnetic fields on the solar surface. They are closely associated with solar activities, such as solar flares and coronal mass ejections [1–8]. These activities will disrupt the atmosphere of space and Earth, affect the ground short wave radio communication, and produce hazards such as magnetic storm [5].
Sunspots tend to appear in magnetically bipolar groups. Even a unipolar spot group really has dual polarity; the magnetic field strength of the other polarity is not intense enough to cause a visible spot. With the evolving of sunspot groups, they show various morphologies and complex magnetic polarities. Therefore, some classification schemes were proposed to describe the generality of sunspot groups. Typically, three classification schemes have been successively proposed: Zurich, McIntosh, and Mount Wilson magnetic classification. The Zurich sunspot classification was proposed to classify the sunspots into nine classes, A to J, comprising almost all stages of sunspots occurring mainly [9, 10]. Later, the Zurich classification was modified and expanded to the McIntosh classification, by adding size, stability, and complexity that appeared to correlate with flares [11, 12]. The form was expanded as , where is the modified Zurich class, is the type of principal spot, and is the degree of compactness in the interior of the group. They present that the correlations with flares are excel with earlier Zurich classification, especially the larger flares correlating with types Dki and Eki [12]. Both the Zurich classification and the McIntosh classification mainly depend on the morphology of sunspots, which require only white-light observations. The Mount Wilson magnetic classification was proposed earlier [13]. The scheme classifies sunspot groups into eight classes based on their morphological and magnetic properties together (for the detailed please, see Section 2). Therefore, both white-light observations and magnetograms are needed. Most M-class and X-class flares are found to be erupting above the complex sunspot groups, just like the , , and sunspots groups [14]. In particular, the sunspot groups have very high probabilities of flares [15]. Whichever classification schemes indicate that the more complex the morphological structure and the magnetic polarity of a sunspot group is, the higher the probability of the flare is [16, 17].
Previous authors have presented some methods for the above three classification schemes. For instance, Nguyen et al. [18] used machine learning techniques, including decision trees, rough sets, hierarchical clustering, and layered learning methods, to classify sunspot groups into seven-class modified Zurich classes. Abd et al. [19] employed support vector machines to achieve modified Zurich classification. Colak and Qahwaji [20] adopted the traditional image processing algorithm to detect sunspots, such as morphological operator, thresholding, and region growing; the features of sunspot groups, like length, height, and area, were then extracted; the McIntosh classification was determined finally by a decision tree using the extracted features. Hong et al. [21] performed the Mount Wilson magnetic classification according to features such as plus polarity, minus polarity, and magnetic neutral line, which are extracted from the continuum images and magnetograms after doing morphological operations and threshold methods. Padinhatteeri et al. [22] proposed an algorithm called SMART-DF to detect the type. They also adopted the threshold method to identify the umbra and penumbra, and then the pairs with a distance less than are reserved, where the distance is between each possible pairing of opposite-polarity umbrae. Finally, the pairs of opposite-polarity spots that pass several conditions involving the areas of umbra and penumbra are marked as type. Recently, Fang et al. [23] used a deep learning method based on Lenet-5 [24, 25] to classify the sunspot groups into , and -x types, where the sunspot type falls into three categories based on the Mount Wilson classification. However, the automatic method for the whole Mount Wilson classification in sunspot groups is still lacking.
In recent years, deep learning methods use various machine learning algorithms based on multilayer neural networks to solve problems [26], such as image detection and classification [27, 28]. The core of these methods is deep feature learning, which acquires hierarchical feature information through hierarchical networks. CornerNet-Saccade [29] is a new object detection algorithm, whose detection scheme based on the attention mechanism [30] and the key points [31–33] make it have advanced performance in the fields of object detection. This paper proposes a method for the Mount Wilson magnetic classification of sunspot groups based on CornerNet-Saccade. It obtains a good performance with the accuracy, precision, recall, and mAP, which all are above 0.90.
The main organizational structure of this paper is as follows. Section 2 introduces the Mount Wilson magnetic classification scheme briefly. Section 3 describes the data source and how to build the dataset. In Section 4, the method including the main steps is listed. Section 5 details the experiments, results, and evaluation of the method. Sections 6 and 7 discuss and summarize the results, respectively.
2. Mount Wilson Magnetic Classification of Sunspot Groups
The Mount Wilson Observatory regards the bipolar sunspot group as the basic type and other types as the deformations of the bipolar sunspot group according to the polarities of the magnetic fields [13]. Following the classification scheme, is a unipolar sunspot group, and is a distinct bipolar sunspot group with opposite polarities. is a complex sunspot group with irregular polarities. is a sunspot group with umbra which have opposite polarities and are separated by less than within one penumbra. is a bipolar sunspot group with more than one continuous line. Besides that, if the sunspot groups contain one or more spots, spots shall be added to the corresponding types. This classification is also called Hale class. The detailed description of the classification is as follows:(i) is a unipolar sunspot group(ii) is a bipolar sunspot group, with the simple and distinct division between opposite polarities(iii) is a complex sunspot group with irregular polarities(iv) is a sunspot group with umbra having opposite polarities within a penumbra and spans less then (v) is a sunspot group containing one or more sunspots(vi) is a bipolar sunspot group with more than one continuous polarity reversal line(vii) is a sunspot group containing one or more sunspots(viii) is a sunspot group containing one or more sunspots
3. Data
The data used in this paper come from the Helioseismic and Magnetic Imager (HMI) of the Solar Dynamics Observatory (SDO) [34] satellite and Michelson Doppler Imager (MDI) of the Solar and Heliospheric Observatory (SOHO) [35]. We used the continuum images and magnetograms, where the HMI data span from June 2010 to April 2020, and the MDI data span from July 2000 to September 2001. Both of them are selected with 12-hour interval.
Since the Mount Wilson magnetic classification needs to take into account the morphological structures and magnetic properties of the sunspot groups, we first composited the sunspot morphological structure information from the continuum image and the magnetic properties information from the magnetogram into one image. The main steps are as follows (Figure 1):(1)The continuum image and the cotemporal magnetogram are smoothed and normalized separately.(2)The penumbra boundaries and umbra regions of the continuum image are extracted by the threshold method [22]. They are represented by and , respectively. The positive and negative polar regions of the magnetogram are extracted, which are represented by and , respectively.(3) and , where is the positive region of the umbra, and is the negative region. is the intersection operation.(4)The penumbra boundaries corresponding to in the continuum image are contoured with green; the positive regions of the umbra corresponding to are filled with red; and the negative regions of the umbra corresponding to are filled with blue. Figure 1 shows an example. Figure 1(a) is the continuum image, Figure 1(b) is the magnetogram, and Figure 1(c) is the composited image.
(a)
(b)
(c)
A total of 2486 composited images were used to build the Mount Wilson magnetic classification datasets. The training set is used to train the deep learning network model, which consists of 1886 composited images. The test set is used to test the performance of the model, which consists of 600 composited images. The HMI data observed daily at 08:00:00 UT and 20:00:00 UT from 1 June 2010 to 31 December 2016 were mainly used for building the training set, and the data from 1 January 2017 to 30 April 2020 were used for the test set. All the samples were selected from them representatively. The MDI data which were observed in July 2000 (NOAA 9601) and September 2001 (NOAA 9087) were supplements because there were no and types occurring from 2010 until April 2020 according to the Solar Region Summary (SRS) text file (https://www.swpc.noaa.gov/). The and types only appeared in NOAA 9601 and NOAA 9087 from 1996 until April 2020. Therefore, all the scarce data including and types were added to the training set and the test set without intersection. Besides that, some data including the and types from 2010 to 2016 were moved from the training set to the test set because these two types have been relatively lacking since 2017. It should be noted that, from 1996 until now, the SRS file has not reported type. Therefore, the remainder of this paper no longer involves type.
LabelMe tool [36] was used to label the samples of the dataset. Each sample, a sunspot group, was given a label according to the combination of the definition of Mount Wilson magnetic classification [13] and the label of the SRS file. If the label of a sunspot group from the SRS was doubtful, then the label was mainly decided by the definition of Mount Wilson magnetic classification. A total of 10,286 samples were labeled in the datasets, including 7805 samples in the training set and 2433 samples in the testing set. Table 1 lists the numbers and proportions of all samples for various types of sunspot groups in the datasets. It can be seen that the numbers of samples differ greatly, which is due to the quite difference in occurrences of different types of sunspot groups. For example, the type appears frequently, while the type appears rarely. We had tried to solve the problem of the unbalanced numbers of the samples by oversampling. However, the experimental results show that the classification effect is not significantly improved. Therefore, this paper does not do anything to balance the samples of various types of sunspot groups.
4. Method
We adopted a new deep learning model, CornerNet-Saccade, to implement the Mount Wilson magnetic classification of sunspot groups. It mainly includes three stages, generating object locations, detecting objects, and merging detections. The main flow of the method is shown in Figure 2. The stage of generating object locations is plotted in blue, which is to find approximate locations and rough sizes of the sunspot groups on the solar full images. The stage of detecting objects is plotted in orange, which is to further determine the specific locations of the sunspot groups on the basis of the first stage and give the classification and confidence score of the sunspot groups. The third stage of merging detections is plotted in red, which is to merge the detections for eliminating the redundant boxes.
The deep learning method needs training the network by the samples in the training set first, which is to adjust the parameters of networks by executing the feedforward propagation and back propagation in iterations. The propagation algorithm performs a series of operations such as convolution and pooling to obtain the classification probability and bounding box of the objects. The loss value of the loss function, which is used to evaluate the error between the estimated value and the true value, is then fed back to the entire network by the back propagation algorithm; that is, the gradient of each layer of the networks is calculated using the gradient descent algorithm to continuously adjust the weight of each parameter to minimize the loss. The training procedure is iterated until the loss value converges steadily. The main steps, seen in Figure 2, are detailed in the following (Figure 3).(1)Downsizing the composited images for reducing inference time under limited memory.(2)Inputting the downsized images to the backbone network, Hourglass-54 [37]. The Hourglass-54 network is composed of three hourglass modules, which has a total depth of 54 layers. Each hourglass is a modular network with a symmetrical structure (see Figure 3). The hourglass network first applies three stages of convolution and downsampling layers to reduce the size of the input feature maps and then upsamples the features back to the original resolution by three stages of convolution and upsampling layers, combining features across scales. Downsampling is achieved by convolution with stride of 2, and upsampling is achieved by nearest neighbour interpolate [38]. One residual unit [39] is applied after each downsampling layer and upsampling layers, which corresponds to each box in Figure 3. Each residual unit is composed of two parts, three weight layers, and one identity mapping (or shortcut connections). Each weight layer is used to obtain deep-level information, which is designed as convolutions and following ReLU (Conv-ReLU). The identity mapping is used to retain the original information, which is designed as a 1 1 convolution for matching dimensions with the output of the weight layer. There are ten residual units in the hourglass module, which better solves the vanishing degradation problem of deep neural networks and improves the network performance. A total of three hourglass modules together allow for repeated bottom-up, top-down inference across scales, which capture and consolidate information across all scales of the image. The outputs of Hourglass-54 are a set of feature maps.(3)Applying 3 3 Conv-ReLU module and a 1 1 Conv-Sigmoid module on each feature map to predict attention maps by the attention mechanism. The attention mechanism is derived from the saccades mechanism, which refers to a sequence of rapid eye movements to fixate different image regions [40].The feature maps at finer scales are used for smaller objects and the ones at coarser scales are for larger objects. A total of three size scales of attention maps are predicted, corresponding to small, medium, and large objects, respectively. The attention map indicates the approximate locations and rough sizes of the object. Meanwhile, Hourglass-54 also generates bounding boxes for the detected objects in the downsized image. The bounding boxes are obtained by detecting the two key points, the top-left corners and the bottom-right corners of the object, through corner pooling layers [33]. If the Intersection over Union (IoU, Rezatofighi et al. [41]), the ratio of intersection and union of the bounding box and ground truth, is greater than a threshold (set as 0.3 in this work), the bounding box is reserved because it is more likely to contain the targets. So far, the approximate locations and sizes of the objects from the attention maps and bounding boxes are obtained separately.(4)the downsampled image at each possible location to find the locations of the objects more accurately. For the locations obtained from the attention maps, the zoom ratios of small, medium, and large objects are commonly set as 4, 2, and 1, respectively. For the locations obtained from the bounding boxes, the downsized image is enlarged according to the size of the bounding box.(5)the enlarged image back to the original image and then cropping the regions by taking the locations as the center points.(6)the regions according to the scores and then picking up the top locations with the highest scores.(7)the possible objects in each region through the second Hourglass-54. The bounding boxes of the objects are fine adjusted by detecting the two key points, the top-left corners and the bottom-right corners of the object, through the corner pooling layers.(8)Training the network. The loss function of the model consists of four losses with different weights [33]: a variant of focal loss, [42], the smooth L1 loss at ground-truth corner locations, [43], the loss of grouping the corners, [44], and the loss of separating the corners, [44].(9)Testing the network. The feedforward propagation is performed only once with the flow of Figure 2, except for back propagation with loss functions. Besides that, all detected bounding boxes are merged in the solar full image, where the redundant bounding boxes are eliminated based on Nonmaximum Suppression (NMS) [45]. The typical NMS is to find local maximum value and suppress nonlocal maximum values for each class, which aims to eliminate the redundant boxes of each class. This work modifies the typical NMS as eliminating the redundant boxes for all bounding boxes as a whole, not repeating for each class. On average, it takes about three seconds to process one image.
The experiments were performed on a personal computer equipped with a GeForce 2080 graphics card. The main programs deployed were Python 3.7, PyTorch 1.2, CUDA 10.1, GCC 7.4, etc. Adam [46] was chosen to optimize both losses for the attention maps and object detections. After repeated experiments, the hyperparameters such as the learning_rate, decay_rate, batch_size, and IoU of the network model were set to 0.000025, 10, 8, and 0.01, respectively. The loss value of the network converges stably after 335,000 iterations of training, which takes about 110 hours.
5. Results
5.1. Instance
Figure 4 shows two cases. The sunspot groups with solid line boxes are detected by our method, where the classification results are displayed at the upper left of each box. The active region (AR) number and classification of the sunspot groups given by NOAA are manually labeled to the box below. Note that the reports of NOAA are issued on today using data from yesterday. We marked them by using the correct reports.
(a)
(b)
It can be seen in Figure 4(a) that our method detects five sunspot groups. A total of three sunspot groups are marked by NOAA: NOAA 12299, NOAA 12297, and NOAA 12298. Our classification results are consistent with them, where the three sunspot groups are classified into , , and , respectively. Besides that, two other sunspot groups (red arrows 1 and 2) are detected by our method, and both of them are classified into type.
In Figure 4(b), eight sunspot groups are detected by our method, while seven are marked by NOAA. There are six sunspot groups that are the same by NOAA and by our method. NOAA 11971 (red arrow 3) is classified as type by our method; however, it is classified as type by NOAA. By analyzing the continuum image and the corresponding magnetogram, this sunspot group is a bipolar group, where there is no mixed polarity and no umbra of opposite polarity in the same penumbra region. Therefore, the sunspot group ought to be type. Besides that, there is a visible sunspot group above NOAA 11973 (red arrow 4). Our method detects the sunspot group and classifies it into type correctly.
5.2. Evaluation Metrics
We evaluated the performance of our method by four metrics: accuracy, precision, recall, and AP [47]. The definitions are as follows:where is the total number of samples. The true positive, , is the number of items correctly labeled as belonging to the positive class. The false positive, , is the number of items incorrectly labeled as belonging to the class. The false negative, , is the number of items which are not labeled as belonging to the positive class but should have been. The average precision, , is a typical performance metric of combination of precision and recall, which is calculated as the precision averaged across all recall values between 0 and 1.
Table 2 lists the metrics of the test set. Note that and are not taken into account in Table 2, because there are few and types of sunspot groups in the training set and the test set. It is meaningless to evaluate their corresponding performances. The mean accuracy, precision, recall, and mean AP (mAP) are all above 0.90. There are three types in which the AP values reach up to 0.90, namely, , , and . On the other hand, the APs of and types are a little lower than 0.90. We found that some small sunspots belonging to type are easily confused with the background, which results in false detections and missed detections. The type has some characteristics confused with , , and , so there are a few false detections.
5.3. Statistics
Based on the satisfactory results of this method, the solar full images of HMI from January 2011 to April 2020 were fed to the trained network for the Mount Wilson magnetic classification of sunspot groups. A total of 3086 composited images are available; most of them are at 20:00:00 UT and a few of them are near 20:00:00 UT. As a result, a total of 11, 681 sunspot groups are detected and classified. Note that only five types of sunspot groups appeared in this decade. They are , , , , and , where the numbers are 3951, 6094, 1158, 110, and 368, respectively. Therefore, only these five types are analyzed in this section. Detailed data are shared on the website (https://61.166.157.71/MWMCSG.html).
We summarized the total number for each type of sunspot group in months in Figure 5. The x-axis represents time in months, and the y-axis represents the number of sunspot groups. Because no data are provided in some days, we fill the missed data by averaging the previous day and the next day in order to plot the lines continuously in the figure.
The number of sunspot groups belonging to the same type varies greatly over time. For type, the highest number is 106 in December 2014; the lowest number falls to 1 in February 2020. The difference between the maximum value and minimum value reaches up to 105. For type, there are two months with the highest number of 144, including November 2011 and May 2013. On the other hand, the lowest number is 0 in October 2019. Besides that, the other types are relatively rare. The highest numbers of and are 49 and 12, respectively, which occur in February 2014 and November 2014, respectively. For type, there are two months with the highest number of 18: July 2012 and October 2013. For , , and , there are 24, 63, and 54 months with the lowest numbers of 0, respectively.
Solar activity level is closely related to three types of sunspot groups, such as , , and . From 2011 to 2016, there are a great number of , , and types. In particular, and types have hundreds of numbers monthly, where the sun is relatively active during this period. Since 2016, these types of sunspot groups have gradually decreased, even down to single digits, where the solar activities decrease. On the contrary, the numbers of and types have a little change, where the monthly amounts are only 12 and 18 at the peak, respectively.
The numbers of various types differ significantly in the same period of time, especially during the solar active period. For example, in November 2011 (the left-dashed vertical line), the numbers of , , , , and are 76, 144, 29, 5, and 7, respectively. The maximum difference (between and ) is up to 139. But, with the solar activity weakening, the difference becomes smaller. For example, in February 2020 (the right vertical line), the number of , , , , and are 1, 1, 0, 0, and 0, respectively. The maximum difference (between and ) is only 1.
5.4. The Relationship between Magnetic Classification and Solar Flares
Solar flares are closely related to the magnetic classification of sunspot groups. The solar flare productivity is used to quantify the relationship, which is calculated by the total number of flares divided by the number of sunspot groups. According to the peak fluxes of soft X-ray, flares are usually classified into four levels: B, C, M, and X. Within a certain time interval, the total importance, , is presented as [48, 49]
The flare productivity with 1, which is equivalent to a C1.0 flare, is analyzed. The occurrence of flares in an AR described in this paper is within 48 hours. The solar flare data come from the NOAA website (https://www.solarmonitor.org/). The number of sunspot groups and the corresponding flare productivity from January 2011 to April 2020 are listed in Table 3. They are also plotted in Figure 6 in order to make the relationship between magnetic classification of sunspot groups and flare productivities more intuitive.
It is found that the more complex the sunspot group is, the higher the corresponding flare productivity is. For instance, the flare productivities of C, M, and X belonging to the type are all lower than those belonging to the types (including and types), and those of the types are all lower than those of the types (including and types). In detail, the flare productivities of C-class, M-class, and X-class belonging to are 0.14, 0.02, and 0, respectively; and those belonging to are 0.28, 0.07, and 0.01, respectively. The flare productivities of these two types of sunspot groups are relatively low; in particular, X-class is almost zero. It means that these two types of sunspot groups with simple structures are difficult to produce large flares. On the other hand, the flare productivities of , , and are relatively high. The values for C-class flares reach up to 0.61, 0.71, and 0.87, respectively, and those for M-class flares are 0.27, 0.45, and 0.65, respectively. The X-class flares are very rare, as only forty occurred in the last ten years. The corresponding flare productivities of , , and are 0.02, 0.10, and 0.22, respectively. Especially, for a total of 105 , there erupted 23 X-class flares. That means , , and have much higher probabilities of flare eruption than the other types, especially . In additional, X-class flares basically occur above the or types.
The above results show that the Mount Wilson magnetic classification of sunspot groups is indeed closely related to solar flares. Therefore, the magnetic classification can be used as a major factor to predict the flare eruptions.
6. Discussion
6.1. Comparison with NOAA
We compared our results with NOAA carefully. Figure 7 shows the evolving procedure and classification results of NOAA 11158 from 13 to 15 February 2011.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
It can be seen that the sunspot group during the three days is mixed in polarity, with opposite polarity within the same penumbra and no more than separation of the umbra in the continuum image and the corresponding magnetogram. According to the description of Mount Wilson magnetic classification, the sunspot group should be . The results obtained by our method are all , while NOAA gives on the 13th and the 14th and on the 15th. X-class large flares erupted above NOAA 11158 on the 14th. Our method classifies it into two days earlier than NOAA and one day earlier than the eruption of this large flare.
Figure 8 shows the evolving procedure and classification results of NOAA 12260 from 8 to 10 January 2015. There is an AR visible at the lower right corner of NOAA 12258 on 8 January 2015. Our method detects the AR and classifies it into in time. This AR is marked as NOAA 12260 and classified into by NOAA on the 9th. Our method detects the AR one day earlier than NOAA.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Moreover, we counted the numbers of various types of sunspot groups detected by our method and NOAA in 2014 separately, as shown in Figure 9. The types of sunspot groups in this year are very abundant. There are a few sunspots in some days and very big sunspots in other days. Importantly, a lot of different levels of flares erupted in that year. Therefore, it is a typical year that we chose for analyzing the magnetic classification of sunspot groups day by day. The x-axis represents the classification of sunspot groups, and the y-axis represents the number of each type. Our results are very similar to those of NOAA. The total numbers detected by our method and NOAA are 2254 and 2073, respectively. In detail, there are 654 sunspot groups belonging to type by our method, 57 more than NOAA; 1155 sunspot groups, 129 more than NOAA; 302 sunspot groups, 8 less than NOAA; 23 sunspot groups, 4 more than NOAA; and 120 sunspot groups, 1 less than NOAA. The main reason why there are more and types by our method than those by NOAA is that our method detects more of small and sunspot groups. On the other hand, the sum of the , , and sunspot groups detected by our method is 445, and that by NOAA is 450. The total difference is very small, and the difference of every type is also very small. We found that the difference is mainly caused by the somewhat similar features of these types during the evolution of sunspot groups (see Figure 7). Note that the statistical results are obtained by 313 days in 2014 due to missed data.
Figure 10 shows the flare productivities by our method and NOAA in 2014. Here, the total flare level, , is set to 10 or greater, which is equivalent to an M1.0 flare. The flares productivities of , , , , and detected by our method are 0.03, 0.10, 0.37, 0.60, and 0.64, respectively. They are 0.05, 0.15, 0.37, 0.40, and 0.48 by NOAA, respectively. The general trends of the flare productivities of different types by our method and NOAA are similar. The more complex the sunspot group is, the higher its flare productivity is. In other words, the flare productivities of , , and are both higher than that of , and that of is higher than that of . If we have to say something different, NOAA obtains higher flare productivities of and compared to our method; our method has the same flare productivity as NOAA for ; for and that are most closely related to flares, the flare productivities of our method are higher than those of NOAA.
Note that the numbers of sunspot groups are counted somewhat differently in Figures 9 and 10. The number of sunspot groups in Figure 9 is the number of occurrences of sunspot groups in days, but in Figure 10, it is the number of different types of sunspot groups in their evolutions. For example, NOAA 11946 appeared on 5 January 2014 and disappeared on the 12th. It belongs to in three days and to in five days. The numbers of and in Figure 9 are 3 and 5, respectively, while both are 1 in Figure 10.
In summary, the multilevel features of various types of sunspot groups are extracted effectively using the deep learning method. This leads to a good performance in the magnetic classification of sunspot groups. Moreover, the complex sunspot groups including , , and have higher flare productivities, especially . Most X-class solar flares erupt above the complex sunspot groups. The Mount Wilson magnetic classification of the sunspot groups can indeed be used as an important predictor to predict the eruption of solar flares.
6.2. Some Problems
Although our method has achieved a good performance in the Mount Wilson magnetic classification of sunspot groups, there are still some problems.
Some small spots are difficult to be affirmed because they are very similar to background. Figure 11(a) shows the result by our method on 2 August 2011. A total of four sunspot groups are detected and classified. Two regions with red circle labeled as 1 and 2 are zoomed in panels 11(b) and 11(c). Neither our method nor NOAA detects Region 1 as an AR, but both of them detect Region 2 as an AR (NOAA 11264). The difference is that our method classifies Region 2 into , while NOAA does not give a classification result. It is not difficult to see that these two small spots are very similar. The main reason for miss-detected Region 1 is its environment. The reason is possibly due to the downsized images in order to reduce memory consumption. We believe this problem can be solved if the hardware conditions can support the high-resolution image without downsizing for deep learning operations.
(a)
(b)
(c)
Some sunspot groups are difficult to be judged when they are located at the edge of the solar surface. Figures 12(a) and 12(b) are subregions observed on May 10 2012 and 6 August 2012, respectively. In Figure 12(a), the sunspot group is about to disappear from the solar surface. On the other hand, the sunspot group in Figure 12(b) just appears on the solar surface. Both of them are classified into type by our method. However, the characteristics of these sunspot groups are not fully shown. It is difficult to judge whether it is true. For such situations, it is not easy to classify them accurately even by hand.
(a)
(b)
Moreover, some sunspot groups are difficult to be classified during their evolution according to the definition of the Mount Wilson magnetic classification of sunspot groups. Figure 13 shows one common example, NOAA 12257, from 9 to 11 January 2015. The sunspot group is judged as by our method but as by NOAA on 9 January 2015. We think that it should not be because of the umbra with the same polarity in the same penumbra. However, whether it belongs to or is not easy to distinguish clearly. The main reason is that there are some unclear words in the definition, such as simple, complex, and irregular. For instance, is a bipolar sunspot group, with the simple and obvious division between polarities; is a complex sunspot group with irregular polarity; is a bipolar sunspot group but with more than one continuous polarity reversal line. What level is simple or complex especially in the transitional period from one type into another type during its evolution? Here, it is judged by our method to be . Now, in the second row that follows, it is judged as because of the umbra with opposite polarity appear in the same penumbra, which are coincident by both methods. Whether it belongs to or is also a problem. We think that the sunspot group is inclined to be for its complex sunspot group with irregular polarity here. In the third row, both methods obtain the same results, .
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
7. Conclusion
This paper proposes a deep learning method for the Mount Wilson magnetic classification of sunspot groups. The continuum images and the magnetograms from HMI spanning from June 2010 to April 2020 and MDI spanning from July 2000 to September 2001 are employed to build the dataset. We first composite the morphological structures of sunspot groups from the continuum image and the magnetic properties from the cotemporal magnetogram into one image. A total of 2486 composited images are then labeled to build the dataset of the Mount Wilson magnetic classification, which are divided into a training set and a test set at a ratio of 3 : 1.
A deep learning model, CornerNet-Saccade, is adopted to detect and classify the sunspot groups. It mainly includes three stages, generating object locations, detecting objects, and merging detections. The first stage is to find approximate locations and rough sizes of the sunspot groups on the solar full images. The backbone of the method is Hourglass-54, which consists of 3 hourglass modules with a total depth of 54 layers. Each hourglass module is a symmetrical structure, which downsamples to a very low resolution and then upsamples and combines features across multiple resolutions. The multiple hourglass modules together allow for repeated bottom-up, top-down inference across scales, which capture and consolidate information across all scales of the image. A total of three size scales of attention maps are predicted using the feature maps gotten from Hourglass-54 by the attention mechanism, corresponding to small, medium, and large objects. Meanwhile, the bounding boxes are generated by detecting the two key points, the top-left corners and the bottom-right corners of the object, through corner pooling layers. The second stage is to further determine the specific locations and bounding boxes of the sunspot groups. The locations gotten from the first stage are ranked by their scores, and then the top ones are picked up. They are fine-adjusted by corner pooling layers using the feature maps through the second Hourglass-54. The third stage is to merge the detections by NMS for eliminating the redundant boxes. These key technologies improve the efficiency of the deep learning method without sacrificing accuracy.
The model is trained on a personal computer equipped with a GeForce 2080 graphics card. After 335,000 iterations, the loss value converges stably. It takes about 3 seconds to detect and classify the sunspot groups in a composited solar full image. The experimental results show that this method has an excellent performance in the detection and classification. The accuracy, precision, recall, and mAP reach up to 0.94, 0.93, 0.94, and 0.90, respectively.
Based on the reliable performance of our method, the classifications of sunspot groups in the past 10 years from January 2011 to April 2020 are analyzed, which are shared on the website (https://61.166.157.71/MWMCSG.html). In addition, we calculate the flare productivities of various types of sunspot groups from January 2011 to April 2020 in order to describe the relationship between the magnetic classification of sunspot groups and solar flares. If is set to greater than or equal to 1, the flare productivities of , , , , and sunspot groups are 0.14, 0.28, 0.61, 0.71, and 0.87, respectively. If is set to greater than or equal to 10, they are 0.02, 0.07, 0.27, 0.45, and 0.65, respectively. Among them, the flare productivities of , , and reach up to 0.60 with 1. The productivities of and reach up to 0.45 even with 10. This means that , , and sunspot groups are indeed very closely related to solar flare eruptions, especially the sunspot groups.
We also compare the classifications of sunspot groups with NOAA using the data in 2014. The results show that our method detects more and sunspot groups which are small and faint; therefore, the numbers of and are 654 and 1155, respectively, 57 and 129 more than those of NOAA, respectively. Our method is good at distinguishing the more complex sunspot groups including , , and classes, owing to the features of different types learned by the deep learning networks, where the detection schemes of CornerNet-Saccade, such as Hourglass-54, key points, and the attention map mechanism, make the features of the multiple levels fused available.
In the future, we plan to use the Mount Wilson magnetic classification of sunspot groups as the main predictor to establish a more reasonable solar flare prediction model.
Data Availability
The magnetogram and continuum image data used to support the findings of this study are observed by SDO/HMI and SOHO/MDI; all of the FITS files we have used are downloaded from https://jsoc.stanford.edu/. The SRS text files can be downloaded from the NOAA/SWPC website https://www.swpc.noaa.gov/. The training set and test set data for deep learning of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
The authors are grateful for the support received from the National Natural Science Foundation of China (nos. 11763004, 11573012, 11803085, 12063003, and U1931107) and the Open Research Program of the Key Laboratory of Solar Activity of the Chinese Academy of Sciences (no. KLSA202019). This work was also supported by the National Key Research and Development Program of China (2018YFA0404603), Yunnan Key Research and Development Program (2018IA054), and Yunnan Applied Basic Research Project (2018FB103). The continuum images and magnetograms used in this study were kindly provided by SDO and SOHO. The text files were kindly provided by NOAA/SWPC. The authors thank them for maintaining and providing the data.