A Performance Evaluation of Potential Intensity over the Tropical Cyclone Passage to South Korea Simulated by CMIP5 and CMIP6 Models

Abstract

Potential intensity (PI) is a metric for climate model evaluation of TC-related thermodynamic conditions. However, PI is utilized usually for assessing basin-wide TC-related thermodynamic conditions, and not for evaluating TC passage to a certain region. Here we evaluate model-simulated PI over the passage of TCs affecting South Korea (KOR PI) as well as the PI over the entire western North Pacific basin (WNP PI) using 25 CMIP5 and 27 CMIP6 models. In terms of pattern correlations and bias-removed root mean square errors, CMIP6 model performances for KOR PI are found to be noticeably improved over CMIP5 models in contrast to negligible improvement for WNP PI, although it is not in terms of normalized standard deviations. This implies that thermodynamic condition on the route of TCs affecting South Korea is likely better captured by CMIP6 models than CMIP5 models.

1. Introduction

Accurate projection of future TC activity requires climate models that reliably simulate TC activity in the present climate and, hence, evaluation of TC simulation accuracy for each model is necessary. The best way should be to detect TCs directly from climate models and use them for evaluation. To do so, horizontal resolutions of the models should be finer than 50 km [1]. However, many climate models in the Coupled Model Intercomparison Project phases 5 and 6 (CMIP6 and CMIP6) have coarse resolutions [2,3], so TCs directly detected from CMIP models have large biases from the observed TCs (e.g., weaker intensity, less TC formation, and rare recurving TCs) [1,4]. Several studies dynamically downscaled CMIP models, by using regional climate models (RCMs) to obtain higher resolution data [5,6]. However, this requires intensive computing resources as much as global climate models.

As an alternative way, several studies used potential intensity (PI) which is an index used to represent the thermodynamical upper limit of TC intensity under a given set of climate conditions [7,8,9]. The PI is based on thermodynamic theory including sea surface temperature (SST) and vertical profiles of temperature and humidity [5]. The PI can be easily calculated from numerical model outputs and utilized as the background thermodynamic environment to explain TC activity directly detected from numerical models [6,10,11].

Most previous studies however evaluated the performance on simulated PI for assessing basin-wide TC-related thermodynamic conditions. Because the PI represents TC’s maximum potential intensity in each grid point, the basin-wide PI may not be suitable for assessing TC passage to a certain region, such as TC passage to South Korea, which is only limited areas of the basin [12]. Thus, in this study, we evaluate PI specifically over the area of TC passage to South Korea to assess model performance for simulated thermodynamic environments for the activity of TCs affecting Korea by using CMIP5 and CMIP6 models. For comparison, we evaluated PI simulation over the entire western North Pacific (WNP) as well.

2. Data and Methods

The historical monthly outputs of 25 CMIP5 and 27 CMIP6 models were used to calculate PI for evaluation [2,3]. The models used are listed in

Table 1. The CMIP5 and CMIP6 models used and their horizontal resolutions.

CMIP5	Resolutions (Latitude × Longitude)	CMIP6	Resolutions (Latitude × Longitude)
ACCESS1-0	1.25 × 1.875	AWI-CM-1-1-MR	0.9375 × 0.9375
ACCESS1-3	1.25 × 1.875	BCC-CSM2-MR	1.125 × 1.125
CMCC-CMS	3.7111 × 3.75	BCC-ESM1	2.8125 × 2.8125
CNRM-CM5	1.4008 × 1.40625	CAMS-CSM1-0	1.125 × 1.125
CanESM2	2.7906 × 2.8125	CESM2	0.9375 × 1.25
GFDL-ESM2G	2.0225 × 2	CESM2-WACCM	0.9375 × 1.25
GFDL-ESM2M	2.0225 × 2	CIESM	0.9375 × 1.25
GISS-E2-H	2 × 2.5	CMCC-CM2-SR5	0.9375 × 1.25
GISS-E2-R	2 × 2.5	CanESM5	2.8125 × 2.8125
HadGEM2-AO	1.25 × 1.875	E3SM-1-1	1 × 1
HadGEM2-CC	1.25 × 1.875	EC-EARTH3	0.703125 × 0.703125
HadGEM2-ES	1.25 × 1.875	EC-EARTH3-VEG	0.703125 × 0.703125
INM-CM4	1.5 × 2	FGOALS-f3-L	1 × 1.25
IPSL-CM5A-LR	1.8947 × 3.75	FIO-ESM-2-0	0.9375 × 1.25
IPSL-CM5A-MR	1.2676 × 2.5	GFDL-CM4	1 × 1.25
IPSL-CM5B-LR	1.8947 × 3.75	GFDL-ESM4	1 × 1.25
MIROC5	1.4008 × 1.40625	GISS-E2-1-G	2 × 2.5
MIROC-ESM	2.7906 × 2.8125	INM-CM4-8	1.5 × 2
MIROC-ESM-CHEM	2.7906 × 2.8125	INM-CM5-0	1.5 × 2
MPI-ESM-LR	1.8653 × 1.875	IPSL-CM6A-LR	1.25 × 2.5
MPI-ESM-MR	1.8653 × 1.875	KACE-1-0-G	1.25 × 1.875
MRI-CGCM3	1.12148 × 1.125	MIROC6	1.40625 × 1.40625
NorESM1-ME	1.8947 × 2.5	MRI-ESM2-0	1.125 × 1.125
NorESM1-M	1.8947 × 2.5	NorESM2-MM	0.9375 × 1.25
		SAM0-UNICON	0.9375 × 1.25
		TaiESM1	0.9375 × 1.25
		UKESM1-0-LL	1.25 × 1.875

. Horizontal and vertical resolutions differ for models and CMIPs. Hence, the horizontal resolutions of all models were identically interpolated into 2.5° × 2.5° grid boxes. Vertical pressure levels were consistently selected (i.e., 1000, 925, 850, 700, 600, 500, 400, 300, 250, and 200 hPa).

For comparison with reanalysis data, monthly ECMWF Reanalysis (ERA) interim data were employed [13]. The same horizontal interpolations and selection of vertical pressure levels were applied to the ERA-interim data. Because the available data period varies by CMIP models and observation, the analysis period was limited to TC seasons (i.e., June–October) of 1979–2005. Note that, for CMIP5, future experiments start from 2006 so we excluded the years after 2006 to directly compare CMIP5 models with CMIP6 models for the historical experiment only. In addition, the ERA-interim data cover the period from 1979 to the present. Meanwhile, the analysis period is 27 years, which is longer than the generally used analysis period for climate analyses in the IPCC 5th report. In calculating the climatological mean, the IPCC report utilized 20 years for the present (1986–2005) and future (2081–2100) climate, respectively [14].

The best-track data for TC locations and maximum wind speeds were obtained from the Joint Typhoon Warning Center. TCs passing through lines less than 300 km from the coastlines of South Korea were regarded as affecting South Korea [15,16]. Since the fast movement of TCs in the mid-latitudes, i.e., about 120 km per 6 h [17] may lead to missing several TCs affecting South Korea, 6-hourly original TC data were linearly interpolated into 1-hourly data in order to more accurately count the number of TCs striking South Korea [18]. Note that only TCs with maximum wind speed in excess of 17 m s⁻¹ were considered in this study. Figure 1 shows the seasonal mean occurrence frequency of TCs affecting South Korea. The seasonal mean occurrence frequency indicates how many times TCs appear in each 2.5° × 2.5° grid box over the TC seasons. For example, the maximum seasonal mean occurrence frequency shown in Figure 1 is 10, which means that TCs are observed in the grid point over 10 h during TC seasons on average.

PI can be calculated as the following equation [5].

$$\mathrm{PI}={\left(\frac{T_s}{T_0}\frac{C_k}{C_D}({\mathrm{CAPE}}^{*}-\mathrm{CAPE})\right)}^{\frac{1}{2}}\left[\mathrm{m}{\mathrm{s}}^{-1}\right]$$

(1)

where $c_p$ is the heat capacity at constant pressure, $T_s$ is the sea surface temperature, $T_0$ is the mean outflow temperature, $C_k$ is the exchange coefficient for enthalpy, $C_D$ is the drag coefficient, CAPE* is the convective available potential energy of saturation air and CAPE is that of boundary layer air. The fortran and MATLAB code for calculating PI is provided by the web site [19].

PI values simulated by CMIP models were first evaluated on a basin-wide scale, (i.e., the entire WNP defined as 100° E–180° and 0°–40° N) by using spatial biases, standardized deviations, and pattern correlations with the ERA-interim. TCs over the entire basin are hereafter referred to as WNP PI. In addition, PI over the grid points, where the seasonal mean occurrence frequency for South Korea is not zero, is called KOR PI hereafter.

3. Results

Spatial distributions of PI in the observation area are shown in Figure 2. PI is generally highest in the tropics (about 62 m s⁻¹) and almost linearly decreases as the latitude increases. Note that the PI spatial pattern generally follows SSTs, which was documented by [20]. Here, first, spatial biases from observed data for each CMIP5 and CMIP6 model’s PI were investigated (Figure 3 and Figure 4).

The largest biases are consistently shown near the Korean peninsula and the Japanese islands although their signs vary by models particularly near the Korean peninsula. In some models, positive biases prevail near the Korean coastal seas while negative biases in the others. In the subtropics, the negative biases are generally dominant in most of the models while in the tropics the signs of biases are opposite among the models. These kinds of characteristics are found regardless of CMIP phases. Comparing mean absolute biases of CMIP6 vs. CMIP5 models does not show significant differences for both WNP (average 4.25 m s⁻¹ vs. 4.23 m s⁻¹, p = 0.90) and KOR PI (average values 3.64 m s⁻¹ vs. 3.76 m s⁻¹, p = 0.25). Meanwhile, the averages of mean absolute biases are smaller for KOR PI than those for WNP PI in both of CMIP5 and CMIP6 models. This is natural because most of the large biases shown in the mid-latitudes are excluded when calculating KOR PI (see black lines in Figure 3 and Figure 4).

To compare the PI performances, a Taylor diagram analysis [21] of CMIP5 and CMIP6 models was performed (Figure 5). The Taylor diagram displays the normalized standard deviations (NSTD), pattern correlations (CORs), and root mean square errors (RMSEs) of each model once the model’s biases are removed [21]. Note that NSTDs are calculated from the models’ standard deviations divided by the observed standard deviation. In other words, a NSTD value of one means that the amplitude of spatial variability exactly corresponds to the observation. The NSTD value of one is shown by the dashed line in Figure 5, hereafter called the base line. As the theory of the Taylor diagram, the distance from the point where both NSTD and COR are one (marked as black dot) becomes the bias-removed RMSE that has been normalized by the observed standard deviation. The relative value of the RMSEs for each model can be checked by the distance from the black dot in Figure 5.

In terms of NSTD, COR, and RMSE, simulations of WNP PI are not noticeably improved when changing models from CMIP5 to CMIP6 (Figure 5). The averages of |NSTD-1|, COR, and RMSE in CMIP6 models are 0.10, 0.95, and 4.73, respectively, which are comparable those in CMIP5 models, 0.10, 0.93, and 5.40 (

Table 2. Average of absolute departures of NSTDs from the baseline (|NSTD-1|), CORs and bias-removed RMSEs for CMIP5 and CMIP6 models, and their differences. The p-values for the differences calculated using the Student’s t-test and Wilcoxon rank sum test are also listed. The single and double asterisks indicate statistical significance at the 90% and 95% confidence levels, respectively.

	WNP PI				KOR PI
	|NSTD-1|	COR	RMSE (Bias-Removed)		|NSTD-1|	COR	RMSE (Bias-Removed)
CMIP5	0.10	0.93	5.40		0.29	0.81	4.24
CMIP6	0.10	0.95	4.73		0.24	0.90	3.58
Difference (CMIP6 minus CMIP5)	0.00	0.02	−0.67		−0.05	0.09	−0.66
p-value (t-test)	0.899	0.053 *	0.058 *		0.245	0.011 **	0.019 **
p-value (rank sum test)	0.756	0.111	0.227		0.305	0.018 **	0.050 *

). Their differences between CMIP5 and CMIP6 models are not statistically significant at the 95% confidence levels with both of the Student’s t-test and the Wilcoxon rank sum test [22] although COR and RMSE are different at the 90% confidence levels with Student’s t-test. This marginal improvement might be related with the fact that PIs in the mid-latitude ocean are poorly simulated by both of CMIP5 and CMIP6 models. However, the model simulations of KOR PI are considerably improved in CMIP6 compared to CMIP5 in terms of COR and RMSE although not in NSTD (Figure 5). The average CORs and RMSEs of CMIP6 are 0.90 and 3.58 while those of CMIP5 models are 0.81 and 4.24, respectively (Table 2). They are significantly different at the 95% confidence levels although those of NSTD are not. Hence, PI simulation of CMIP6 is better than that of CMIP5 particularly in terms of KOR PI.

Meanwhile, all CMIP models better simulate WNP PI than KOR PI particularly in NSTD and COR (Figure 5). Most of the CMIP5 and CMIP6 models are located near 0.95 COR and the NSTD base line for WNP PI. However, for KOR PI, NSTDs are generally on the left of the base line in most of the CMIP5 and CMIP6 models. This corresponds to the strong positive biases in the vicinity of the Korean peninsula found in Figure 3 and Figure 4. CORs for KOR PI are also lower than those for WNP PI, although CORs are still large, between 0.9 and 0.95. On the other hand, RMSEs for KOR PI are smaller than those for WNP PI (Table 2), meaning that KOR PI is better simulated than WNP PI in terms of RMSEs. This is associated with larger absolute biases in the east of Japanese islands than those in the other oceans (Figure 3 and Figure 4). Since the area for WNP PI covers the Japanese islands and adjacent oceans, the RMSEs for WNP PI should be larger than those for KOR PI.

4. Conclusions

This study evaluated simulations of KOR and WNP PIs by using 25 CMIP5 and 27 CMIP6 models. In terms of the spatial biases and mean absolute biases of PI, CMIP6 models are not notably improved from CMIP5 models. Moreover, the largest biases near the Korean peninsula and the Japanese islands are consistently found in both of CMIP5 and CMIP6 models. In CMIP6 models, the simulation of KOR PI is significantly improved compared to CMIP5 models particularly in the aspects of COR and RMSE although that of WNP PI is not. This implies that the model simulation of PI particularly on the TC routes to South Korea is improved in CMIP6 models in general.

The better performance of CMIP6 over the East Asia in other aspects were also suggested by several previous studies as well. Kim et al. [23] showed that the extreme precipitation including East Asian regions are better modeled by CMIP6 models than CMIP5 models. In another study, the interannual variation of heat waves over South Korea is also better simulated by CMIP6 models than CMIP5 models [24]. The series of studies including this study suggest that the regional variation of climate extremes near South Korea can be simulated better in the CMIP6 models.