Forecasting Japanese inflation with a news-based leading indicator of economic activities

2.1 Text data: the Nikkei and the Economy Watchers Survey

To construct the daily index of real economic activities, we utilize daily news articles in the Nikkei, the leading economic newspaper in Japan, with a circulation of 2.8 million as of June 2018. The Nikkei is a newspaper that specializes in financial, business, and industry sectors. We use news articles, from both morning and evening papers from April 1, 1989 to December 31, 2017. Table 1 shows the summary statistics of our text data from the Nikkei. Our main data consists of about 3.8 million articles with more than 30 million sentences in total. All texts in our data are written in Japanese. Unlike English texts, Japanese texts do not have spaces between words. Hence, we split sentences into words in advance using MeCab, which is a morphological analysis library for Japanese texts.^[7] After we divide articles into individual sentences, we estimate the news sentiment score on a sentence basis because our training data set, the Economy Watchers Survey, provides annotated scores on a sentence basis.

In what follows, we use a machine learning method to learn text classification models from our training data set for the purpose of building two types of news-based business cycle indexes, called the News-based Coincident Indicator (hereafter, NCI) and the News-based Leading Indicator (hereafter, NLI). NCI and NLI are designed to capture, on a daily basis, current and future economic conditions, respectively. We suppose that the sentences in the Nikkei on Japanese economic activities have the same (and time-invariant) structure as those in the Economy Watchers Survey. In other words, from the view point of natural language processing (NLP), the domain of our training data set, the Economy Watchers Survey, is same as that of our input data of the Nikkei. Figure 1 shows an outline of the procedure we use to construct our business cycle indexes. In the first step, we train text classification models from the Economy Watchers Survey with supervised learning. In the second step, we estimate the sentiment scores of sentences in the Nikkei articles using trained models. In the last step, we aggregate news sentiment scores to build news-based business cycle indexes.

Figure 1:

Outline of the procedure.

To train our model, we utilize descriptions for the assessment of the economy in the Economy Watchers Survey, which is published by Cabinet Office of the government of Japan.^[8] The purpose of this monthly survey is to promptly grasp current and future economic conditions by consulting people involved in regional economy activity. The assessment of current conditions refers to the direction of change in the economic conditions compared to previous conditions from three months before on a five-point scale. Similarly, the assessment of future conditions refers to the expected direction of change within the next two or three months on a five-point scale. In the Economy Watchers Survey, an assessment by each respondent is accompanied by his or her description. Our supervised learning utilizes descriptions as input data and the assessments as output labels. Table 2 shows selected example descriptions of the training data set from the Economy Watchers Survey. We use the survey data from January 2011 to June 2018. The number of total sentences in descriptions for the assessment of current and future economic conditions are 112,214 and 125,055, respectively. Table 3 shows the breakdown of descriptions in the training data set. Descriptions for the assessment of current economic conditions tend to use the present tense and the present progressive tense. On the other hand, descriptions for the assessment of future economic conditions include auxiliary verbs related to the future tense, such as “will,” and words to anticipate the future, such as “expect.”

2.2 Text classification model

In economic applications of sentiment analysis, the two most frequently used approaches are the dictionary-based approach and the machine learning approach. The advantage of the dictionary-based approach is its tractability and an easy implementation. The method quantifies text sentiments by simply counting the number of positive and negative words using a pre-defined dictionary. This approach, however, can fail to judge whether a sentence has good or bad information because word meaning often depends on the context. For example, the sentence, “The firm’s performance is not good,” can wrongly be taken as a good signal on economic conditions because it contains the word “good.” Therefore, it takes considerable time to incorporate all the possible patterns into the pre-defined dictionary. In addition, no established Japanese sentiment dictionary specializing in economic fields is available in the literature. Unlike the dictionary-based approach, the machine learning approach automatically recognizes patterns from the text data (input) and annotated scores (output). Among machine learning methods, neural networks in recent years have achieved high performance in text classification tasks. They excel in utilizing syntax information, such as sequence alignment and word co-occurrence.

Over the years, various neural network-based models have been developed in the field of computer science, including the recurrent neural network (Chung et al. 2014), the recursive neural network (Socher et al. 2013), the convolutional neural network (CNN, Kim 2014; Zhang, Zhao, and LeCun 2015) and the self-attention network (Lin et al. 2017). In our analysis, we use a text classification model based on neural networks called fastText, which was developed by Joulin et al. (2017).^[9] According to Joulin et al. (2017), fastText is a simple and computationally efficient network architecture that at the same time performs as well as classifiers based on other neural network models, such as the CNN and long short-term memory (LSTM, Hochreiter and Schmidhuber 1997) in terms of accuracy. Joulin et al. (2017) compared the classification accuracy of test data sets using eight corpora in comparison with six models, including three (multinomial) logistic regression-based models and three neural network-based models: the character-level CNN (char-CNN) of Zhang, Zhao, and LeCun (2015), the character-level convolutional recurrent neural network (char-CRNN) of Xiao and Cho (2016) and the very deep convolutional neural network (VDCNN) of Conneau et al. (2016). In the panel (a) of Table 4, we summarize the performance of fastText and other competing models from the experiment of Joulin et al. (2017). In addition, we also use our training data set, the Economy Watchers Survey, and compare the binary classification accuracy of fastText with those of LSTM, CNN, char-CNN and support vector machine (SVM). In this experiment, the binary classification task is classifying texts into two topic classes, current and future economic conditions. The results of our own experiment in terms of the accuracy of the test data set are reported in panel (b) of Table 4.^[10] Based on the results of experiments conducted by Joulin et al. (2017) and by ourselves, it seems fair to say that the performance of fastText is comparable to alternative machine learning models.

Finally, in terms of computation time, Joulin et al. (2017) report that training and evaluation of sentiment analysis data sets using fastText are many orders of magnitude faster than char-CNN and VDCNN. In our own experiment with the Economy Watchers Survey, we also confirm that fastText can be trained much more quickly than other models. Therefore, the method seems to be effective in assigning sentiment scores to our large-scale news article data set without losing much of accuracy.

2.3 Estimation and aggregation

In order to compute NCI and NLI from sentences in the Nikkei, we use three types of learners based on fastText, two of which are regression learners and the other, a classification learner. The first learner (learner 1) is trained from the assessment of the current economy on a five-point scale and its descriptions from the training data set, the Economy Watchers Survey. This learner assigns continuous sentiment scores, Score-CI, to sentences in the Nikkei. The second learner (learner 2) is trained from the assessment of the future economy on a five-point scale and its descriptions. This learner assigns continuous sentiment scores, Score-LI. Higher Score-CI and Score-LI indicate that texts contain information about better current and future economic conditions, respectively. The third learner (learner 3) is trained using assessments of both the current and future economic conditions as well as their descriptions. This learner assigns topic probabilities, W, to sentences in the Nikkei. We regard outputs from a sigmoid function as topic probabilities. The topic probability W takes a value near zero if a sentence is similar to descriptions for the assessment of the current economy, and takes a value near one if a sentence is similar to descriptions for the assessment of the future economy. Table 5 summarizes three types of our learners.

We give sentiment scores and topic probabilities to all sentences in news articles with three learners.^[11] When a trained model assigns a sentiment score to each sentence in news articles, out-of-vocabulary words (words or tokens not included in a training data set) are replaced by a common special character, such as <UNKNOWN>. Sentences in news articles longer than the longest sentence in the training data set are truncated. In general, even if exactly the same neural network model is employed, slightly different outputs can be obtained on each run depending on initialization. Therefore, we train the models 10 times, and use average scores from all cases.

In the next step, we construct two news-based business cycle indexes using the average of scores weighted by the topic probabilities. In particular, NCI and NLI are respectively defined by

where N _t is the number of sentences in day t, Score-CI _t,k is a score of kth sentence assigned by learner 1 in day t, Score-LI _t,k is a score of kth sentence assigned by learner 2 in day t and W _t,k is a topic probability of kth sentence assigned by learner 3 in day t. Figure 2 plots our constructed news-based business cycle indexes, NCI and NLI. Table 6 shows their summary statistics. Overall, two indicators tend to comove. However, there are some differences between the two, reflecting the fact that the NCI captures the current economic conditions while the NLI captures future economic conditions. In particular, the median of the NLI seems to be higher than that of the NCI, which may indicate that the typical newspaper article tends to be relatively optimistic about future economic conditions. This tendency becomes clearer during the post-financial crisis period.

Figure 2:

News-based business cycle indexes.

For the purpose of comparing our new business cycle indexes with other indicators of economic activity, we converted the daily series into a monthly series by using monthly averages. Table 7 reports the correlations of our business cycle indexes and official business cycle indicators, namely two diffusion indexes (DIs) from the summary results of the Economy Watchers Survey and three composite indexes (CIs) of business conditions from the Economic and Social Research Institute (ESRI) of the Cabinet Office. Both our business cycle indexes, the NCI and the NLI, turn out to be highly correlated with the current and future DIs in the Economy Watchers Survey. This outcome may not be very surprising given the fact that our indexes are calculated from the same survey information as the DIs in the Economy Watchers Survey. However, among three official CIs, namely, (i) the leading index, (ii) the coincident index, and (iii) the lagging index, the NLI is highly correlated with the leading index, which is computed without using newspaper articles or the Economy Watchers Survey. Figure 3 plots NCI, NLI, and ESRI’s leading index along with official recession episodes. The figure shows that both the NCI and the NLI tend to decrease during economic downturns. In particular, all indexes clearly dropped during the financial crisis of 2008. Overall, it seems fair to say that our news-based business cycle indexes capture well the business cycle properties of the Japanese economy.^[12] In the following sections, we mainly focus on using the NLI in our simulated out-of-sample forecasting exercise of daily inflation series.

Figure 3:

News-based business cycle indexes and official business cycle index.

Note: All the series are normalized to have zero mean and unit variance. news-based coincident indicator (NCI) and news-based leading indicator (NLI) are news-based business cycle indexes. The leading index is from the composite indexes of business conditions by economic and social research institute (ESRI), the Cabinet Office. The shaded area shows the official recession episodes of ESRI, the Cabinet Office.

3.1 Phillips curve inflation forecast

Motivated by the well-known Phillips curve model, which describes the correlation between the inflation and unemployment rate, Stock and Watson (1999) conducted a simulated out-of-sample forecasting analysis of US inflation at the 12-month horizon. They claimed that the inflation forecast could be improved by replacing the unemployment rate with a single real economic activity index, especially with one constructed from 168 economic indicators. Atkeson and Ohanian (2001) reviewed the literature of the 1990s and challenged the belief that the conventional Phillips curve model is useful tool for inflation forecasting. They revisited the forecast performance of the Phillips curve model adopted in Stock and Watson (1999) and claimed that their naive forecasts outperformed the Phillips curve forecast. Later, Stock and Watson (2009) provided a comprehensive literature review on inflation forecasting and pointed out that the results against the Phillips curve forecast obtained by Atkeson and Ohanian (2001) could disappear, depending on the forecast horizon and sample period.^[13] In summary, it seems fair to say that no consensus has been reached regarding the validity of the Phillips curve relationship in forecasting inflation. However, almost all the existing studies utilized monthly or quarterly data to examine the performance of the Phillips curve inflation forecast. Here, we use the daily data to evaluate the usefulness of the Phillips curve in forecasting Japanese inflation.

In our analysis, we use a Phillips curve model similar to the one considered by Stock and Watson (1999), who replaced unemployment rate with a single real economic activity index. In particular, we consider the daily Japanese inflation series called Nikkei CPINow and investigate its relationship with our news-based business cycle index. Nikkei CPINow is known for the first real time price index in Japan. NOWCAST, Inc. releases two CPINow series called the CPINow-T index and the CPINow-S index. The CPINow-T index is a daily series calculated from POS data and has been available since April 1, 1989. More than 800 stores and 300,000 different products such as food and daily necessities items are covered in the survey. Unlike the official CPI, shares of the items are taken into account everyday by taking the advantage of the POS data. On the other hand, the CPINow-S index is a monthly series designed to closely match the official CPI by selecting the same representative items and by using the same index formula. The CPINow-S index has been available since January 2015.

The summary statistics of the two CPINow series are provided in Table 8.^[14] Table 9 shows the correlations between the two CPINow series and the official CPI at a monthly frequency. Here, the daily series of the CPINow-T index series is transformed by using the monthly average. In the table, a remarkably high correlation stands out between the CPINow-S index and the official CPI for all items less fresh food and energy. This result suggests that the CPINow series can serve as a good proxy for the official CPI at a higher frequency. Since the CPINow-T index is released two days after actual transactions, it is also useful for nowcasting purposes. A direct comparison between CPINow-T index and official CPI inflation series is provided in Figure 4.

Figure 4:

Inflation based on the CPINow-T index and official consumer price index (CPI).

In what follows, we transform the CPINow-T index to construct 1- to 12-month inflation at an annual rate, which is the target variable in our forecasting analysis. Following the previous studies, including Atkeson and Ohanian (2001) and Stock and Watson (1999, 2009), our target variable is m-period inflation defined by

where m is a window size and π t is the CPINow-T index series in the form of the daily inflation rate at an annual rate. For the forecast horizon h, we consider approximately one month to one year by setting h = 30 × k for k = 1, 2, … , 12.

The h-period ahead forecast is constructed from the Phillips curve model given by

where NLI _t is our news-based leading indicator (2) designed to capture the future economic conditions, ϕ h ( L ) = ∑ ​ j = 1 h ϕ j L j − 1 and e _t + h is the forecast error. It should be noted that if β = 0 , (4) reduces to an AR(h) forecast with no restriction on the AR coefficients. In our exercise, we consider an AR model as a benchmark model and investigate whether the Phillips curve model combined with the news-based leading indicator can outperform the benchmark model. However, if no restriction is imposed on AR coefficients, the number of unknown parameters in the AR(h) model can be as large as 360. To avoid the issue of overfitting with too many parameters in the AR model, we consider several parsimonious specifications.

First, as in Atkeson and Ohanian (2001), a naive forecast of h-period inflation can be constructed using the current value of h-period inflation or ϕ h ( L ) π t = π t h = ( 1 / h ) ∑ ​ j = 0 h − 1 π t − j . It should be noted that this specification can be obtained as the h-period moving average of the random walk forecast and only α needs to be estimated. Since it imposes a non-stationary (NS) restriction on AR coefficients, we refer this specification to AR-NS. Second, we can combine several values of m-period inflation with different window size m and use

For example, in this specification, a one-month ahead inflation forecast is computed by combining daily inflation, weekly inflation and monthly inflation. A similar parsimonious specification has also been employed by Ito and Yabu (2007) and Fatum and Hutchison (2010) in their government intervention analysis of daily foreign exchange rates, and by Corsi (2009) in his forecasting analysis of realized volatilities.^[15] Since the regressors are inflation series in mixed-frequencies (MF), we refer this specification to AR-MF. Third, the lag length can be selected using information criteria such as AIC and BIC and we refer these specifications to AR-AIC and AR-BIC, respectively. Using the simulated out-of-sample forecasting methodology explained below, we select a benchmark AR forecast from these alternative specifications of AR coefficients (namely, AR(h), AR-NS, AR-MF, AR-AIC, and AR-BIC).

3.2 Simulated out-of-sample forecasting

We use the simulated out-of-sample forecasting methodology in evaluating the forecasting model. In this approach, out-of-sample forecasts are computed as if a real-time forecaster were estimating the model using only the data available at the time of the past forecast. In particular, by using the sample only through the period t, the h-period ahead forecast of inflation, π t + h | t h is obtained. We then compare the forecast value π t + h | t h with a realized value π t + h h to compute the forecast error at the period t + h, namely, e ˆ t + h . Next, we follow the same procedure by using the sample through the period t + 1, estimate the model and compute e ˆ t + h + 1 . We repeat this process P − h + 1 times to obtain P − h + 1 forecast errors.^[16] Then, the MSE of h-period ahead inflation forecast, defined as σ 2 = E ( e t + h 2 ) , can be estimated by σ ˆ 2 = ( P − h + 1 ) − 1 ∑ t = R P + R − h e ˆ t + h 2 where R is the sample size used to estimate the forecast model at the beginning of the forecast evaluation.

We consider two estimation schemes to evaluate the forecasting performance. The first is the rolling scheme where the model is estimated using a moving data window of the length R. The second is the recursive scheme where the data with an increasing number of observations is used each time the new model is estimated. In the recursive scheme, R represents the sample size used in the initial step. For the sake of robustness, we conduct simulated out-of-sample forecasting experiments with P/R = 0.4 and 1.0. The exact numbers of P and R are selected using the identity P + R = T − h + 1 where T, the full sample size, is 10,413.

We are interested in determining if the MSE of the Phillips curve model is less than the benchmark AR models without using the news-based leading indicator. In the first step, we investigate an appropriate benchmark AR model by estimating the MSE of inflation forecast using the simulated out-of-sample forecasting methodology described above. The results of the MSE estimates for various forecast horizons h are summarized in Table 10. The performance of AR-NS is uniformly worse than that of an unrestricted AR(h) model. The AR-MF specification performs almost the same as an unrestricted AR(h) model. Both AR-AIC and AR-BIC perform well in shorter horizons but not in longer horizons. On balance, we select AR-MF as our preferred specification for the benchmark AR model. Once the benchmark model is selected, we can estimate the Phillips curve model (4) in the next step. For the case when the full sample period from April 1, 1989 to December 31, 2017 is used, estimated coefficients are provided in Table 11. This table demonstrates that when the forecast horizon h becomes 120 or longer, coefficients on the news-based leading indicator (β) become positive and significant. The table also reports the estimates of the sum of AR coefficients, ϕ h ( 1 ) = ∑ ​ j = 1 h ϕ j . The sum of AR coefficients tends to be decreasing with the forecast horizon h, suggesting the stationarity of inflation in Japan. In summary, for four-month to one-year horizons, signs of the coefficients turn out to be consistent with the notion of the Phillips curve.

The benchmark AR model is clearly nested by the Phillips curve model (4). Since the forecasting model of our interest nests the benchmark model, we use the out-of-sample F type test statistic proposed in McCracken (2007), which is designed to compare the forecasting performance of nested models. The out-of-sample F type test statistic is defined as

where σ ˆ A R 2 is the estimator of the MSE of the AR inflation forecast σ A R 2 and σ ˆ P C 2 is the estimator of the MSE of the Phillips curve inflation forecast σ P C 2 .

We use this statistic to test for the null hypothesis of H ₀: σ A R 2 / σ P C 2 = 1 against an alternative hypothesis of H ₁: σ A R 2 / σ P C 2 > 1 . McCracken (2007) shows that when h = 1, the F statistic asymptotically follows non-standard distribution under the null hypothesis. He provides the critical values which depend on P/R and the difference in the number of regressors. In general, for the case of a longer forecast horizon h > 1, the distribution becomes data dependent. However, for the case when the number of additional regressors is exactly one, as in our setting, the critical values in McCracken (2007) can still be valid (see West (2006) for this point more in detail). We compute the relative size of MSEs of the benchmark AR model and the Phillips curve model, then conduct the out-of-sample F-test for various h.

Table 12 shows the relative MSE given by σ ˆ A R 2 / σ ˆ P C 2 with out-of-sample F-test statistics in parenthesis. Note that the sample period in the analysis depends on forecast horizon (h). For example, the sampling period for estimation in the first step of the rolling scheme with P/R = 1 for h = 360 is from June 29, 1989 to February 19, 2010. In this case, both R and P are 2873.

Based on the point estimates of MSEs, the Phillips curve forecast performs better than the AR forecast when h is 120 and longer for the rolling scheme and when h is 90 and longer for the recursive scheme. Furthermore, the results of the out-of-sample F-test imply that when the forecasting horizon becomes longer, the estimate of σ P C 2 becomes significantly less than that of σ A R 2 . These results confirm that the Phillips curve forecast outperforms the AR forecast at least for horizons greater than three months. The cumulative squared prediction error differences of two models for h = 120, 240 and 360 are also shown in Figure 5. This figure implies that the improvements are clearly observed in both in the beginning and after 2016 in the evaluation period.^[17] The significant reduction of the MSE does not change among the out-of-sample simulation designs with P/R = 0.4 and 1.0. Our finding that the news-based leading indicator contains valuable information about future inflation is consistent with the fact that the assessment of future economic conditions refers to the 2–3 months ahead projected status of the Japanese economy in the Economy Watchers Survey.

Figure 5:

Cumulative squared prediction error differences between the AR and the Phillips curve forecasts.

Note: Mean squared errors (MSEs) are estimated by the rolling scheme and P/R = 0.4.