Skip to main content
Log in

Sparse factor model based on trend filtering

  • S.I. : Regression Methods based on OR techniques
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

This article has been updated

Abstract

We present a model based on trend filtering and regularization for performing factor analysis. Furthermore, the trend filtering method proposed in our model allows incorporating views represented as scenarios. Therefore, factor models can be optimized to explain not only trends of a given data series but also trends reflecting outlooks. As an application to finance, the proposed model constructs a sparse factor model on the trend of an index. Factor analyses on trend-filtered series provide factor models for describing trends, which are valuable for modeling long-term horizons. We also analyze the effect of trends on factor model selection in the US stock market. Further analyses include an investigation of factors during crisis periods and a comparison when various scenarios are considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Change history

  • 03 April 2021

    The article has been updated due to the word “oneword” erroneously used instead of “dataset” present in line 387 of this article.

Notes

  1. For the use of regularization in portfolio construction, see Brodie et al. (2009), Chen et al. (2020), and Ho et al. (2015).

  2. Empirical analyses in this section are performed using Matlab R2016b (64-bit) and optimization is solved with CVX Version 2.1.

  3. Factors (1)–(11) in Table 1 are retrieved from the Federal Reserve Economic Data (FRED), and industry data along with US stock market returns are collected from the data library of Kenneth R. French. Industry factors are included to represent various sectors of the stock market (Cavaglia et al., 2000).

  4. Note that, while key indicators are included in the list, this is not a comprehensive list or the ideal combination of market factors.

  5. While there is no general guideline for selecting the value of γ, Yamada and Yoon (2016) offer insight on selecting the tuning parameter based on the relation between H–P and \(\ell_{1}\) trend filtering methods.

References

  • Ang, A. (2014). Asset management: A systematic approach to factor investing. Oxford University Press.

    Book  Google Scholar 

  • Banz, R. W. (1981). The relationship between return and market value of common stocks. Journal of Financial Economics, 9(1), 3–18.

    Article  Google Scholar 

  • Baxter, M., & King, R. G. (1999). Measuring business cycles: Approximate band-pass filters for economic time series. Review of Economics and Statistics, 81(4), 575–593.

    Article  Google Scholar 

  • Bertocchi, M., Consigli, G., & Dempster, M. A. H. (2011). Stochastic optimization methods in finance and energy: New financial products and energy market strategies. Springer.

    Book  Google Scholar 

  • Brodie, J., Daubechies, I., De Mol, C., Giannone, D., & Loris, I. (2009). Sparse and stable Markowitz portfolios. Proceedings of the National Academy of Sciences, 106(30), 12267–12272.

    Article  Google Scholar 

  • Carhart, M. M. (1997). On persistence in mutual fund performance. Journal of Finance, 52(1), 57–82.

    Article  Google Scholar 

  • Cavaglia, S., Brightman, C., & Aked, M. (2000). The increasing importance of industry factors. Financial Analysts Journal, 56(5), 41–54.

    Article  Google Scholar 

  • Cerniglia, J., & Fabozzi, F. J. (2018). Academic, practitioner, and investor perspectives on factor investing. Journal of Portfolio Management, 44(4), 10–16.

    Article  Google Scholar 

  • Chen, J., Dai, G., & Zhang, N. (2020). An application of sparse-group lasso regularization to equity portfolio optimization and sector selection. Annals of Operations Research, 284(1), 243–262.

    Article  Google Scholar 

  • Consiglio, A., Cocco, F., & Zenios, S. A. (2007). Scenario optimization asset and liability modelling for individual investors. Annals of Operations Research, 152, 167–191.

    Article  Google Scholar 

  • Dimson, E., Marsh, P., & Staunton, M. (2017). Factor-based investing: The long-term evidence. Journal of Portfolio Management, 4(15), 15–37.

    Article  Google Scholar 

  • Dopfel, F. E., & Lester, A. (2018). Optimal blending of smart beta and multifactor portfolios. Journal of Portfolio Management, 44(4), 93–105.

    Article  Google Scholar 

  • Dupačová, J., Consigli, G., & Wallace, S. W. (2000). Scenarios for multistage stochastic programs. Annals of Operations Research, 100(1–4), 25–53.

    Article  Google Scholar 

  • Fama, E. F., & French, K. R. (1992). The cross-section of expected stock returns. Journal of Finance, 47(2), 426–465.

    Article  Google Scholar 

  • Fama, E. F., & French, K. R. (1993). Common risk factors in the returns on stocks and bonds. Journal of Financial Economics, 33(1), 3–56.

    Article  Google Scholar 

  • Fama, E. F., & French, K. R. (2015). A five-factor asset pricing model. Journal of Financial Economics, 116(1), 1–22.

    Article  Google Scholar 

  • Fama, E. F., & MacBeth, J. D. (1973). Risk, return, and equilibrium: Empirical tests. Journal of Political Economy, 81(3), 607–636.

    Article  Google Scholar 

  • Hodrick, R. J., & Prescott, E. C. (1997). Postwar U.S. business cycles: An empirical investigation. Journal of Money, Credit, and Banking, 29(1), 1–16.

    Article  Google Scholar 

  • Ho, M., Sun, Z., & Xin, J. (2015). Weighted elastic net penalized mean-variance portfolio design and computation. SIAM Journal on Financial Mathematics, 6(1), 1220–1244.

    Article  Google Scholar 

  • Kim, J. H., Kim, W. C., & Fabozzi, F. J. (2017). Robust factor-based investing. Journal of Portfolio Management, 43(5), 157–164.

    Article  Google Scholar 

  • Kim, S. J., Koh, K., Boyd, S., & Gorinevsky, D. (2009). L1 trend filtering. SIAM Review, 51(2), 339–360.

    Article  Google Scholar 

  • Kim, W. C., Kwon, D. G., Lee, Y., Kim, J. H., & Lin, C. (2020). Personalized goal-based investing via multi-stage stochastic goal programming. Quantitative Finance, 20(3), 515–526.

    Article  Google Scholar 

  • Kim, W. C., & Mulvey, J. M. (2009). Evaluating style investment—Does a fund market defined along equity styles add value? Quantitative Finance, 9(6), 637–651.

    Article  Google Scholar 

  • Köksalan, M., & Şakar, C. T. (2016). An interactive approach to stochastic programming-based portfolio optimization. Annals of Operations Research, 245(1–2), 47–66.

    Article  Google Scholar 

  • Kremer, P. J., Talmaciu, A., & Paterlini, S. (2018). Risk minimization in multi-factor portfolios: What is the best strategy? Annals of Operations Research, 266(1–2), 255–291.

    Article  Google Scholar 

  • Madhavan, A., Sobczyk, A., & Ang, A. (2018). What’s in your benchmark? A factor analysis of major market indexes. Journal of Portfolio Management, 44(4), 46–59.

    Article  Google Scholar 

  • Melas, D., Suryanarayanan, R., & Cavaglia, S. (2010). Efficient replication of factor returns: Theory and applications. Journal of Portfolio Management, 36(2), 39–51.

    Article  Google Scholar 

  • Mulvey, J. M., Hao, H., & Li, N. (2018). Machine learning, economic regimes and portfolio optimisation. International Journal of Financial Engineering and Risk Management, 2(4), 260–282.

    Article  Google Scholar 

  • Mulvey, J. M., & Liu, H. (2016). Identifying economic regimes: Reducing downside risks for university endowments and foundations. Journal of Portfolio Management, 43(1), 100–108.

    Article  Google Scholar 

  • Pedersen, A. M. B., Weissensteiner, A., & Poulsen, R. (2013). Financial planning for young households. Annals of Operations Research, 205(1), 55–76.

    Article  Google Scholar 

  • Ramdas, A., & Tibshirani, R. J. (2016). Fast and flexible ADMM algorithms for trend filtering. Journal of Computational and Graphical Statistics, 25(3), 839–858.

    Article  Google Scholar 

  • Roll, R., & Srivastava, A. (2018). Mimicking portfolios. Journal of Portfolio Management, 44(5), 21–35.

    Article  Google Scholar 

  • Tibshirani, R. J. (2014). Adaptive piecewise polynomial estimation via trend filtering. Annals of Statistics, 42(1), 285–323.

    Article  Google Scholar 

  • Vidal-García, J., Vidal, M., Boubaker, S., & Manita, R. (2019). Idiosyncratic risk and mutual fund performance. Annals of Operations Research, 281(1–2), 349–372.

    Article  Google Scholar 

  • Yamada, H. (2017). Estimating the trend in US real GDP using the \(\ell_{1}\) trend filtering. Applied Economics Letters, 24(10), 713–716.

    Google Scholar 

  • Yamada, H., & Jin, L. (2013). Japan’s output gap estimation and \(\ell_{1}\) trend filtering. Empirical Economics, 45(1), 81–88.

    Google Scholar 

  • Yamada, H., & Yoon, G. (2016). Selecting the tuning parameter of the \(\ell_{1}\) trend filter. Studies in Nonlinear Dynamics and Econometrics, 20(1), 97–105.

    Google Scholar 

Download references

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2018R1C1B6004271).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Frank J. Fabozzi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 Appendix 1. Property of trend filtering with scenarios

Property Suppose trend filtering results in trend \(z \in {\mathbb{R}}^{n}\) for a given data \(y \in {\mathbb{R}}^{n}\). Then, consider appending scenarios to the given data y. Trend filtering for y with scenarios will result in a trend that is different from z if the mean of the scenario values lies on the line that extends the final segment of the original trend z.

Proof We show the case for when the second-order difference matrix, D(2), is used for trend filtering and scenarios are added for a single period, but the argument can be extended to more general cases.

Suppose in addition to n observed data points, N scenarios for a single period is considered, which are denoted as

$$ s_{1} ,s_{2} , \ldots ,s_{N} . $$

Then, the length of the trend will be extended one period due to added scenarios for a single period (i.e., \(z \in {\mathbb{R}}^{n}\) becomes \(z \in {\mathbb{R}}^{n + 1}\) where \(z_{n + 1}\) reflects the added period), and the objective function will be extended to also include

$$ \left( {\mathop \sum \limits_{i = 1}^{N} \left( {s_{i} - z_{n + 1} } \right)^{2} } \right) + \left| {z_{n + 1} - 2z_{n} + z_{n - 1} } \right|. $$

If the trend z were not changed and the final line segment of the trend is extended one period, the following must hold,

$$ \begin{aligned} & z_{n + 1}^{*} - 2z_{n} + z_{n - 1} = 0 \\ & z_{n + 1}^{*} = 2z_{n} - z_{n - 1} . \\ \end{aligned} $$

Then, for any other \(z_{n + 1}^{0}\), it must be that

$$ \mathop \sum \limits_{i = 1}^{N} \left( {s_{i} - z_{n + 1}^{*} } \right)^{2} \le \left( {\mathop \sum \limits_{i = 1}^{N} \left( {s_{i} - z_{n + 1}^{0} } \right)^{2} } \right) + \left| {z_{n + 1}^{0} - 2z_{n} + z_{n - 1} } \right| $$

For large N, the right-hand side is minimized for \(z_{n + 1}^{0} \approx \frac{{\sum s_{i} }}{N}\).

Thus, the trend (at least the final segment of the trend) will be altered in general unless \(z_{n + 1}^{*}\) is purposely set to \(z_{n + 1}^{*} = 2z_{n} - z_{n - 1} \ne \frac{{\sum s_{i} }}{N}\).

1.2 Appendix 2. Two-step procedure for analyzing factors

The formulation given by (8) can be performed in two steps by optimizing the trend and then finding the factor model. For simplicity, we show that formulations (5) and (6) can be solved through (9) and (10).

$$ \begin{array}{*{20}l} {{\text{Step}}\;1{:}} \hfill & {\mathop {\min }\limits_{z} \frac{1}{2}\left\| {y - z} \right\|_{2}^{2} + \gamma \left\| {D^{\left( k \right)} z} \right\|_{1} } \hfill \\ \end{array} $$
(9)
$$ \begin{array}{*{20}l} {{\text{Step}}\;2{:}} \hfill & {\mathop {\min }\limits_{\beta } \left\| {z - X\beta } \right\|_{2}^{2} } \hfill \\ {} \hfill & {\left( {{\text{with}}\;{\text{regularization}}} \right){:}\; + \lambda_{1} \left\| \beta \right\|_{1} + \lambda_{2} \left\| \beta \right\|_{2}^{2} } \hfill \\ \end{array} $$
(10)

In formulation (5), small values of δ relative to γ will result in factors having minimal influence on the trend filtering part of the objective function. Therefore, the two-step procedure has the effect of solving the combined problem with a small value of δ. In other words, if we define \(\beta_{0}\) as the factor loadings from solving (10), then factor loadings from solving (5) with small values of δ should be the same as \(\beta_{0}\). This is clearly shown empirically in Fig. 

Fig. 6
figure 6

Differences in factor loadings for various values of δ

6, which plots the L2 distance between the factor loadings from the two-step procedure, \(\beta_{0}\), and the factor loadings from the original formulation with various values of δ. When δ is set to values smaller than 0.01, the distance becomes less than 0.0003 and the curve flattens. This is the case when the same dataset (1970–2018) from Sect. 4 is used, the value of γ is set to 0.5, and without regularization. Since the primary goal of the analyses we perform in Sect. 4 is to find a factor model of a trend, we use the two-step procedure to find the factor model after filtering the trend.

1.3 Appendix 3. Description of factors

Table

Table 8 Industry factor description

8 includes description of the industry factors, which are retrieved from the data library of Kenneth R. French. Table

Table 9 Summary statistics of factors

9 presents summary statistics and Table

Table 10 Correlation of factors

10 shows correlation of all factors that are used in the analyses from 1970 to 2018.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, J.H., Kim, W.C. & Fabozzi, F.J. Sparse factor model based on trend filtering. Ann Oper Res 306, 321–342 (2021). https://doi.org/10.1007/s10479-021-04029-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-021-04029-9

Keywords

Navigation