Skip to main content
SpringerLink
Log in

Efficient high-dimensional metamodeling strategy using recursive decomposition coupled with sequential sampling method

  • Research Paper
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Metamodel has been widely used to solve computationally expensive engineering problems, and there have been many studies on how to efficiently and accurately generate metamodels with limited number of samples. However, applications of these methods could be limited in high-dimensional problems since it is still challenging due to curse of dimensionality to generate accurate metamodels in high-dimensional design space. In this paper, recursive decomposition coupled with a sequential sampling method is proposed to identify latent decomposability and efficiently generate high-dimensional metamodels. Whenever a new sample is inserted, variable decomposition is repeatedly performed using interaction estimation from a full-dimension Kriging metamodel. The sampling strategy of the proposed method consists of two units: decomposition unit and accuracy improvement unit. Using the proposed method, latent decomposability of a function can be identified using reasonable number of samples, and a high-dimensional metamodel can be generated very efficiently and accurately using the identified decomposability. Numerical examples using both decomposable and indecomposable problems show that the proposed method shows reasonable decomposition results, and thus improves metamodel accuracy using similar number of samples compared with conventional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Allison JT, Kokkolaras M, Papalambros PY (2009) Optimal partitioning and coordination decisions in decomposition-based design optimization. J Mech Des 131:081008

    Google Scholar 

  • Bachoc F (2013) Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecification. Comput Stat Data Anal 66:55–69

    MathSciNet  MATH  Google Scholar 

  • Blanchet-Scalliet C, Helbert C, Ribaud M, Vial C (2019) Four algorithms to construct a sparse kriging kernel for dimensionality reduction. Comput Stat 34:1889–1909

    MathSciNet  MATH  Google Scholar 

  • Chen R-B, Wang W, Wu CJ (2010) Building surrogates with overcomplete bases in computer experiments with applications to bistable laser diodes. IIE Trans 43:39–53

    Google Scholar 

  • Chen Z, Peng S, Li X, Qiu H, Xiong H, Gao L, Li P (2015) An important boundary sampling method for reliability-based design optimization using kriging model. Struct Multidiscip Optim 52:55–70

    MathSciNet  Google Scholar 

  • Cho H, Choi K, Gaul NJ, Lee I, Lamb D, Gorsich D (2016) Conservative reliability-based design optimization method with insufficient input data. Struct Multidiscip Optim 54:1609–1630

    MathSciNet  Google Scholar 

  • Constantine PG, Dow E, Wang Q (2014) Active subspace methods in theory and practice: applications to kriging surfaces. SIAM J Sci Comput 36:A1500–A1524

    MathSciNet  MATH  Google Scholar 

  • Crombecq K, Gorissen D, Deschrijver D, Dhaene T (2011) A novel hybrid sequential design strategy for global surrogate modeling of computer experiments. SIAM J Sci Comput 33:1948–1974

    MathSciNet  MATH  Google Scholar 

  • Da Veiga S (2015) Global sensitivity analysis with dependence measures. J Stat Comput Simul 85:1283–1305

    MathSciNet  MATH  Google Scholar 

  • Dimov I, Georgieva R (2010) Monte Carlo algorithms for evaluating Sobol’sensitivity indices. Math Comput Simul 81:506–514

    MathSciNet  MATH  Google Scholar 

  • Dunteman GH (1989), Principal Components Analysis. Sage Publications

  • Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat:1189–1232

  • Friedman JH, Popescu BE (2008) Predictive learning via rule ensembles. Ann Appl Stat 2:916–954

    MathSciNet  MATH  Google Scholar 

  • Garbo A, German BJ (2020) A model-independent adaptive sequential sampling technique based on response nonlinearity estimation. Struct Multidiscip Optim 61:1051–1069

    MathSciNet  Google Scholar 

  • Gretton A, Bousquet O, Smola A, Schölkopf B (2005) Measuring statistical dependence with Hilbert-Schmidt norms, International conference on algorithmic learning theory, Springer, pp 63–77

  • Haftka RT, Adelman HM (1989) Recent developments in structural sensitivity analysis. Struct Optim 1:137–151

    Google Scholar 

  • Hajikolaei KH, Pirmoradi Z, Cheng GH, Wang GG (2015) Decomposition for large-scale global optimization based on quantified variable correlations uncovered by metamodelling. Eng Optim 47:429–452

    MathSciNet  Google Scholar 

  • Hajikolaei KH, Cheng GH, Wang GG (2016) Optimization on metamodeling-supported iterative decomposition. J Mech Des 138:021401

    Google Scholar 

  • Helton JC, Davis FJ (2003) Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliab Eng Syst Saf 81:23–69

    Google Scholar 

  • Hock W, Schittkowski K (1980) Test examples for nonlinear programming codes. J Optim Theory Appl 30:127–129

    MATH  Google Scholar 

  • Huang Z, Qiu H, Zhao M, Cai X, Gao L (2015) An adaptive SVR-HDMR model for approximating high dimensional problems. Eng Comput 32:643–667

    Google Scholar 

  • Iooss B, Lemaître P (2015) A review on global sensitivity analysis methods, Uncertainty management in simulation-optimization of complex systems, Springer, pp 101–122

  • Iott J, Haftka RT, Adelman HM (1985) Selecting step sizes in sensitivity analysis by finite differences, NASA, Technical Memorandum 86382

  • Jiang Z, Chen W, Fu Y, Yang R-J (2013) Reliability-based design optimization with model bias and data uncertainty. SAE Int J Mater Manuf 6:502–516

    Google Scholar 

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodelling techniques under multiple modelling criteria. Struct Multidiscip Optim 23:1–13

    Google Scholar 

  • Jin R, Chen W, Sudjianto A (2002) On sequential sampling for global metamodeling in engineering design, ASME 2002 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers, pp. 539–548

  • Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stati Plan Inf 26:131–148

    MathSciNet  Google Scholar 

  • Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492

    MathSciNet  MATH  Google Scholar 

  • Jung Y, Kang N, Lee I (2018) Modified augmented Lagrangian coordination and alternating direction method of multipliers with parallelization in non-hierarchical analytical target cascading. Struct Multidiscip Optim 58:555–573

    MathSciNet  Google Scholar 

  • Kang SB, Park JW, Lee I (2017) Accuracy improvement of the most probable point-based dimension reduction method using the hessian matrix. Int J Numer Methods Eng 111:203–217

    MathSciNet  Google Scholar 

  • Kang K, Qin C, Lee B, Lee I (2019) Modified screening-based Kriging method with cross validation and application to engineering design. Appl Math Model 70:626–642

    MathSciNet  MATH  Google Scholar 

  • Kim HM (2001) Target cascading in optimal system design. University of Michigan, Ann Arbor

    Google Scholar 

  • Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97:273–324

    MATH  Google Scholar 

  • Kucherenko S, Iooss B (2014) Derivative based global sensitivity measures, arXiv preprint arXiv:1412.2619

  • Kucherenko S, Sobol IM (2009) Derivative based global sensitivity measures and their link with global sensitivity indices. Math Comput Simul 79:3009–3017

    MathSciNet  MATH  Google Scholar 

  • Kucherenko S, Rodriguez-Fernandez M, Pantelides C, Shah N (2009) Monte Carlo evaluation of derivative-based global sensitivity measures. Reliab Eng Syst Saf 94:1135–1148

    Google Scholar 

  • Laguna M, Martí R (2005) Experimental testing of advanced scatter search designs for global optimization of multimodal functions. J Glob Optim 33:235–255

    MathSciNet  MATH  Google Scholar 

  • Lee I, Choi KK, Du L, Gorsich D (2008) Dimension reduction method for reliability-based robust design optimization. Comput Struct 86:1550–1562

    MATH  Google Scholar 

  • Lee I, Choi KK, Zhao L (2011) Sampling-based RBDO using the stochastic sensitivity analysis and dynamic Kriging method. Struct Multidiscip Optim 44:299–317

    MathSciNet  MATH  Google Scholar 

  • Lee K, Cho H, Lee I (2018) Variable selection using Gaussian process regression-based metrics for high-dimensional model approximation with limited data. Struct Multidiscip Optim 59:1439–1454

    MathSciNet  Google Scholar 

  • Li M, Wang Z (2019) Deep learning for high-dimensional reliability analysis. Mech Syst Signal Process 106399

  • Li G, Wang S-W, Rosenthal C, Rabitz H (2001) High dimensional model representations generated from low dimensional data samples. I. mp-Cut-HDMR. J Math Chem 30:1–30

    MathSciNet  MATH  Google Scholar 

  • Li G, Wang S-W, Rabitz H (2002) Practical approaches to construct RS-HDMR component functions. J Phys Chem A 106:8721–8733

    Google Scholar 

  • Li G, Hu J, Wang S-W, Georgopoulos PG, Schoendorf J, Rabitz H (2006) Random sampling-high dimensional model representation (RS-HDMR) and orthogonality of its different order component functions. J Phys Chem A 110:2474–2485

    Google Scholar 

  • Li E, Ye F, Wang H (2017) Alternative Kriging-HDMR optimization method with expected improvement sampling strategy. Eng Comput 34:1807–1828

    Google Scholar 

  • Liang H, Zhu M, Wu Z (2014) Using cross-validation to design trend function in Kriging surrogate modeling. AIAA J 52:2313–2327

    Google Scholar 

  • Liu H, Xu S, Ma Y, Chen X, Wang X (2016) An adaptive Bayesian sequential sampling approach for global metamodeling. J Mech Des 138:011404

    Google Scholar 

  • Liu H, Wang X, Xu S (2017) Generalized radial basis function-based high-dimensional model representation handling existing random data. J Mech Des 139:011404

    Google Scholar 

  • Lophaven SN, Nielsen HB, Søndergaard J (2002) DACE-A Matlab Kriging toolbox, version 2.0

  • Martin JD, Simpson TW (2004) On the use of kriging models to approximate deterministic computer models, ASME 2004 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers, pp. 481–492

  • Montgomery DC (2017) Design and analysis of experiments. John wiley & sons.

  • Morris MD (1991) Factorial sampling plans for preliminary computational experiments. Technometrics 33:161–174

    Google Scholar 

  • Park J-S (1994) Optimal Latin-hypercube designs for computer experiments. J Stat Plan Inf 39:95–111

    MathSciNet  MATH  Google Scholar 

  • Qazi M-u-D, He L, Mateen P (2007) Hammersley sampling and support-vector-regression-driven launch vehicle design. Journal of Spacecraft and Rockets 44:1094–1106

    Google Scholar 

  • Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker PK (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41:1–28

    Google Scholar 

  • Rasmussen CE, Williams CK (2006) Gaussian processes for machine learning, MIT press

  • Salem MB, Bachoc F, Roustant O, Gamboa F, Tomaso L (2019) Sequential dimension reduction for learning features of expensive black-box functions, working paper or preprint, February 2019. URL https://hal.archives-ouvertes.fr/hal-01688329

  • Saltelli A (2002) Making best use of model evaluations to compute sensitivity indices. Comput Phys Commun 145:280–297

    MATH  Google Scholar 

  • Schittkowski K (2012) More test examples for nonlinear programming codes. Springer Science & Business Media, Vol. 282

  • Schonlau M, Welch WJ (2006) Screening the input variables to a computer model via analysis of variance and visualization In Screening Methods for Experimentation and Industry Drug Discovery and Genetics (A. M. Dean and S. M. Lewis, eds.), Springer, pp 308–327

  • Shan S, Wang GG (2010a) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41:219–241

    MathSciNet  MATH  Google Scholar 

  • Shan S, Wang GG (2010b) Metamodeling for high dimensional simulation-based design problems. J Mech Des 132:051009

    Google Scholar 

  • Simpson TW, Toropov V, Balabanov V, Viana FA (2008) Design and analysis of computer experiments in multidisciplinary design optimization: a review of how far we have come or not, 12th AIAA/ISSMO multidisciplinary analysis and optimization conference, pp. 10–12

  • Sobol IM (1993) Sensitivity estimates for nonlinear mathematical models. Math Model Comput Exp 1:407–414

    MathSciNet  MATH  Google Scholar 

  • Sobol IM (2003) Theorems and examples on high dimensional model representation. Reliabi Eng Syst Saf 79:187–193

    Google Scholar 

  • Toal DJ, Bressloff NW, Keane AJ (2008) Kriging hyperparameter tuning strategies. AIAA J 46:1240–1252

    Google Scholar 

  • van der Herten J, Couckuyt I, Deschrijver D, Dhaene T (2015) A fuzzy hybrid sequential design strategy for global surrogate modeling of high-dimensional computer experiments. SIAM J Sci Comput 37:A1020–A1039

    MathSciNet  MATH  Google Scholar 

  • Wang GG, Shan S (2004) Design space reduction for multi-objective optimization and robust design optimization problems, SAE Technical Paper

  • Wang H, Tang L, Li G (2011) Adaptive MLS-HDMR metamodeling techniques for high dimensional problems. Expert Syst Appl 38:14117–14126

    Google Scholar 

  • Wang Z, Hutter F, Zoghi M, Matheson D, de Feitas N (2016) Bayesian optimization in a billion dimensions via random embeddings. J Artif Intell Res 55:361–387

    MathSciNet  MATH  Google Scholar 

  • Xu H, Rahman S (2004) A generalized dimension-reduction method for multidimensional integration in stochastic mechanics. Int J Numer Methods Eng 61:1992–2019

    MATH  Google Scholar 

  • Yamada M, Jitkrittum W, Sigal L, Xing EP, Sugiyama M (2014) High-dimensional feature selection by feature-wise kernelized lasso. Neural Comput 26:185–207

    MathSciNet  MATH  Google Scholar 

  • Zhang Q, Chen D (2005) A model for the low cycle fatigue life prediction of discontinuously reinforced MMCs. Int J Fatigue 27:417–427

    MATH  Google Scholar 

  • Zhao L, Choi KK, Lee I (2011) Metamodeling method using dynamic kriging for design optimization. AIAA J 49:2034–2046

    Google Scholar 

  • Zhao L, Wang P, Song B, Wang X, Dong H (2020). An efficient kriging modeling method for highdimensional design problems based on maximal information coefficient. Struct Multidiscip Optim 61:39–57

Download references

Funding

This research was supported by Energy Cloud R&D Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT (No. 2016006843).

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South Korea

    Kyeonghwan Kang & Ikjin Lee

Authors
  1. Kyeonghwan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ikjin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ikjin Lee.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Replication of results

Matlab codes for the proposed method and illustration using 3D example are available at “https://drive.google.com/file/d/1r9lF5ECY_TLGSeAe7jFFo8HIo64HKI_C/view?usp=sharing”. In addition, DACE Kriging toolbox from “http://www2.imm.dtu.dk/-pubdb/views/publication_details.php?id=1460” (Lophaven et al. 2002) and GPML toolbox from “http://www.gaussianprocess.org/gpml/code/matlab/doc/” (Rasmussen and Williams 2006) should be included in the same path to run this code.

Additional information

Responsible Editor: Shapour Azarm

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, K., Lee, I. Efficient high-dimensional metamodeling strategy using recursive decomposition coupled with sequential sampling method. Struct Multidisc Optim 63, 375–390 (2021). https://doi.org/10.1007/s00158-020-02705-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-020-02705-1

Keywords

Search

Navigation