Abstract
Complete modified stacked denoising auto-encoder (CMSDAE) machines constitute a version of stacked auto-encoders in which a target estimate is included at the input, and are trained layer-by-layer by minimizing a convex combination of the errors corresponding to the input sample and the target. This permits to carry out the transformation of the observation space without forgetting what the target is. It has been shown in recent publications that this method produces a clear performance advantage in classification tasks. The above facts motivate to explore whether CMSDAE machines also offer performance improvements in regression problems, and in particular for time series prediction where conventional discriminative machines find difficulties: The layer-by-layer reconstruction of the target (together with the input) can help to reduce these difficulties. This contribution presents the CMSDAE regression/prediction machines and their design, showing experimental evidence of their frequent superior performance —never lower— with respect to other machine architectures. Some subsequent research directions are indicated together with the conclusions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bourlard H, Kamp Y (1988) Auto-association by multilayer perceptrons and singular value decomposition. Biol Cybern 59:291–294
Vincent P, Larochelle H, Lajoie I, Bengio Y, Manzagol PA (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408
Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35:1798–1828
Najafabadi M, Villanustre F, Khoshgoftaar T, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2:1–9
Brahma P, Wu D, She Y (2015) Why deep learning works: a manifold disentanglement perspective. IEEE Trans Neural Netw Learn Syst 27:1997–2008
Yu J, Hong C, Rui Y, Tao D (2018) Multitask autoencoder model for recovering human poses. IEEE Trans Ind Electron 65(6):5060–5068
Dörner S, Cammerer S, Hoydis J, ten Brink S (2018) Deep learning based communication over the air. IEEE J Selected Topics Signal Process 12(1):132–143
Mohammadi M, Al-Fuqaha A, Guizani M, Oh J (2018) Semisupervised deep reinforcement learning in support of IoT and smart city services. IEEE Internet of Things J 5(2):624–635
Yu J, Yang X, Gao F, Tao D (2017) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 47(12):4014–4024
Ge H, Sun W, Zhao M, Yao Y (2019) Stacked denoising extreme learning machine autoencoder based on graph embedding for feature representation. IEEE Access 7:13433–13444
Pang S, Yang X (2019) A cross-domain stacked denoising autoencoders for rotating machinery fault diagnosis under different working conditions. IEEE Access 7:77277–77292
Chen J, Li J, Chen W, Wang Y, Jiang T (2020) Anomaly detection for wind turbines based on the reconstruction of condition parameters using stacked denoising autoencoders. Renewable Energy 147:1469–1480
Dai T, Zhu L, Wang Y, Carley KM (2020) Attentive stacked denoising autoencoder with Bi-LSTM for personalized context-aware citation recommendation. IEEE-ACM Trans Audio Speech Lang 28:553–568
Ni Q, Fan Z, Zhang L, Nugent CD, Cleland I, Zhang Y, Zhou N (2020) Leveraging wearable sensors for human daily activity recognition with stacked denoising autoencoders. Sensors 20(18):5114
Alvear-Sandoval RF, Figueiras-Vidal AR (2018) On building ensembles of stacked denoising auto-encoding classifiers and their further improvement. Inf Fusion 39:41–52
Sánchez-Morales A, Sancho-Gómez JL, Figueiras-Vidal AR (2019) Exploiting label information to improve auto-encoding based classifiers. Neurocomputing 370:104–108
Sánchez-Morales A, Sancho-Gómez JL, Figueiras-Vidal AR (2020) Complete autoencoders for classification with missing values. Neural Comput Appl. https://doi.org/10.1007/s00521-020-05066-4
Dua D, Graff C (2017) UCI Machine Learning Repository. School of Information and Computer Sci., Univ. California at Irvine. http://archive.ics.uci.edu/ml
Clark D, Schreter Z, Adams A (1996) A quantitative comparison of dystal and backpropagation. In: Proceedings of the 7th Australian Conference Neural Networks, The Australian National University, pp 132–137
Yeh IC (1998) Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete Res 28(12):1797–1808
Delve database (1995-1996) Univ. Toronto. https://www.dcc.fc.up.pt/~ltorgo/Regression/DataSets.html, (Accessed: 2019-09-19)
Ballester-Ripoll R, Paredes EG, Pajarola R (2019) Sobol tensor trains for global sensitivity analysis. Reliab Eng Syst Safety 183:311–322
De Vito S, Massera E, Piga M, Martinotto L, Di Francia G (2008) On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens Actuators B: Chem 129:750–757
Szeged Weather database (2016) Kaggle. https://www.kaggle.com/budincsevity/szeged-weather, (Accessed: 2019-09-19)
Fernández-García ME, Larrey-Ruiz J, Ros-Ros A, Figueiras-Vidal AR, Sancho-Gómez JL (2019) Machine-health application based on machine learning techniques for prediction of valve wear in a manufacturing plant. In: Proceedings of international work-conference interplay between natural and artificial computation, (IWINAC) (LNCS 11487), Springer, pp 389–398
Acknowledgements
Thanks to ILBOC, and Fundación Séneca (Program 20348/FPI/17 and Project 20901/PI/18) and Instituto de Salud Carlos III (Project 2018-PI17-00771) for supporting and funding this research work. This work has been carried out under the cooperation scheme “MAPAS” (TIN2017-90567-REDT, MINECO/FEDER EU). The work of ARFV was supported by the Teldat-UC3M Chair, and the other authors by the KONERY-UPCT Chair. The experimental results were obtained using the George Mason University Office of Research computing Argo Research Cluster.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fernández-García, ME., Sancho-Gómez, JL., Ros-Ros, A. et al. Complete Stacked Denoising Auto-Encoders for Regression. Neural Process Lett 53, 787–797 (2021). https://doi.org/10.1007/s11063-020-10419-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-020-10419-0