Abstract
The nonnegative matrix factorization (NMF) has been widely used because it can accomplish both feature representation learning and dimension reduction. However, there are two critical and challenging issues affecting the performance of NMF models. One is the selection of matrix factorization rank, while most of the existing methods are based on experiments or experience. For tackling this issue, an adaptive and stable NMF model is constructed based on an adaptive factorization rank selection (AFRS) strategy, which skillfully and simply integrates a row constraint similar to the generalized elastic net. The other is the sensitivity to the initial value of the iteration, which seriously affects the result of matrix factorization. This issue is alleviated by complementing NMF and deep learning each other and avoiding complex network structure. The proposed NMF model is called deep AFRS-NMF model for short, and the corresponding optimization solution, convergence and stability are analyzed. Moreover, the statistical consistency is discussed between the rank obtained by the proposed model and the ideal rank. The performance of the proposed deep AFRS-NMF model is demonstrated by applying in genetic data-based tumor recognition. Experiments show that the factorization rank obtained by the deep AFRS-NMF model is stable and superior to classical and state-of-the-art methods.
Similar content being viewed by others
References
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791
Gillis N, Glineur F (2010) Using under approximations for sparse non-negative matrix factorization. Pattern Recogn 43:1676–1687
Wild S, Curry J, Dougherty A (2004) Improving non-negative matrix factorizations through structured initialization. Pattern Recogn 37:2217–2232
Boutsidis C, Gallopoulos E (2008) SVD based initialization: a head start for non-negative matrix factorization. Pattern Recogn 41:1350–1362
Zheng CH, Ng TY, Zhang L, Shiu CK, Wang HQ (2011) Tumor classification based on non-negative matrix factorization using gene expression data. IEEE Trans Nanobiosci 10:86–93
Tu D, Chen L, Chen GC, Wu Y, Wang JC (2018) Hierarchical online NMF for detecting and tracking topic hierarchies in a text stream. Pattern Recogn 76:203–214
Cichocki A, Zdunek R (2006) Multilayer non-negative matrix factorization. Electron Lett 42:947–948
Hoyer PO (2002) Non-negative sparse coding. IEEE Workshop on Neural Networks for Signal Processing, vol 0202009. pp 557–565
Miura I, Tachioka Y, Narita T (2016) Multi-channel non-negative matrix factorization with binary mask initialization for automatic speech recognition. J Acoust Soc Am 140:3450–3450
Liu XS, Wang B, Zhang LM (2010) A novel approach for hyperspectral unmixing based on non-negative matrix factorization. In IEEE International geoscience & remote sensing symposium. pp 1289–1292
Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436
Shen D, Wu G, Suk HI (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248
Zhang W, Li R, Deng H et al (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 108:214–224
Suk HI, Lee SW, Shen D et al (2014) Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. Neuroimage 101:569–582
Hinton GE, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554
Han ZY, Wei BZ, Zheng YJ et al (2017) Breast cancer multi-classification from histopathological images with structured deep learning. Sci Rep 4172:1–10
Bengio Y, Lamblin P, Popovici D et al (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153–160
Salakhutdinov R, Hinton G (2012) An efficient learning procedure for deep Boltzmann machines. Neural Comput 24:1967–2006
Sarikaya R, Hinton GE, Deoras A (2014) Application of deep belief networks for natural language understanding. IEEE/ACM Trans Audio Speech Lang Process 22:778–784
Le Roux J, Hershey JR, Weninger F (2015) Deep NMF for speech separation. IEEE International conference on acoustics, speech and signal processing. pp 66–70
Trigeorgis G, Bousmalis K, Zafeiriou S et al (2016) A deep matrix factorization method for learning attribute representations. IEEE Trans Pattern Anal Mach Intell 39:417–429
Yang XH, Wu WM, Chen YM et al (2019) An integrated inverse space sparse representation framework for tumor classification. Pattern Recogn 93:293–311
Xue Y, Tong CS, Chen YCW (2008) Clustering-based initialization for non-negative matrix factorization. Appl Math Comput 205:525–536
Yang Z, Zhu Z, Oja E (2010) Automatic rank determination in projective non-negative matrix factorization. In: Proceedings of 9th international conference on latent variable analysis and signal separation. pp 514–521
Said M, Brie D, Djafari AM et al (2006) Separation of non-negative mixture of non-negative sources using a Bayesian approach and MCMC sampling. IEEE Trans Signal Process 54:4133–4145
Tibshirani R (2011) Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc Ser B (Statistical Methodology) 73:267–288
Pan X, Xu Y (2018) A safe reinforced feature screening strategy for lasso based on feasible solutions. Inf Sci 477:132–147
Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Statistical Methodology) 67:768
Xu Y, Tian Y, Pan X et al (2019) E-ENDPP: a safe feature selection rule for speeding up elastic net. Appl Intell 49:592–604
Gao Y, Church PG (2005) Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21:3970–3975
Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn 30:1145–1159
Alon U, Barkai N, Notterman DA et al (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci 96:6745–6750
Gan B, Zheng CH, Zhang J et al (2014) Sparse representation for tumor classification based on feature extraction using latent low-rank representation. Biomed Res Int 10:63–68
Zheng CH, Zhang L, Ng TY et al (2011) Metasample-based sparse representation for tumor classification. IEEE/ACM Trans Comput Biol Bioinf 8:1273–1282
Liu JX, Xu Y, Zheng CH et al (2015) RPCA-based tumor classification using gene expression data. IEEE/ACM Trans Comput Biol Bioinf 12:964–970
Xiao YH, Chen L, Li D (2018) A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming. Math Program Comput 10:533–555
Chen L, Huang JZ (2012) Sparse reduced-rank regression for simultaneous dimension reduction and variable selection. J Am Stat Assoc 107:1533–1545
Xin X, Hu J, Liu L (2017) On the oracle property of a generalized adaptive elastic-net for multivariate linear regression with a diverging number of parameters. J Multivar Anal 162:16–31
Wright J, Ganesh A, Zhou Z et al (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227
Yang XH, Liu F, Tian L et al (2018) Pseudo-full-space representation based classification for robust face recognition. Signal Process Image Commun 60:64–78
Veer LJV, Dai H, Vijver MJVD et al (2001) Expression profiling predicts poor outcome of disease in young breast cancer patients. Eur J Cancer 37:271–271
Tamayo P (2002) Diffuse large B-cell lymphoma outcome prediction by gene expression profiling and supervised machine learning. Nat Med 8:68–74
Armstrong SA, Staunton JE, Silverman LB et al (2002) MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat Genet 30:41–47
Van't Veer LJ, Dai H, Van De Vijver MJ et al (2002) Gene expression profiling predicts clinical outcome of breast cancer. Nature 415:530–536
Deng W, Hu J, Guo J (2012) Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Trans Pattern Anal Mach Intell 34:1864–1870
Deng H, Runger G (2013) Gene selection with guided regularized random forest. Pattern Recogn 46:3483–3489
García V, Sánchez JS (2015) Mapping microarray gene expression data into dissimilarity spaces for tumor classification. Inf Sci 294:362–375
Dettling M, Bühlmann P (2004) BagBoosting for tumor classification with gene expression data. Bioinformatics 20:1061–1069
Ruiz R, Riquelme JC, Ruiz JSA (2006) Incremental wrapper-based gene selection from microarray data for cancer classification. Pattern Recogn 39:2383–2392
Younsi R, Bagnall A (2016) Ensembles of random sphere cover classifiers. Pattern Recogn 49:213–225
Gan B, Zheng CH, Liu JX (2016) Metasample-based robust sparse representation for tumor classification. Engineering 5:78–83
Hong JH, Cho SB (2009) Gene boosting for cancer classification based on gene expression profiles. Pattern Recogn 42:1761–1767
Piao Y, Piao M, Park K et al (2012) An ensemble correlation-based gene selection algorithm for cancer classification with gene expression data. Bioinformatics 28:3306–3315
Zheng D, Jia J, Fang X, et al (2017) Main and interaction effects selection for quadratic discriminant analysis via penalized linear regression. arXiv:1702.04570
Fan Y, Kong Y, Li D et al (2015) Innovated interaction screening for high-dimensional nonlinear classification. Ann Stat 43:1243–1272
Jiang BY, Chen ZQ, Leng CL (2020) Dynamic linear discriminant analysis in high dimensional space. Bernoulli 26:1234–1268
Su Q, Wang YN, Jiang XB, et al (2017) A cancer gene selection algorithm based on the KS test and CFS. BioMed Res Int 2017:1645619
Acknowledgements
The authors would like to thank https://tumorgenome.nih.gov/ for their breast datasets. We also thank Prof. Bingsheng He and Yunhai Xiao for their optimization suggestion. This work was supported in part by Natural Science Foundations of China (41771375, 11571025, 11701144, 62003004), Natural Science Foundation of Beijing (1182002), Open Fund of Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education (IPIU2019010), Natural Science Foundations of Henan (202102310087, 202300410066, 212102310305, 202102210125).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yang, X., Wu, W., Xin, X. et al. Adaptive factorization rank selection-based NMF and its application in tumor recognition. Int. J. Mach. Learn. & Cyber. 12, 2673–2691 (2021). https://doi.org/10.1007/s13042-021-01353-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-021-01353-1