Skip to main content
SpringerLink
Log in

Adaptive Dimension-Discriminative Low-Rank Tensor Recovery for Computational Hyperspectral Imaging

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Exploiting the prior information is fundamental for image reconstruction in computational hyperspectral imaging (CHI). Existing methods usually unfold the 3D signal as a 1D vector and then handle the prior information among different dimensions in an indiscriminative manner, which inevitably ignores the high-dimensionality nature of the hyperspectral image (HSI) and thus results in poor reconstruction performance. In this paper, we propose a high-order tensor optimization based reconstruction method to boost the quality of CHI. Specifically, we first propose an adaptive dimension-discriminative low-rank tensor recovery (ADLTR) model to exploit the high-dimensionality prior of HSI faithfully. In the ADLTR model, we utilize the 3D tensors as the basic elements to fundamentally preserve the structure information in the spatial and spectral dimensions, introduce a dimension-discriminative low-rankness model to fully characterize the prior in the basic elements, and propose a weight estimation strategy by adaptively exploiting the diversity in each dimension. Then, we develop an optimization framework for the CHI reconstruction by integrating the structure prior in ADLTR with the system imaging principle, which is finally solved via the alternating minimization scheme. Extensive experiments on both synthetic and real data demonstrate that our method outperforms state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  • Arce, G. R., Brady, D. J., Carin, L., Arguello, H., & Kittle, D. S. (2014). Compressive coded aperture spectral imaging: An introduction. IEEE Signal Processing Magazine, 31(1), 105–115.

    Article  Google Scholar 

  • Baraniuk, R. G. (2007). Compressive sensing [lecture notes]. IEEE Signal Processing Magazine, 24(4), 118–121.

    Article  Google Scholar 

  • Basedow, R. W., Carmer, D. C., & Anderson, M. E. (1995). Hydice system: Implementation and performance. Imaging Spectrometry, 2480, 258–268.

    Article  Google Scholar 

  • Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2007). A new twist: Two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Transactions on Image Processing, 16(12), 2992–3004.

    Article  MathSciNet  Google Scholar 

  • Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 1–122.

    Article  MATH  Google Scholar 

  • Candes, E. J. (2006). Compressive sampling. Proceedings of the International Congress of Mathematicians, 3, 1433–1452.

    MathSciNet  MATH  Google Scholar 

  • Candes, E. J., Eldar, Y. C., Needell, D., & Randall, P. (2011). Compressed sensing with coherent and redundant dictionaries. Applied and Computational Harmonic Analysis, 31(1), 59–73.

    Article  MathSciNet  MATH  Google Scholar 

  • Cao, W., Wang, Y., Yang, C., Chang, X., Han, Z., & Xu, Z. (2015). Folded-concave penalization approaches to tensor completion. Neurocomputing, 152, 261–273.

    Article  Google Scholar 

  • Cao, X., Du, H., Tong, X., Dai, Q., & Lin, S. (2011). A prism-mask system for multispectral video acquisition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2423–2435.

    Article  Google Scholar 

  • Cao, X., Yue, T., Lin, X., Lin, S., Yuan, X., Dai, Q., et al. (2016). Computational snapshot multispectral cameras: Toward dynamic capture of the spectral world. IEEE Signal Processing Magazine, 33(5), 95–108.

    Article  Google Scholar 

  • Chakrabarti, A., & Zickler, T. (2011). Statistics of real-world hyperspectral images. In IEEE conference on computer vision and pattern recognition (pp. 193–200).

  • Chang, Y., Yan, L., & Zhong, S. (2017). Hyper-laplacian regularized unidirectional low-rank tensor recovery for multispectral image denoising. In IEEE conference on computer vision and pattern recognition (pp. 5901–5909).

  • Choi, I., Jeon, D. S., Nam, G., Gutierrez, D., & Kim, M. H. (2017). High-quality hyperspectral reconstruction using a spectral prior. ACM Transactions on Graphics, 36(6), 218.

    Article  Google Scholar 

  • Descour, M., & Dereniak, E. (1995). Computed-tomography imaging spectrometer: Experimental calibration and reconstruction results. Applied Optics, 34(22), 4817–26.

    Article  Google Scholar 

  • Dian, R., Fang, L., & Li, S. (2017). Hyperspectral image super-resolution via non-local sparse tensor factorization. In IEEE conference on computer vision and pattern recognition (pp. 5344–5353).

  • Donoho, D. L., Maleki, A., & Montanari, A. (2009). Message-passing algorithms for compressed sensing. Proceedings of the National Academy of Sciences, 106(45), 18914–18919.

    Article  Google Scholar 

  • Figueiredo, M. A., Nowak, R. D., & Wright, S. J. (2007). Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE Journal of Selected Topics in Signal Processing, 1(4), 586–597.

    Article  Google Scholar 

  • Fu, Y., Zheng, Y., Sato, I., & Sato, Y. (2016). Exploiting spectral–spatial correlation for coded hyperspectral image restoration. In IEEE conference on computer vision and pattern recognition (pp. 3727–3736).

  • Fu, Y., Lam, A., Sato, I., & Sato, Y. (2017). Adaptive spatial–spectral dictionary learning for hyperspectral image restoration. International Journal of Computer Vision, 122(2), 228–245.

    Article  MathSciNet  MATH  Google Scholar 

  • Gao, L., & Wang, L. V. (2016). A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel. Physics Reports, 616, 1–37.

    Article  MathSciNet  Google Scholar 

  • Gevers, T., & Stokman, H. M. G. (2003). Robust photometric invariant region detection in multispectral images. International Journal of Computer Vision, 53(2), 135–151.

    Article  Google Scholar 

  • Giryes, R., & Elad, M. (2014). Sparsity-based Poisson denoising with dictionary learning. IEEE Transactions on Image Processing, 23(12), 5057–5069.

    Article  MathSciNet  MATH  Google Scholar 

  • Gregor, K., & LeCun, Y. (2010). Learning fast approximations of sparse coding. In International conference on machine learning (pp. 399–406).

  • He, K. S., Rocchini, D., Neteler, M., & Nagendra, H. (2011). Benefits of hyperspectral remote sensing for tracking plant invasions. Diversity and Distributions, 17(3), 381–392.

    Article  Google Scholar 

  • He, R., Cao, J., Song, L., Sun, Z., & Tan, T. (2019). Adversarial cross-spectral face completion for NIR-VIS face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42, 1025–1037.

    Article  Google Scholar 

  • Hestenes, M. R., & Stiefel, E. (1953). Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49, 409–436.

    Article  MathSciNet  MATH  Google Scholar 

  • Huang, C., Ding, X., Fang, C., & Wen, D. (2014). Robust image restoration via adaptive low-rank approximation and joint kernel regression. IEEE Transactions on Image Processing, 23(12), 5284–5297.

    Article  MathSciNet  MATH  Google Scholar 

  • Kittle, D., Choi, K., Wagadarikar, A., & Brady, D. J. (2010). Multiframe image estimation for coded aperture snapshot spectral imagers. Applied Optics, 49(36), 6824.

    Article  Google Scholar 

  • Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51(3), 455–500.

    Article  MathSciNet  MATH  Google Scholar 

  • Kwon, H., & Nasrabadi, N. M. (2007). Kernel spectral matched filter for hyperspectral imagery. International Journal of Computer Vision, 71(2), 127–141.

    Article  Google Scholar 

  • Li, C., Ma, Y., Huang, J., Mei, X., & Ma, J. (2015). Hyperspectral image denoising using the robust low-rank tensor recovery. Journal of the Optical Society of America A, 32(9), 1604–1612.

    Article  Google Scholar 

  • Lin, X., Liu, Y., Wu, J., & Dai, Q. (2014a). Spatial-spectral encoded compressive hyperspectral imaging. ACM Transactions on Graphics, 33(6), 1–11.

    Article  Google Scholar 

  • Lin, X., Wetzstein, G., Liu, Y., & Dai, Q. (2014b). Dual-coded compressive hyperspectral imaging. Optics Letters, 39(7), 2044–2047.

    Article  Google Scholar 

  • Liu, J., Musialski, P., Wonka, P., & Ye, J. (2013). Tensor completion for estimating missing values in visual data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 208–220.

    Article  Google Scholar 

  • Liu, Y., Yuan, X., Suo, J., Brady, D., & Dai, Q. (2018). Rank minimization for snapshot compressive imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(12), 2990–3006.

    Article  Google Scholar 

  • Llull, P., Liao, X., Yuan, X., Yang, J., Kittle, D., Carin, L., et al. (2013). Coded aperture compressive temporal imaging. Optics Express, 21(9), 10526–10545.

    Article  Google Scholar 

  • Ma, C., Cao, X., Tong, X., Dai, Q., & Lin, S. (2014). Acquisition of high spatial and spectral resolution video with a hybrid camera system. International Journal of Computer Vision, 110(2), 141–155.

    Article  Google Scholar 

  • Mejía, Y., Arguello, H., Costa, F., Tourneret, J., & Batatia, H. (2017). Bayesian reconstruction of hyperspectral images by using compressed sensing measurements and a local structured prior. In IEEE international conference on acoustics, speech and signal processing (pp. 3116–3120).

  • Miao, X., Yuan, X., Pu, Y., & Athitsos, V. (2019). lambda-net: Reconstruct hyperspectral images from a snapshot measurement. In IEEE international conference on computer vision (pp. 4058–4068).

  • Nguyen, H. V., Banerjee, A., & Chellappa, R. (2010). Tracking via object reflectance using a hyperspectral video camera. In IEEE conference on computer vision and pattern recognition workshops (pp. 44–51).

  • Okamoto, T., & Yamaguchi, I. (1991). Simultaneous acquisition of spectral image information. Optics Letters, 16(16), 1277–1279.

    Article  Google Scholar 

  • Pan, Z., Healey, G., Prasad, M., & Tromberg, B. (2003). Face recognition in hyperspectral images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1552–1560.

    Article  Google Scholar 

  • Peng, Y., Meng, D., Xu, Z., Gao, C., Yang, Y., & Zhang, B. (2014). Decomposable nonlocal tensor dictionary learning for multispectral image denoising. In IEEE conference on computer vision and pattern recognition (pp. 2949–2956).

  • Porter, W. M., & Enmark, H. T. (1987). A system overview of the airborne visible/infrared imaging spectrometer (AVIRIS). Imaging Spectroscopy II, 834, 22–32.

    Article  Google Scholar 

  • Rajwade, A., Rangarajan, A., & Banerjee, A. (2013). Image denoising using the higher order singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(4), 849–862.

    Article  Google Scholar 

  • Romera-Paredes, B., & Pontil, M. (2013). A new convex relaxation for tensor completion. In Advances in neural information processing systems (pp. 2967–2975).

  • Salmon, J., Harmany, Z., Deledalle, C. A., & Willett, R. (2014). Poisson noise reduction with non-local PCA. Journal of Mathematical Imaging and Vision, 48(2), 279–294.

    Article  MathSciNet  MATH  Google Scholar 

  • Salzenstein, F., & Collet, C. (2006). Fuzzy Markov random fields versus chains for multispectral image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11), 1753–1767.

    Article  Google Scholar 

  • Schechner, Y. Y., & Nayar, S. K. (2002). Generalized mosaicing: Wide field of view multispectral imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(10), 1334–1348.

    Article  Google Scholar 

  • Shewchuk J. R., et al. (1994). An introduction to the conjugate gradient method without the agonizing pain.

  • Sun, J., Li, H., & Xu, Z., et al. (2016). Deep ADMM-net for compressive sensing MRI. In Advances in neural information processing systems (pp. 10–18).

  • Tan, J., Ma, Y., Rueda, H., Baron, D., & Arce, G. R. (2016). Compressive hyperspectral imaging via approximate message passing. IEEE Journal of Selected Topics in Signal Processing, 10(2), 389–401.

    Article  Google Scholar 

  • Tarabalka, Y., Chanussot, J., & Benediktsson, J. A. (2009). Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 40(5), 1267–1279.

    Article  Google Scholar 

  • Tucker, L. R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3), 279–311.

    Article  MathSciNet  Google Scholar 

  • Wagadarikar, A., John, R., Willett, R., & Brady, D. (2008). Single disperser design for coded aperture snapshot spectral imaging. Applied Optics, 47(10), B44–B51.

    Article  Google Scholar 

  • Wald, L. (2002). Data fusion: Definitions and architectures: Fusion of images of different spatial resolutions. Presses des MINES.

  • Wang, L., Xiong, Z., Gao, D., Shi, G., & Wu, F. (2015a). Dual-camera design for coded aperture snapshot spectral imaging. Applied Optics, 54(4), 848–58.

    Article  Google Scholar 

  • Wang, L., Xiong, Z., Gao, D., Shi, G., Zeng, W., & Wu, F. (2015b). High-speed hyperspectral video acquisition with a dual-camera architecture. In IEEE conference on computer vision and pattern recognition (pp. 4942–4950).

  • Wang, L., Xiong, Z., Shi, G., Wu, F., & Zeng, W. (2017). Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(10), 2104–2111.

    Article  Google Scholar 

  • Wang, L., Zhang, T., Fu, Y., & Huang, H. (2018). Hyperreconnet: Joint coded aperture optimization and image reconstruction for compressive hyperspectral imaging. IEEE Transactions on Imaging Processing, 28(5), 2257–2270.

    Article  MathSciNet  Google Scholar 

  • Wang, L., Sun, C., Fu, Y., Kim, M. H., & Huang, H. (2019). Hyperspectral image reconstruction using a deep spatial-spectral prior. In IEEE conference on computer vision and pattern recognition (pp. 8032–8041).

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Wu, Y., Mirza, I. O., Arce, G. R., & Prather, D. W. (2011). Development of a digital-micromirror-device-based multishot snapshot spectral imaging system. Optics Letters, 36(14), 2692–2694.

    Article  Google Scholar 

  • Xie, Q., Zhao, Q., Meng, D., Xu, Z., Gu, S., Zuo, W., & Zhang, L. (2016). Multispectral images denoising by intrinsic tensor sparsity regularization. In IEEE conference on computer vision and pattern recognition (pp. 1692–1700).

  • Xie, Q., Zhao, Q., Meng, D., & Xu, Z. (2018a). Kronecker-basis-representation based tensor sparsity and its applications to tensor recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(8), 1888–1902.

    Article  Google Scholar 

  • Xie, T., Li, S., Fang, L., & Liu, L. (2018b). Tensor completion via nonlocal low-rank regularization. IEEE Transactions on Cybernetics, 49, 2344–2354.

    Article  Google Scholar 

  • Xiong, Z., Shi, Z., Li, H., Wang, L., Liu, D., & Wu, F. (2017). Hscnn: Cnn-based hyperspectral image recovery from spectrally undersampled projections. In IEEE international conference on computer vision workshops (pp. 518–525).

  • Xue, S., Qiu, W., Liu, F., & Jin, X. (2018). Low-rank tensor completion by truncated nuclear norm regularization. In International conference on pattern recognition (pp. 2600–2605).

  • Yamaguchi, M., Haneishi, H., Fukuda, H., Kishimoto, J., Kanazawa, H., Tsuchida, M., Iwama, R., & Ohyama, N. (2006). High-fidelity video and still-image communication based on spectral information: Natural vision system and its applications. In Spectral imaging: eighth international symposium on multispectral color science (vol. 6062, p. 60620G).

  • Yasuma, F., Mitsunaga, T., Iso, D., & Nayar, S. K. (2010). Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum. IEEE Transactions on Image Processing, 19(9), 2241–53.

    Article  MathSciNet  MATH  Google Scholar 

  • Yokoya, N., & Iwasaki, A. (2016). Airborne hyperspectral data over chikusei. Technical Report; Space Application Laboratory -2016-05-27.

  • Yuan, X., Llull, P., Liao, X., Yang, J., Brady, D. J., Sapiro, G., & Carin, L. (2014). Low-cost compressive sensing for color video and depth. In IEEE conference on computer vision and pattern recognition (pp. 3318–3325).

  • Zhang, J., & Ghanem, B. (2018). Ista-net: Interpretable optimization-inspired deep network for image compressive sensing. In IEEE conference on computer vision and pattern recognition (pp. 1828–1837).

  • Zhang, M., Wang, L., Zhang, L., & Huang, H. (2019). Compressive hyperspectral imaging with non-zero mean noise. Optics Express, 27(13), 17449–17462.

    Article  Google Scholar 

  • Zhang, S., Hua, H., & Fu, Y. (2018a). Fast parallel implementation of dual-camera compressive hyperspectral imaging system. IEEE Transactions on Circuits and Systems for Video Technology, 29, 3404–3414.

    Article  Google Scholar 

  • Zhang, S., Wang, L., Fu, Y., & Huang, H. (2018b). Gpu assisted towards real-time reconstruction for dual-camera compressive hyperspectral imaging. In Pacific rim conference on multimedia (pp. 711–720). Springer.

  • Zhang, Z., Ely, G., Aeron, S., Hao, N., & Kilmer, M. (2014). Novel methods for multilinear data completion and de-noising based on tensor-svd. In IEEE conference on computer vision and pattern recognition (pp. 3842–3849).

  • Zhao, W., & Du, S. (2016). Spectral-spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach. IEEE Transactions on Geoscience and Remote Sensing, 54(8), 4544–4554.

    Article  Google Scholar 

  • Zontak, M., & Irani, M. (2011). Internal statistics of a single natural image. In IEEE conference on computer vision and pattern recognition (pp. 977–984).

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China under Grant 62072038 and Grant 61922014.

Author information

Authors and Affiliations

  1. Beijing Institute of Technology, Beijing, China

    Lizhi Wang

  2. Xi’an Jiaotong University, Xi’an, China

    Shipeng Zhang

  3. Beijing Normal University, Beijing, China

    Hua Huang

Authors
  1. Lizhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shipeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hua Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua Huang.

Additional information

Communicated by Jiaya Jia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, L., Zhang, S. & Huang, H. Adaptive Dimension-Discriminative Low-Rank Tensor Recovery for Computational Hyperspectral Imaging. Int J Comput Vis 129, 2907–2926 (2021). https://doi.org/10.1007/s11263-021-01481-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-021-01481-9

Keywords

Search

Navigation