Skip to main content
SpringerLink
Log in

A compressed matrix sequence method for solving normal equations of bundle adjustment

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

A Correction to this article was published on 13 June 2021

This article has been updated

Abstract

Bundle adjustment is a least squares method-based algorithm for minimizing the global reprojection error and has provided an effective solution for structure from motion (SfM). The Levenberg–Marquardt algorithm provides a feasible and convenient way for bundle adjustment and creates a system of linear equations which are normal equations. For the special sparsity structure of an augmented Hessian matrix, a Schur complement trick is introduced to reduce computation complexity. However, the general sparse matrix storage formats are not optimized for the augmented Hessian matrix and consume too much computation time. According to the Schur complement trick, this paper divides the arrow-like augmented Hessian matrix into a structure matrix, a camera matrix and an observation matrix, and then proposes a new compressed matrix sequence (CMS) method to reduce time complexity for matrix operations. Under the definition of CMS, all of the matrices are stored in a dense form to accelerate the matrix operations, of which the matrix operations are redefined as well. CMS costs little computation time to build sparse matrices or access sparse matrices. The experimental results show that CMS achieves a significant speedup over general sparse matrix storage formats. Also, CMS being insensitive to the data input is more stable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Change history

References

  1. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Chapter 21: Bundle Adjustment-A Modern Synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice, vol. 1883, pp. 298–372. Springer, Berlin (2000)

    Chapter  Google Scholar 

  2. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)

    Book  Google Scholar 

  3. Jeong, Y., Nister, D., Steedly, D., Szeliski, R., Kweon, I.: Pushing the envelope of modern methods for bundle adjustment. In: CVPR10, pp. 1474-1481 (2010)

  4. Atkinson, K.B. (ed.): Close Range Photogrammetry and Machine Vision. Whittles Publishing, Roseleigh House, Latheronwheel, Caithness, Scotland (1996)

  5. Granshaw, S.: Bundle adjustment methods in engineering photogrammetry. Photogramm. Record 10(56), 181–207 (1980)

    Article  Google Scholar 

  6. Heinly, J., Schonberger, J. L., Dunn, E., Frahm, J. M.: Reconstructing the world* in six days*(as captured by the yahoo 100 million image dataset). In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3287-3295 (2015)

  7. Çalışkan, A., Çevik, U.: Three-dimensional modeling in medical image processing by using fractal geometry. J. Comput. 12(5), 479–485 (2017)

    Article  Google Scholar 

  8. Dellaert, F., Kaess, M.: Square Root SAM: Simultaneous localization and mappingvia square root information smoothing. Int. J. Robot. Res. 25(12), 1181–1203 (2006)

  9. Kaess, M., Ranganathan, A., Dellaert F.: iSAM: Incremental Smoothing and Mapping, IEEE Trans Robot (TRO), 24(6), 1365–1378 (2008)

  10. Kaess, M., Johannsson, H., Roberts, R. et al.: iSAM2: Incremental smoothing and mapping using the Bayes tree. Int. J. Robot. Res. 31(2), 216–235 (2012)

  11. Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2000)

    MATH  Google Scholar 

  12. Moré, J. J.: The levenberg-marquardt algorithm: implementation and theory. In: Numerical analysis, pp. 105-116. Springer (1978)

  13. Björck, A.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  Google Scholar 

  14. Madsen, K., Nielsen, H., Tingleff, O.: Methods for non-linear least squares problems (2004)

  15. Lourakis, M., Argyros, A.A.: SBA: A software package for generic sparse bundle adjustment. TOMS 36, 2 (2009)

    Article  MathSciNet  Google Scholar 

  16. Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Croz, J. Du, Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, Third Edition. SIAM Press, Philadelphia, (1999). LAPACK home page: http://www.netlib.org/lapack

  17. Lawson, C.L., Hanson, R.J., Kincaid, D., Krogh, F.T.: Subprograms, basic linear algebra., for Fortran Usage, ACM Trans. Math Soft 5, 308–323 (1979)

    Article  Google Scholar 

  18. Agarwal, S., Snavely, N., Seitz, S. M., Szeliski, R.: “Bundle adjustment in the large,” In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 29-42 (2010)

  19. Byröd, M., Åström, K.: Conjugate Gradient Bundle Adjustment. In: ECCV10, pp. II: 114–127 (2010)

  20. Bell, N., Garland, M.: Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: SC –09, pp. 1-11 (2009)

  21. Li, R., Saad, Y.: GPU-accelerated preconditioned iterative linear solvers. Technical Report UMSI-2010-xx3, Minnesota Supercomputer Institute, University of Minnesota (2010)

  22. Frahm, J., Georgel, P. Fite, Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y., Dunn, E., Clipp, B., Lazebnik, S., Pollefeys, M.: Building rome on a cloudless day. In: ECCV10, pp. IV: 368-381 (2010)

  23. Gupta, S., Choudhary, S., Narayanan, P.J.: Practical time bundle adjustment for 3d reconstruction on gpu. In: ECCV Workshop on Computer Vision on GPUs (2010)

  24. Wu, C., Agarwal, S., Curless, B., Seitz, S. M.: Multicore bundle adjustment. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 3057-3064. IEEE (2011)

  25. Li, Y., Xie, P., Chen, X., Liu, J., Yang, B., Li, S., Gong, C., Gan, X., Xu, H.: VBSF: a new storage format for SIMD sparse matrixvector multiplication on modern processors. J. Supercomput. 76, 2063–2081 (2020)

  26. Montagne, E., Ekambaram, A.: An optimal storage format for sparse matrices. Inf. Process. Lett. 90(1), 87–92 (2004)

    Article  MathSciNet  Google Scholar 

  27. Vassiliadis, S., Cotofana, S. D., Stathis, P.: Block based compression storage expected performance. In: Proceedings of 14th International Conference on High Performance Computing Systems and Applications (HPCS 2000) June (2000)

  28. Saad, Y.: Iterative methods forsparse linear systems. 2nd edn. Society for Industrial and Applied Mathematics, pp. 95–226 (2003)

  29. Chen, Y., Davis, T.,Hager, W., Rajamanickam, S.: Algorithm 887: CHOLMOD, Supernodal sparse Cholesky factorization and update/downdate. ACM Trans. Math. Softw. 35(3), 22(2008). https://doi.org/10.1145/1391989.1391995

  30. Davis, T., Hager, W.: Dynamic supernodes in sparse Cholesky update/downdate and triangular solves. ACM Trans. Math. Softw. 35(4) (2009)

  31. Rotkin, V., Toledo, S.: The design and implementation of a new out-of-core sparse Cholesky factorization method. ACM Trans. Math. Softw. 30(1), 19–46 (2004)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research work was supported by the National Key Research and Development Program of China(2018YFB0204301). We would also like to thank Dr. Haiyan Chen for her helpful suggestions.

Author information

Authors and Affiliations

  1. Laboratory of Software Engineering for Complex Systems, School of Computer Science, National University of Defense Technology, Changsha, 410073, Hunan, China

    Jiaxin Peng & Jie Liu

  2. Parallel and Distributed Processing Laboratory, School of Computer Science, National University of Defense Technology, Changsha, 410073, Hunan, China

    Jiaxin Peng & Jie Liu

  3. Water and Land Resources Research Center of the Middle Reaches of Yangtze River, School of Resources Environment Science and Engineering, Hubei University of Science and Technology, Xianning, 437100, Hubei, China

    Hua Wei

  4. Algorithm Lab, Changsha Morale Network Technology Co., Ltd., Changsha, 410073, Hunan, China

    Jiaxin Peng

Authors
  1. Jiaxin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hua Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: Error in affiliation for Jiaxin Peng corrected.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, J., Liu, J. & Wei, H. A compressed matrix sequence method for solving normal equations of bundle adjustment. Machine Vision and Applications 32, 81 (2021). https://doi.org/10.1007/s00138-021-01212-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01212-7

Keywords

Search

Navigation