Skip to main content
SpringerLink
Log in

Integrating global and local image features for enhanced loop closure detection in RGB-D SLAM systems

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Loop closure detection is essential for simultaneous localization and mapping systems to decrease accumulating drift of trajectory estimations. Robust loop closure detection is specifically important in large-scale mapping, but it gets more challenging as the mapping environment grows. This paper proposes a SLAM system utilizing a two-pass loop closure detection method to improve mapping accuracy in large-scale environments. The proposed system finds loop closure candidates by employing global and local image features together. After selecting a group of candidates by similarity of global features, the system applies keypoint matching on this group to improve scene matching accuracy and determines loop closure candidates. We extensively evaluate the system on the widely used TUM RGB-D dataset, which contains sequences of small to large-scale indoor environments, with respect to different parameter combinations. The results show that the proposed method increases accuracy substantially and achieves large-scale mapping with acceptable overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. OpenCV implementations are used for Intersection, Hellinger, Chi-Square, and Correlation distance metrics.

References

  1. Agrawal, M., Konolige, K., Blas, M.: Censure: center surround extremas for realtime feature detection and matching. In: Computer Vision—ECCV 2008, pp. 102–115. Springer (2008)

  2. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517. IEEE (2012)

  3. Angeli, A., Filliat, D., Doncieux, S., Meyer, J.A.: Fast and incremental method for loop-closure detection using bags of visual words. IEEE Trans. Robot. 24(5), 1027–1037 (2008)

    Article  Google Scholar 

  4. Arandjelović, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2911–2918. IEEE (2012)

  5. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  6. Calonder, M., Lepetit, V., Fua, P.: Keypoint signatures for fast learning and recognition. In: Computer Vision—ECCV 2008, pp. 58–71. Springer (2008)

  7. Cummins, M., Newman, P.: Appearance-only slam at large scale with fab-map 2.0. Int. J. Robot. Res. 30(9), 1100–1123 (2011)

    Article  Google Scholar 

  8. Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the RGB-D slam system. In: 2012 IEEE International Conference on Robotics and Automation (ICRA), pp. 1691–1696. IEEE (2012)

  9. Endres, F., Hess, J., Sturm, J., Cremers, D., Burgard, W.: 3-d mapping with an RGB-D camera. IEEE Trans. Robot. 30(1), 177–187 (2014)

    Article  Google Scholar 

  10. Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  11. Fu, Z., Guo, Y., An, W.: Simultaneous context feature learning and hashing for large scale loop closure detection. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 1689–1694. IEEE (2018)

  12. Gálvez-López, D., Tardos, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28(5), 1188–1197 (2012)

    Article  Google Scholar 

  13. Gao, X., Zhang, T.: Unsupervised learning to detect loops using deep neural networks for visual slam system. Autonom. Robots 41(1), 1–18 (2017)

    Article  MathSciNet  Google Scholar 

  14. Glocker, B., Shotton, J., Criminisi, A., Izadi, S.: Real-time RGB-D camera relocalization via randomized ferns for keyframe encoding. IEEE Trans. Vis. Comput. Graph. 21(5), 571–583 (2015)

    Article  Google Scholar 

  15. Grisetti, G., Grzonka, S., Stachniss, C., Pfaff, P., Burgard, W.: Efficient estimation of accurate maximum likelihood maps in 3d. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. IROS 2007, pp. 3472–3478. IEEE (2007)

  16. Guclu, O., Can, A.: A comparison of feature detectors and descriptors in rgb-d slam methods. In: Image Analysis and Recognition, pp. 297–305. Springer (2015)

  17. Guclu, O., Can, A.: Histogram based visual place recognition for improving slam performance. In: 2016 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 174–180. IEEE (2016)

  18. Guclu, O., Can, A.B.: Fast and effective loop closure detection to improve slam performance. J. Intell. Robot. Syst. 93(3), 495–517 (2019)

    Article  Google Scholar 

  19. Gutierrez-Gomez, D., Mayol-Cuevas, W., Guerrero, J.: Dense RGB-D visual odometry using inverse depth. Robot. Auton. Syst. 75, 571–583 (2016)

    Article  Google Scholar 

  20. Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: Rgb-d mapping: using depth cameras for dense 3d modeling of indoor environments. In: In the 12th International Symposium on Experimental Robotics (ISER). Citeseer (2010)

  21. Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using kinect-style depth cameras for dense 3d modeling of indoor environments. Int. J. Robot. Res. 31(5), 647–663 (2012)

    Article  Google Scholar 

  22. Huang, J., Kumar, S.R., Mitra, M., Zhu, W.J., Zabih, R.: Image indexing using color correlograms. In: Proceedings of the 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 762–768. IEEE (1997)

  23. Kerl, C., Sturm, J., Cremers, D.: Dense visual slam for RGB-D cameras. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2100–2106. IEEE (2013)

  24. Konolige, K.: Sparse bundle adjustment. In: BMVC, pp. 1–11 (2010)

  25. Kümmerle, R., Grisetti, G., Strasdat, H., Konolige, K., Burgard, W.: g2o: a general framework for graph optimization. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 3607–3613. IEEE (2011)

  26. Labbe, M., Michaud, F.: Online global loop closure detection for large-scale multi-session graph-based slam. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), pp. 2661–2666. IEEE (2014)

  27. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  28. Maier, R., Sturm, J., Cremers, D.: Submap-based bundle adjustment for 3d reconstruction from RGB-D data. In: Pattern Recognition, pp. 54–65. Springer (2014)

  29. Merrill, N., Huang, G.: Lightweight unsupervised deep loop closure. In: Proceedings of the Robotics: Science and Systems (RSS), Pittsburgh, PA (2018)

  30. Mur-Artal, R., Tardos, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot. PP(99), 1–8 (2017)

    Google Scholar 

  31. Newcombe, R., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A., Kohi, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and augmented reality (ISMAR), pp. 127–136. IEEE (2011)

  32. Nicosevici, T., Garcia, R.: Automatic visual bag-of-words for online robot navigation and mapping. IEEE Trans. Robot. 28(4), 886–898 (2012)

    Article  Google Scholar 

  33. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Computer Vision—ECCV 2006, pp. 430–443. Springer (2006)

  34. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)

  35. Segal, A., Haehnel, D., Thrun, S.: Generalized-icp. In: Robotics: Science and Systems, vol. 2 (2009)

  36. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, 2003, pp. 1470–1477. IEEE (2003)

  37. Stückler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3d modeling and tracking. J. Vis. Commun. Image Represent. 25(1), 137–147 (2014)

    Article  Google Scholar 

  38. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D slam systems. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 573–580. IEEE (2012)

  39. Swain, M.J., Ballard, D.H.: Color indexing. Int. J. Comput. Vis. 7(1), 11–32 (1991)

    Article  Google Scholar 

  40. Whelan, T., Kaess, M., Johannsson, H., Fallon, M., Leonard, J., McDonald, J.: Real-time large-scale dense RGB-D slam with volumetric fusion. Int. J. Robot. Res. 34(4–5), 598–626 (2015)

    Article  Google Scholar 

  41. Whelan, T., McDonald, J., Kaess, M., Fallon, M., Johannsson, H., Leonard, J.: Kintinuous: Spatially extended KinectFusion. In: RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras (2012)

  42. Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: Elasticfusion: real-time dense slam and light source estimation. Int. J. Robot. Res. 35(14), 1697–1716 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Hacettepe University, Ankara, Turkey

    Oguzhan Guclu & Ahmet Burak Can

Authors
  1. Oguzhan Guclu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ahmet Burak Can
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmet Burak Can.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guclu, O., Can, A.B. Integrating global and local image features for enhanced loop closure detection in RGB-D SLAM systems. Vis Comput 36, 1271–1290 (2020). https://doi.org/10.1007/s00371-019-01720-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01720-8

Keywords

Search

Navigation