Skip to main content
Log in

Image real-time augmented reality technology based on spatial color and depth consistency

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Augmented reality can enhance people’s perception of the environment by embedding virtual objects or other information in real-time images. In this paper, the color image is used as a reference to calculate the confidence of the original depth map, and stereo matching is performed according to the feature points. The depth map is mainly enhanced by the color, edge, and segmentation results of the color image. A deep computing system based on augmented reality is designed. The system can use a binocular camera to collect object images in real time and obtain better parallax images by correcting the calibrated image. The semi-global block matching algorithm and depth calculation are used to realize the tracking registration of virtual objects. Experiments in different environments show that the system has good real-time performance, light invariance, and depth consistency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Liou, H.H., Yang, S.J.H., Chen, S.Y., et al.: The influences of the 2D image-based augmented reality and virtual reality on student learning. Educ. Technol. Soc. 20(3), 110–121 (2017)

    Google Scholar 

  2. Ai, D., Yang, J., Fan, J., et al.: Augmented reality based real-time subcutaneous vein imaging system. Biomed. Opt. Express 7(7), 2565–2585 (2016)

    Article  Google Scholar 

  3. Macdougall, R.D., Don, S., Scherrer, B.: SU-C-209-06: improving X-ray imaging with computer vision and augmented reality. Med. Phys. 43(6), 3329 (2016)

    Article  Google Scholar 

  4. Xu, Q., Wang, L., Hei, X.H., Shen, P., Shi, W., Shan, L.: GI/Geom/1 queue based on communication model for mesh networks. Int. J. Commun. Syst. 27(11), 3013–3029 (2013)

    Google Scholar 

  5. Yang, X.L., Shen, P.Y., et al.: Holes detection in anisotropic sensornets: topological methods. Int. J. Distrib. Sens. Netw. 8(10), 135054 (2012)

    Article  Google Scholar 

  6. Punpongsanon, P., Iwai, D., Sato, K.: SoftAR: visually manipulating haptic softness perception in spatial augmented reality. IEEE Trans. Vis. Comput. Graph. 21(11), 1279–1288 (2015)

    Article  Google Scholar 

  7. Mura, M.D., Dini, G., Failli, F.: An integrated environment based on augmented reality and sensing device for manual assembly workstations. Procedia Cirp 41, 340–345 (2016)

    Article  Google Scholar 

  8. Phon, D.N.E., Ali, M.B., Halim, N.D.A.: Learning with augmented reality: effects toward student with different spatial abilities. Adv. Sci. Lett. 21(7), 2200–2204 (2015)

    Article  Google Scholar 

  9. Bach, B., Sicat, R., Beyer, J., et al.: The hologram in my hand: how effective is interactive exploration of 3D visualizations in immersive tangible augmented reality. IEEE Trans. Vis. Comput. Graph. PP(99), 1 (2018)

    Google Scholar 

  10. Huang, H., Hua, H.: High-performance integral-imaging-based light field augmented reality display using freeform optics. Opt. Express 26(13), 17578 (2018)

    Article  Google Scholar 

  11. Dong, S.Y., Kamat, V.R.: Resolving incorrect visual occlusion in outdoor augmented reality using TOF camera and OpenGL frame buffer. In: International Conference on Construction Applications of Virtual Reality (CONVR), pp. 55–63. Sendai, Japan (2010)

  12. Hayashi, K., Kato, H., Nishida, S.: Occlusion detection of real objects using contour based stereo matching. In: International Conference on Augmented Tele-existence, pp. 180–186. ACM, New York, USA (2005)

  13. Behzadan, A.H., Kamat, V.R.: Scalable algorithm for resolving incorrect occlusion in dynamic augmented reality engineering environment. Comput. Aided Civ. Infrastruct. Eng. 25(1), 3–19 (2010)

    Article  Google Scholar 

  14. Lee, A., Lee, J.H., Kim, J.: Data-driven kinematic control for robotic spatial augmented reality system with loose kinematic specifications. ETRI J. 38(2), 337–346 (2016)

    Article  Google Scholar 

  15. Carrera, C.C., Avarvarei, B.V., Chelariu, E.L., et al.: Augmented reality as a digital teaching environment to develop spatial thinking. Am. Cartogr. 44(3), 259–270 (2017)

    Article  Google Scholar 

  16. Zehao, H.E., Xiaomeng, S., Yan, Z., et al.: The development trend of virtual reality and augmented reality technology based on holographic optics. Sci. Technol. Rev. 36(9), 8–17 (2018)

    Google Scholar 

  17. Doshi, A., Smith, R.T., Thomas, B.H., et al.: Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 89(5–8), 1279–1293 (2017)

    Article  Google Scholar 

  18. Riera, A.S., Redondo, E., Fonseca, D.: Geo-located teaching using handheld augmented reality: good practices to improve the motivation and qualifications of architecture students. Univers. Access Inf. Soc. 14(3), 363–374 (2015)

    Article  Google Scholar 

  19. Hamasaki, T., Itoh, Y., Hiroi, Y., et al.: HySAR: hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection. IEEE Trans. Vis. Comput. Graph. 24(4), 1457–1466 (2018)

    Article  Google Scholar 

  20. Duraiswami, R., Zotkin, D.N.: Efficient physics based simulation of spatial audio for virtual and augmented reality. J. Acoust. Soc. Am. 140(4), 2999–3000 (2016)

    Article  Google Scholar 

  21. Zhang, Z.Y.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  22. Sanches, S.R.R., Tokunaga, D.M., Silva, V.F., et al.: Mutual occlusion between real and virtual elements in augmented reality based on fiducial markers. In: IEEE Workshop on the Applications of Computer Vision, pp. 49–54. IEEE Computer Society, Breckenridge, Colorado, USA (2012)

  23. Vretos, N., Daras, P.: Temporal and color consistent disparity estimation in stereo videos. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3798–3802. IEEE, Paris, France (2014)

Download references

Acknowledgements

This work is supported by the Special Scientific Research Project of Shaanxi Provincial Department of Education (2019): Application of Content-based Image Retrieval Algorithm in X-ray Point Feeder (19JK0288).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lijie Zhai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhai, L., Chen, D. Image real-time augmented reality technology based on spatial color and depth consistency. J Real-Time Image Proc 18, 369–377 (2021). https://doi.org/10.1007/s11554-020-00988-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00988-7

Keywords

Navigation