Abstract
Augmented reality can enhance people’s perception of the environment by embedding virtual objects or other information in real-time images. In this paper, the color image is used as a reference to calculate the confidence of the original depth map, and stereo matching is performed according to the feature points. The depth map is mainly enhanced by the color, edge, and segmentation results of the color image. A deep computing system based on augmented reality is designed. The system can use a binocular camera to collect object images in real time and obtain better parallax images by correcting the calibrated image. The semi-global block matching algorithm and depth calculation are used to realize the tracking registration of virtual objects. Experiments in different environments show that the system has good real-time performance, light invariance, and depth consistency.
Similar content being viewed by others
References
Liou, H.H., Yang, S.J.H., Chen, S.Y., et al.: The influences of the 2D image-based augmented reality and virtual reality on student learning. Educ. Technol. Soc. 20(3), 110–121 (2017)
Ai, D., Yang, J., Fan, J., et al.: Augmented reality based real-time subcutaneous vein imaging system. Biomed. Opt. Express 7(7), 2565–2585 (2016)
Macdougall, R.D., Don, S., Scherrer, B.: SU-C-209-06: improving X-ray imaging with computer vision and augmented reality. Med. Phys. 43(6), 3329 (2016)
Xu, Q., Wang, L., Hei, X.H., Shen, P., Shi, W., Shan, L.: GI/Geom/1 queue based on communication model for mesh networks. Int. J. Commun. Syst. 27(11), 3013–3029 (2013)
Yang, X.L., Shen, P.Y., et al.: Holes detection in anisotropic sensornets: topological methods. Int. J. Distrib. Sens. Netw. 8(10), 135054 (2012)
Punpongsanon, P., Iwai, D., Sato, K.: SoftAR: visually manipulating haptic softness perception in spatial augmented reality. IEEE Trans. Vis. Comput. Graph. 21(11), 1279–1288 (2015)
Mura, M.D., Dini, G., Failli, F.: An integrated environment based on augmented reality and sensing device for manual assembly workstations. Procedia Cirp 41, 340–345 (2016)
Phon, D.N.E., Ali, M.B., Halim, N.D.A.: Learning with augmented reality: effects toward student with different spatial abilities. Adv. Sci. Lett. 21(7), 2200–2204 (2015)
Bach, B., Sicat, R., Beyer, J., et al.: The hologram in my hand: how effective is interactive exploration of 3D visualizations in immersive tangible augmented reality. IEEE Trans. Vis. Comput. Graph. PP(99), 1 (2018)
Huang, H., Hua, H.: High-performance integral-imaging-based light field augmented reality display using freeform optics. Opt. Express 26(13), 17578 (2018)
Dong, S.Y., Kamat, V.R.: Resolving incorrect visual occlusion in outdoor augmented reality using TOF camera and OpenGL frame buffer. In: International Conference on Construction Applications of Virtual Reality (CONVR), pp. 55–63. Sendai, Japan (2010)
Hayashi, K., Kato, H., Nishida, S.: Occlusion detection of real objects using contour based stereo matching. In: International Conference on Augmented Tele-existence, pp. 180–186. ACM, New York, USA (2005)
Behzadan, A.H., Kamat, V.R.: Scalable algorithm for resolving incorrect occlusion in dynamic augmented reality engineering environment. Comput. Aided Civ. Infrastruct. Eng. 25(1), 3–19 (2010)
Lee, A., Lee, J.H., Kim, J.: Data-driven kinematic control for robotic spatial augmented reality system with loose kinematic specifications. ETRI J. 38(2), 337–346 (2016)
Carrera, C.C., Avarvarei, B.V., Chelariu, E.L., et al.: Augmented reality as a digital teaching environment to develop spatial thinking. Am. Cartogr. 44(3), 259–270 (2017)
Zehao, H.E., Xiaomeng, S., Yan, Z., et al.: The development trend of virtual reality and augmented reality technology based on holographic optics. Sci. Technol. Rev. 36(9), 8–17 (2018)
Doshi, A., Smith, R.T., Thomas, B.H., et al.: Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 89(5–8), 1279–1293 (2017)
Riera, A.S., Redondo, E., Fonseca, D.: Geo-located teaching using handheld augmented reality: good practices to improve the motivation and qualifications of architecture students. Univers. Access Inf. Soc. 14(3), 363–374 (2015)
Hamasaki, T., Itoh, Y., Hiroi, Y., et al.: HySAR: hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection. IEEE Trans. Vis. Comput. Graph. 24(4), 1457–1466 (2018)
Duraiswami, R., Zotkin, D.N.: Efficient physics based simulation of spatial audio for virtual and augmented reality. J. Acoust. Soc. Am. 140(4), 2999–3000 (2016)
Zhang, Z.Y.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
Sanches, S.R.R., Tokunaga, D.M., Silva, V.F., et al.: Mutual occlusion between real and virtual elements in augmented reality based on fiducial markers. In: IEEE Workshop on the Applications of Computer Vision, pp. 49–54. IEEE Computer Society, Breckenridge, Colorado, USA (2012)
Vretos, N., Daras, P.: Temporal and color consistent disparity estimation in stereo videos. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3798–3802. IEEE, Paris, France (2014)
Acknowledgements
This work is supported by the Special Scientific Research Project of Shaanxi Provincial Department of Education (2019): Application of Content-based Image Retrieval Algorithm in X-ray Point Feeder (19JK0288).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhai, L., Chen, D. Image real-time augmented reality technology based on spatial color and depth consistency. J Real-Time Image Proc 18, 369–377 (2021). https://doi.org/10.1007/s11554-020-00988-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-020-00988-7