Skip to main content
SpringerLink
Log in

Algorithm for Three-Dimensional Reconstruction of Nonrigid Objects Using a Depth Camera

  • MATHEMATICAL MODELS AND COMPUTATIONAL METHODS
  • Published:
Journal of Communications Technology and Electronics Aims and scope Submit manuscript

Abstract—An algorithm for 3D reconstruction of objects with nonrigid shape using an RGB-D depth camera is proposed. The algorithm can be used in medicine, agriculture, robotics, virtual reality, and human–computer interaction. The proposed algorithm makes it possible to accurately reconstruct a 3D object with one depth camera without restricting camera movement and without using a priori information about an object shape. The reconstruction process consists of the following steps: input of information using an RGB-D camera, registration with a modified iterative closest point algorithm, and dynamic construction of a dense 3D model of objects. The efficiency of the proposed algorithm is evaluated using experimental data and is compared with the modern methods of registration. The results show that the proposed algorithm can accurately reconstruct 3D nonrigid objects on complex scenes with one depth camera.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

Similar content being viewed by others

REFERENCES

  1. H. Boukamcha, A. Ben Amara, F. Smach, and M. Atri, “Robust technique for 3D shape reconstruction,” J. Comput. Sci. 21, 333−339 (2017).

    Google Scholar 

  2. B. A. Echeagaray-Patron, V. Kober, V. Karnaukhov, and V. Kuznetsov, “A method of face recognition using 3D facial surfaces,” J. Commun. Technol. Electron. 62, 648−652 (2017).

    Article  Google Scholar 

  3. B. A. Echeagaray-Patron and V. Kober, “3D face recognition based on matching of facial surfaces,” Proc. SPIE 9598, 95980-8 (2015).

    Google Scholar 

  4. B. A. Echeagaray-Patron, D. Miramontes-Jaramillo, and V. Kober, “Conformal parameterization and curvature analysis for 3D facial recognition,” in Proc. IEEE Conf. on Computational Science and Computational Intelligence, Enathy, Madurai, Tmilnadu, Dec.2015 (IEEE, New York, 2015), pp. 843−844.

  5. K. Picos, V. Diaz-Ramirez, V. Kober, A. Montemayor, J. Pantrigo, “Accurate three-dimensional pose recognition from monocular images using template matched filtering,” Opt. Eng. 55 (6), 102−113 (2016).

    Article  Google Scholar 

  6. S. Akkoul, A. Hafiane, O. Rozenbaum, E. Lespessailles, and R. Jennane, “3D reconstruction of the proximal femur shape from few pairs of x-ray radiographs,” Signal Process.: Image Commun. 59, 65−72 (2017).

    Google Scholar 

  7. J. A. Gonzalez-Fraga, V. Kober, V. H. Diaz-Ramire, E. Gutierrez, and O. Alvarez-Xochihua, “Accurate generation of the 3D map of environment with a RGB-D camera,” Proc. SPIE 10396 103962A-8 (2017).

    Google Scholar 

  8. J. Gonzalez-Fraga, V. Kober, E. Gutierrez, “Accurate alignment of rgb-d frames for 3d map generation,” Proc. SPIE 10752, 107522J-7 (2018).

    Google Scholar 

  9. A. Ortiz-Gonzґalez and V. Kober, “Adaptive algorithm for the SLAM design with a RGB-D camera,” Proc. SPIE 11137, 1113726-11 (2019).

    Google Scholar 

  10. P. J. Besl and N. D. McKay, “A method for registration of 3D shapes,” IEEE Trans. Pattern Anal. Mach. Intell. 14, 239−256 (1992).

    Article  Google Scholar 

  11. A. Makovetskii, S. Voronin, V. Kober, and A. Voronin, “A point-to-plane registration algorithm for orthogonal transformations,” Proc. SPIE 10752, 107522R-8 (2018).

    MATH  Google Scholar 

  12. J. Shi and C. Tomasi, “Good features to track,” Tech. Rep. (Ithaca, NY, USA, 1993).

    Google Scholar 

  13. D. Miramontes-Jaramillo, V. Kober, V. H. Diaz-Ramirez, and V. Karnaukhov, “Descriptor-based tracking algorithm using a depth camera,” J. Commun. Technol. Electron. 62, 638−647 (2017).

    Article  Google Scholar 

  14. J. Diaz-Escobar, V. Kober, and J. A. Gonzalez-Fraga, “LUIFT: LUminance invariant feature transform,” Math. Problems Eng. 2018, ID 3758102 (2018).

  15. J. Diaz-Escobar, V. Kober, V. Karnaukhov, and J. A. Gonzalez-Fraga, “A new invariant to illumination feature descriptor for pattern recognition,” J. Commun. Technol. Electron. 63, 1469−1474 (2018).

    Article  Google Scholar 

  16. J. Diaz-Escobar and V. Kober, “A robust HOG-based descriptor for pattern recognition,” Proc. SPIE 9971, 99712 (2016).

    Google Scholar 

  17. B. A. Echeagaray-Patron and V. Kober, “Face recognition based on matching of local features on 3D dynamic range sequences,” Proc. SPIE 9971, 997131-6 (2016).

    Article  Google Scholar 

  18. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vision 60, 91−110 (2004).

    Article  Google Scholar 

  19. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Underst. 110, 346−359 (2008).

    Article  Google Scholar 

  20. I. Sipiran and B. Bustos, “Harris 3D: a robust extension of the Harris operator for interest point detection on 3D meshes,” Visual Comput. 27, 963 (2011).

    Article  Google Scholar 

  21. S. M. Smith and J. M. Brady, “SUSAN- a new approach to low level image processing,” Int. J. Comput. Vision 23, 45−78 (1997).

    Article  Google Scholar 

  22. Y. Zhong, “Intrinsic shape signatures: A shape descriptor for 3D object recognition,” in Proc. IEEE Conf. on Computer Vision Workshops, Kyoto, Japan, Sept. 29–Oct. 2,2009 (IEEE, New York, 2010), pp. 689−696.

  23. K. Binmore and J. Davies, Calculus Concepts and Methods (Cambridge Univ. Press, Cambridge, 2007).

    MATH  Google Scholar 

  24. R. B. Rusu, Z. C. Marton, N. Blodow, and M. Beetz, “Persistent point feature histograms for 3D point clouds,” in Proc. 10th Int. Conf. on Intelligent Autonomous Systems (IAS-10), Baden-Baden, Germany,2008 (IAS-10, 2008), pp. 119−128.

  25. F. Tombari, S. Salti, and L. Di Stefano, “Unique signatures of histograms for local surface description,” in Proc. Eur. Conf. on Computer Vision, (ECCV), Hersonissos, Greece, Sept. 5–11,2010 (ECCV, 2010), 356−369.

  26. A. Frome, D. Huber, R. Kolluri, T. Bulow, J. Malik, “Recognizing objects in range data using regional point descriptors,” in Proc. Eur. Conf. on Computer Vision, Prague, Czech Republic, May,2004 (Springer-Verlag, 2004), pp. 224−237.

  27. S. Lazebnik, C. Schmid, and J. Ponce, “A sparse texture representation using local affine regions,” IEEE Trans. Pattern Anal. Mach. Intell. 27, 1265−1278 (2005).

    Article  Google Scholar 

  28. Z. C. Marton, D. Pangercic, N. Blodow, J. Kleinehellefort, and M. Beetz, “General 3D modelling of novel objects from a single view,” in Proc. IEEE/RSJ Conf. on Intelligent Robots and Systems, Taipei, Taiwan, Oct. 18–22,2010 (IEEE, New York, 2010), pp. 3700−3705.

  29. S. Rusinkiewicz and M. Levoy, “Efficient variants of the ICP algorithm,” in Proc. 3rd Int. Conf. on 3-D Digital Imaging and Modeling, Quebec City, Que., Canada,2001 (IEEE Comput. Soc., New York, 2001), pp. 145−152.

  30. A. Makovetskii, S. Voronin, V. Kober, and A. Voronin, “A non-iterative method for approximation of the exact solution to the point-to-plane variational problem for orthogonal transformations,” Math. Methods Appl. Sci. 41 (18), 9218−9230 (2018).

    Article  MathSciNet  Google Scholar 

  31. A. Censi, “An ICP variant using a point-to-line metric,” in Proc. IEEE Conf. on Robotics and Automation (ICRA-2008), Pasadena, CA, USA, May, 19−25,2008 (IEEE, New York, 2008), pp. 19−25.

  32. J. Yang, H. Li, D. Campbell, and Y. Jia, “GO-ICP: A globally optimal solution to 3D ICP point-set registration,” IEEE Trans. Pattern Anal. Mach. Intell. 38, 2241−2254 (2015).

    Article  Google Scholar 

  33. Y. Zheng and D. Doermann, Robust point matching for non-rigid shapes by preserving local neighborhood structures," IEEE Trans. Pattern Anal. Mach. Intell. 28, 643−649 (2006).

    Article  Google Scholar 

  34. A. Myronenko and X. Song, “Point set registration: Coherent point drift” IEEE Trans. Pattern Anal. Mach. Intell. 32, 2262−2275 (2010).

    Article  Google Scholar 

  35. B. Amberg, S. Romdhani, and T. Vetter, “Optimal step non-rigid ICP algorithms for surface registration,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, June 24−26,2008 (IEEE, New York, 2008), pp. 1−8.

  36. J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of RGB-D SLAM systems,” in Proc. IEEE/RSJ Conf. on Intelligent Robots and Systems, Vilamoura, Oct. 7–12,2012 (IEEE, New York, 2012), pp. 573−580.

  37. M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM 24, 381−395 (1981).

    Article  MathSciNet  Google Scholar 

  38. W. Kabsch, “A solution for the best rotation to relate two sets of vectors,” Acta Crystallograph. Section A: Crystal Phys., Diffraction, Theor. & Gen. Crystallogr. 32, 922−923 (1976).

    Google Scholar 

  39. F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin, “The ball-pivoting algorithm for surface reconstruction,” IEEE Trans. on Visualiz. Comp. Graph. 5, 349−359 (1999).

    Google Scholar 

Download references

Funding

This work was supported by the Russian Foundation for Basic Research, project no. 18-08-00782.

Author information

Authors and Affiliations

  1. Department of Computer Sciences, Center for Scientific Research and Higher Education, 22860, Ensenada, Mexico

    M. Ruiz-Rodriguez & V. I. Kober

  2. Institute for Information Transmission Problems, Russian Academy of Sciences, 127051, Moscow, Russia

    V. I. Kober, V. N. Karnaukhov & M. G. Mozerov

  3. Chelyabinsk State University, 454001, Chelyabinsk, Russia

    V. I. Kober

Authors
  1. M. Ruiz-Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. I. Kober
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. V. N. Karnaukhov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. G. Mozerov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. I. Kober.

Additional information

Translated by L. Mukhortova

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ruiz-Rodriguez, M., Kober, V.I., Karnaukhov, V.N. et al. Algorithm for Three-Dimensional Reconstruction of Nonrigid Objects Using a Depth Camera. J. Commun. Technol. Electron. 65, 698–705 (2020). https://doi.org/10.1134/S1064226920060248

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1064226920060248

Search

Navigation