Skip to main content

Advertisement

SpringerLink
Log in

Robotic grasp manipulation using evolutionary computing and deep reinforcement learning

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Intelligent object manipulation for grasping is a challenging problem for robots. Unlike robots, humans almost immediately know how to manipulate objects for grasping due to learning over the years. In this paper, we have developed learning-based pose estimation by decomposing the problem into both position and orientation learning. More specifically, for grasp position estimation, we explore three different methods such as genetic algorithm (GA)-based optimization method to minimize error between calculated image points and predicted end-effector (EE) position, a regression-based method (RM) where collected data points of robot EE and image points have been regressed with a linear model, a pseudoinverse (PI) model which has been formulated in the form of a mapping matrix with robot EE position and image points for several observations. Further for grasp orientation learning, we develop a deep reinforcement learning (DRL) model which we name as grasp deep Q-network (GDQN) and benchmarked our results with Modified VGG16 (MVGG16). Rigorous experimentation shows that due to inherent capability of producing very high-quality solutions for optimization problems and search problems, GA-based predictor performs much better than the other two models for position estimation. For orientation, learning results indicate that off policy learning through GDQN outperforms MVGG16, since GDQN architecture is specially made suitable for the reinforcement learning. Experimentation based on our proposed architectures and algorithms shows that the robot is capable of grasping nearly all rigid body objects having regular shapes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Bohg J, Morales A, Asfour T, Kragic D (2014) Data-driven grasp synthesis-a survey. IEEE Trans Robot 30(2):289–309. https://doi.org/10.1109/TRO.2013.2289018

    Article  Google Scholar 

  2. Depierre A, Dellandrea E, Chen L (2018) Jacquard: a largescale dataset for robotic grasp detection. In: Proceedings of the 2018 IEEE/RSJ international conference on Intelligent Robots and Systems (IROS), pp 3511–3516 (2018)

  3. Fu J, Lin Z, Liu M, Leonard N, Feng J, Chua TS (2016) Deep q-networks for accelerating the training of deep neural networks

  4. Goldberg DE (1994) Genetic algorithms in search, optimization and machine learning, 2nd edn. Addison-Wesley, Boston

    Google Scholar 

  5. Huber PJ (2011) Robust statistics. Springer, Berlin, pp 1248–1251

    Google Scholar 

  6. Jiao J, Yuan L, Tang W, Deng Z, Wu Q (2017) A post-rectification approach of depth images of kinect v2 for 3d reconstruction of indoor scenes. ISPRS Int J Geo-Information 6:349. https://doi.org/10.3390/ijgi6110349

    Article  Google Scholar 

  7. Ju Z, Yang C, Ma H (2014) Kinematics modeling and experimental verification of baxter robot. In: Proceedings of the 33rd Chinese control conference, pp. 8518–8523 (2014). https://doi.org/10.1109/ChiCC.2014.6896430

  8. Kalashnikov D, Irpan A, Pastor P, Ibarz J, Herzog A, Quillen D, Holly E, Kalakrishnan M, Vanhoucke V, Levine S (2018) Qt-opt: scalable deep reinforcement learning for vision-based robotic manipulation (2018). arXiv:1806.10293

  9. Konidaris G, Kuindersma S, Grupen R, Barto A (2012) Robot learning from demonstration by constructing skill trees. Int J Robot Res 31(3):360–375. https://doi.org/10.1177/0278364911428653

    Article  Google Scholar 

  10. Kragic D, Christensen HAF (2002) Survey on visual servoing for manipulation. Comput Vis Act Percept Lab Fiskartorpsv 15:2002

    Google Scholar 

  11. Langlois J, Mouchère H, Normand N, Viard-Gaudin C (2018) 3d orientation estimation of industrial parts from 2d images using neural networks. In: ICPRAM

  12. Lenz I, Lee H, Saxena A (2015) Deep learning for detecting robotic grasps. Int J Robot Res 34(4–5):705–724. https://doi.org/10.1177/0278364914549607

    Article  Google Scholar 

  13. Levine S, Pastor P, Krizhevsky A, Ibarz J, Quillen D (2018) Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. Int J Robot Res 37(4–5):421–436. https://doi.org/10.1177/0278364917710318

    Article  Google Scholar 

  14. Li M, Hang K, Kragic D, Billard A (2016) Dexterous grasping under shape uncertainty. Robot Auton Syst 75:352–364

    Article  Google Scholar 

  15. Lin T, Maire M, Belongie SJ, Bourdev LD, Girshick RB, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. CoRR arXiv:1405.0312

  16. Mahler J, Liang J, Niyaz S, Laskey M, Doan R, Liu X, Ojea JA, Goldberg K (2017) Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. CoRR arXiv:1703.09312

  17. Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs, 3rd edn. Springer, Berlin

    Book  Google Scholar 

  18. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller MA (2013) Playing atari with deep reinforcement learning. CoRR arXiv:1312.5602

  19. Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J, Bellemare M, Graves A, Riedmiller M, Fidjeland A, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518:529–33. https://doi.org/10.1038/nature14236

    Article  Google Scholar 

  20. Nandi GC, Agarwal P, Gupta P, Singh A (2018) Deep learning based intelligent robot grasping strategy. In: 2018 IEEE 14th International conference on control and automation (ICCA), pp. 1064–1069 (2018). https://doi.org/10.1109/ICCA.2018.8444265

  21. Peters J, Lee D, Kober J, Nguyen-Tuong D, Bagnell J, Schaal S (2017) Robot learning, 2nd edn., chap. 15. Springer International Publishing, pp 357–394

  22. Pinto L, Gupta A (2015) Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours. CoRR arXiv:1509.06825

  23. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. CoRR arXiv:1804.02767

  24. Ren S, He K, Girshick RB, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. CoRR arXiv:1506.01497

  25. Ritter H, Haschke R (2015) Hands, dexterity, and the brain. In: PhD Cheng G (ed) Humanoid robotics and neuroscience: science, engineering and society, 3. CRC Press, Boca Raton

    Google Scholar 

  26. Sahbani A, El-Khoury S, Bidaud P (2012) An overview of 3d object grasp synthesis algorithms. Robot Auton Syst 60(3):326–336

    Article  Google Scholar 

  27. Saxena A, Driemeyer J, Ng AY (2009) Learning 3-d object orientation from images. In: 2009 IEEE International conference on robotics and automation, pp. 794–800 (2009). https://doi.org/10.1109/ROBOT.2009.5152855

  28. Shukla P, Nandi GC (2019) Robotized grasp: grasp manipulation using evolutionary computing. In: Proceedings of 2019 international conference on electrical, electronics and computer engineering (UPCON) (2019). DOI 978-1-7281-3455-0/19/\$31.002019IEEE

  29. Tremblay J, To T, Sundaralingam B, Xiang Y, Fox D, Birchfield S (2018) Deep object pose estimation for semantic robotic grasping of household objects. CoRR arXiv:1809.10790

  30. Viereck U, ten Pas A, Saenko K, Jr, RP (2017) Learning a visuomotor controller for real world robotic grasping using easily simulated depth images. CoRR arXiv:1706.04652

  31. Zhu H, Gupta A, Rajeswaran A, Levine S, Kumar V (2018) Dexterous manipulation with deep reinforcement learning: efficient, general, and low-cost. CoRR arXiv:1810.06045

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Information Technology Allahabad, Allahabad, Uttar Pradesh, India

    Priya Shukla & G C Nandi

  2. Indian Institute of Technology (Indian School of Mines), Dhanbad, Dhanbad, India

    Hitesh Kumar

Authors
  1. Priya Shukla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hitesh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G C Nandi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Priya Shukla.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Hitesh Kumar—Summer Intern at Indian Institute of Information Technology Allahabad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shukla, P., Kumar, H. & Nandi, G.C. Robotic grasp manipulation using evolutionary computing and deep reinforcement learning. Intel Serv Robotics 14, 61–77 (2021). https://doi.org/10.1007/s11370-020-00342-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-020-00342-7

Keywords

Search

Navigation