Abstract
In a cluttered environment in which objects are lying very closely to each other, the arranging motion is required before the robot attempts to grasp the target object. Thus, a robot must determine which motion to perform based on a given situation. This study presents an approach to learning a decision-making ability for the robot to grasp the target object after rearranging the surrounding objects obstructing the target object. The learning is performed in the virtual environment, and the image, which is an input of the deep Q-network, is preprocessed to directly apply the results of the learning to the real environment. That is, the difference between the two environments is minimized by making the states obtained from the virtual and real environments similar to each other. In addition, image preprocessing can be used to generalize the results of learning so that the robot can determine the appropriate actions to take when objects that were not used for learning are given. A hierarchical structure, which consists of high-level and low-level motion selectors, is used for the learning: the former determines the grasping or pushing actions while the latter determines how to perform such selected actions. The results of various experiments show that the proposed scheme is effective in grasping the target object in a cluttered environment without the need for any additional learning in the real world.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
S. Caldera, A. Rassau, and D. Chai, “Review of deep learning methods in robotic grasp detection,” Multimodal Technologies and Interaction, vol. 2, no. 3, Article number 57, 2018.
J. Bohg, A. Morales, T. Asfour, and D. Kragic, “Datadriven grasp synthesis—a survey,” IEEE Transactions on Robotics, vol. 30, no. 2, pp. 289–309, 2014.
J. Weisz and P. K. Allen, “Pose error robust grasping from contact wrench space metrics,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 557–562, 2012.
M. Gualtieri, A. Ten Pas, K. Saenko, and R. Platt, “High precision grasp pose detection in dense clutter,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 598–605, 2016.
J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea, and K. Goldberg, “Dex-Net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics,” Proc. of Robotics: Science and Systems, 2017.
S. Levine, P. Pastor, A. Krizhevsky, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” Int. Symposium on Experimental Robotics, vol. 37, no. 4-5, pp. 421–436, 2017.
J. Won, Y. Park, and I. H. Suh, “Scattering for robotic grasping in cluttered environments,” Int. Conf. on Ubiquitous Robots, 2018.
M. R. Dogar and S. S. Srinivasa, “A planning framework for non-prehensile manipulation under clutter and uncertainty,” Autonomous Robots, vol. 33, no. 3, pp. 217–236, 2012.
S. Amarjyoti, “Deep reinforcement learning for robotic manipulation-The state of the art,” arXiv preprint arXiv:1701.08878, 2017.
A. Zeng, S. Song, S. Welker, J. Lee, A. Rodriguez, and T. Funkhouser, “Learning synergies between pushing and grasping with self-supervised deep reinforcement learning,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 4238–4245, 2018.
R. Meyes, H. Tercan, S. Roggendorf, T. Thiele, C. Buscher, M. Obdenbusch, C. Brecher, S. Jeschke, and T. Meisen, “Motion planning for industrial robots using reinforcement learning,” Procedia CIRP, vol. 63, pp. 107–112, 2017.
R. Tang and H. Yuan, “Cyclic error correction based Qlearning for mobile robots navigation,” Int. Journal of Control, Automation, and Systems, vol. 15, no. 4, pp. 1790–1798, 2017.
L. Pinto and A. Gupta, “Supersizing self-supervision: Learning to grasp from 50K tries and 700 robot hours,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 3406–3413, 2016.
J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 23–30, 2017.
S. James, A. J. Davison, and E. Johns, “Transferring endto-end visuomotor control from simulation to real world for a multi-stage task,” Proc. of Conf. on Robot Learning, 2017.
K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhouke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 4243–4250, 2018.
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.
T. D. Kulkarni, K. Narasimhan, A. Saeedi, and J. Tenenbaum, “Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation,” Advances in Neural Information Processing Systems, pp. 3675–3683, 2016.
K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-CNN,” Proc. of IEEE Int. Conf. on Computer Vision, pp. 2961–2969, 2017.
Q. Lei, G. Chen, and M. Wisse, “Fast grasping of unknown objects using principal component analysis,” AIP Advances, vol. 7, no. 9, 095126, 2017.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by IITP grant funded by the Korea Government MSIT (No. 2018-0-00622).
Kuk-Hyun Ahn received his B.S. degree in Mechanical Engineering from Korea University in Seoul, Korea, in 2014. He is currently pursuing a Ph.D. degree from the School of Mechanical Engineering, Korea University. His research interests include manipulator control, reinforcement learning, and AI-based robot applications.
Jae-Bok Song received his B.S. and M.S. degrees in Mechanical Engineering from Seoul National University in Seoul, Korea, in 1983 and 1985, respectively. He received his Ph.D. degree in Mechanical Engineering from M.I.T. in 1992. He is a professor in the Department of Mechanical Engineering in Korea University since 1993. His research interests include robot safety, manipulator design and control, and AI-based robot applications. He is a senior member of IEEE.
Rights and permissions
About this article
Cite this article
Ahn, KH., Song, JB. Image Preprocessing-based Generalization and Transfer of Learning for Grasping in Cluttered Environments. Int. J. Control Autom. Syst. 18, 2306–2314 (2020). https://doi.org/10.1007/s12555-019-9618-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12555-019-9618-z