Skip to main content
SpringerLink
Log in

Handling crowdsourced data using state space discretization for robot learning and synthesizing physical skills

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

Intelligent physical skills are a fundamental element needed by robots to interact with the real world. Instead of learning from individual sources in single cases, continuous robot learning from crowdsourced mentors over long terms provides a practical path towards realizing ubiquitous robot physical intelligence. The mentors can be human drivers that teleoperate robots when their intelligence is not yet enough for acting autonomously. A large amount of sensorimotor data can be obtained constantly from a group of teleoperators, and processed by machine learning to continuously generate and improve the autonomous physical skills of robots. This paper presents a learning method that utilizes state space discretization to sustainably manage constantly collected data and synthesize autonomous robot skills. Two types of state space discretization have been proposed. Their advantages and limits are examined and compared. Simulation and physical tests of two object manipulation challenges are conducted to examine the proposed learning method. The capability of handling system uncertainty, sustainably managing high-dimensional state spaces, as well as synthesizing new skills or ones that have only been partly demonstrated are validated. The work is expected to provide a long-term and big-scale measure to produce advanced robot physical intelligence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Ackerman, E.: Agility robotics introduces cassie, a dynamic and talented robot delivery ostrich. IEEE Spectrum 2017, 28 (2017)

    Google Scholar 

  • Argall, B.D., Chernova, S., Veloso, M., Browning, B.: A survey of robot learning from demonstration. Robot. Autonomous Syst. 57(5), 469–483 (2009)

    Article  Google Scholar 

  • Benchmarks of ANN. https://github.com/erikbern/ann-benchmarks (2020). Accessed 16 Oct 2020

  • Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975)

    Article  Google Scholar 

  • Chung, M.J.Y., Forbes, M., Cakmak, M., Rao, R.P.N.: Accelerating imitation learning through crowdsourcing. In: IEEE international conference on robotics and automation, pp. 4777–4784 (2014)

  • Deisenroth, M.P., Neumann, G., Peters, J., et al.: A survey on policy search for robotics. Found. Trends Robot. 2(1–2), 1–142 (2013)

    Google Scholar 

  • Dijkstra, E.W.: A note on two problems in connexion with graphs. Numer. Math. 1(1), 269–271 (1959)

    Article  MathSciNet  Google Scholar 

  • Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world-wide web. Commun. ACM 54(4), 86–96 (2011)

    Article  Google Scholar 

  • Forbes, M., Chung, M., Cakmak, M., Rao, R.P.N.: Robot programming by demonstration with crowdsourced action fixes. In: The Second AAAI conference on human computation and crowdsourcing (2014)

  • Fu, C., Xiang, C., Wang, C., Cai, D.: Fast approximate nearest neighbor search with the navigating spreading-out graph. Proc. VLDB Endowment 12(5), 461–474 (2019)

    Article  Google Scholar 

  • Furukawa, N., Namiki, A., Taku, S., Ishikawa, M.: Dynamic regrasping using a high-speed multifingered hand and a high-speed vision system. In: IEEE international conference on robotics and automation (ICRA), pp. 181–187 (2006)

  • Geiger, D., Seedorf, S., Schulze, T., Nickerson, R.C., Schader, M.: Managing the crowd: towards a taxonomy of crowdsourcing processes. In: Americas conference on information systems (2011)

  • Gionis, A., Indyk, P., Motwani, R.: Similarity search in high dimensions via hashing. In: Proceedings of The 25th international conference on very large data bases, pp. 518–529 (1999)

  • Harwood, B., Drummond, T.: Fanng: Fast approximate nearest neighbour graphs. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 5713–5722 (2016)

  • Howe, J.: Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business. Crown Business, New York (2009)

    Google Scholar 

  • Ijspeert, A.J., Nakanishi, J., Schaal, S.: Learning attractor landscapes for learning motor primitives. Adv. Neural Inf. Process. Syst. 2003, 1547–1554 (2003)

    Google Scholar 

  • Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), 117–128 (2011)

    Article  Google Scholar 

  • Joe, S., Kuo, F.Y.: Remark on algorithm 659: implementing sobol’s quasirandom sequence generator. ACM Trans. Math. Softw. 29(1), 49–57 (2003)

    Article  MathSciNet  Google Scholar 

  • Johnson, M.K., Hasher, L.: Human learning and memory. Annu. Rev. Psychol. 38(1), 631–668 (1987)

    Article  Google Scholar 

  • Knight, W.: An AI-driven robot hand spent a hundred years teaching itself to rotate a cube. https://www.technologyreview.com/s/611724 (2018) Accessed 16 Oct 2020

  • Lee, I.S., Lau, H.Y.: Adaptive state space partitioning for reinforcement learning. Eng. Appl. Artif. Intell. 17(6), 577–588 (2004)

    Article  Google Scholar 

  • Li, W., Zhang, Y., Sun, Y., Wang, W., Li, M., Zhang, W., Lin, X.: Approximate nearest neighbor search on high dimensional data-experiments, analyses, and improvement. IEEE Trans. Knowl. Data Eng. 2019, 1 (2019)

    Google Scholar 

  • Lipton, J.I., Fay, A.J., Rus, D.: Baxter’s homunculus: virtual reality spaces for teleoperation in manufacturing. IEEE Robot. Autom. Lett. 3(1), 179–186 (2018)

    Article  Google Scholar 

  • Little, G., Chilton, L.B., Goldman, M., Miller, R.C.: Turkit: Human computation algorithms on mechanical turk. In: The 23rd Annual ACM symposium on user interface software and technology, pp. 57–66 (2010)

  • Malkov, Y.A., Yashunin, D.A.: Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Trans. Pattern Anal. Mach. Intell. 2018, 1 (2018)

    Google Scholar 

  • Mandlekar, A., Zhu, Y., Garg, A., Booher, J., Spero, M., Tung, A., Gao, J., Emmons, J., Gupta, A., Orbay, E., et al.: Roboturk: A crowdsourcing platform for robotic skill learning through imitation. In: Proceedings of The 2nd Conference on Robot Learning, vol. 87, pp. 879–893 (2018)

  • Nguyen-Tuong, D., Peters, J.: Online kernel-based learning for task-space tracking robot control. IEEE Trans. Neural Netw. Learn. Syst. 23(9), 1417–1425 (2012)

    Article  Google Scholar 

  • Niederreiter, H.: Low-discrepancy and low-dispersion sequences. J. Number Theory 30(1), 51–70 (1988)

    Article  MathSciNet  Google Scholar 

  • Quinn, A.J., Bederson, B.B.: Human computation: a survey and taxonomy of a growing field. In: The SIGCHI conference on human factors in computing systems, pp. 1403–1412 (2011)

  • Rasmussen, C.E., Williams, C.K.: Gaussian Processes for Machine Learning, vol. 1. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  • Sorokin, A., Berenson, D., Srinivasa, S.S., Hebert, M.: People helping robots helping people: crowdsourcing for grasping novel objects. In: IEEE/RSJ international conference on intelligent robots and systems, pp. 2117–2122 (2010)

  • Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  • Uther, W.T., Veloso, M.M.: Tree based discretization for continuous state space reinforcement learning. In: The annual conference on innovative applications of artificial intelligence (IAAI), pp. 769–774 (1998)

  • Van Hoof, H., Hermans, T., Neumann, G., Peters, J.: Learning robot in-hand manipulation with tactile features. In: The 15th IEEE-RAS international conference on humanoid robots, pp. 121–127 (2015)

  • von Ahn, L.: Human Computation. PhD Thesis, Carnegie Mellon University, Pittsburgh, PA (2005)

  • Wang, C., Zhao, Y., Lin, C.Y., Tomizuka, M.: Fast planning of well conditioned trajectories for model learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1460–1465 (2014)

  • Wang, C., Lin, C.Y., Tomizuka, M.: Statistical learning algorithms to compensate slow visual feedback for industrial robots. J. Dyn. Syst. Measure. Control 137(3), 031011 (2015)

    Article  Google Scholar 

  • Wang, C., Zhao, Y., Chen, Y., Tomizuka, M.: Nonparametric statistical learning control of robot manipulators for trajectory or contour tracking. Robot. Comput.-Integr. Manuf. 35, 96–103 (2015)

    Article  Google Scholar 

  • Zhang, T., McCarthy, Z., Jow, O., Lee, D., Chen, X., Goldberg, K., Abbeel, P.: Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1–8 (2018)

  • Zhao, L., Lawhorn, R., Wang, C., Lu, L., Ouyang, B.: Synthesis of robot hand skills powered by crowdsourced learning. In: IEEE International Conference on Mechatronics, pp. 211–216 (2019)

  • Zhao, L., Zhao, Y., Patil, S., Davies, D., Wang, C., Lu, L., Ouyang, B.: Robot composite learning and the nunchaku flipping challenge. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3160–3165 (2018)

  • Zhao, L., Lu, L., Wang, C.: Data-oriented state space discretization for crowdsourced robot learning of physical skills. ASME Lett. Dyn. Syst. Control 1, 2 (2020)

    Google Scholar 

Download references

Funding

This work is part of the project “CAREER: Enhancing Robot Physical Intelligence via Crowdsourced Surrogate Learning” funded by the National Science Foundation (Award number 1944069).

Author information

Authors and Affiliations

  1. New Jersey Institute of Technology, Electrical and Computer Engineering, 323 Martin Luther King Blvd, Newark, NJ, 07102, USA

    Leidi Zhao, Lu Lu & Cong Wang

Authors
  1. Leidi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, L., Lu, L. & Wang, C. Handling crowdsourced data using state space discretization for robot learning and synthesizing physical skills. Int J Intell Robot Appl 4, 390–402 (2020). https://doi.org/10.1007/s41315-020-00152-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-020-00152-1

Keywords

Search

Navigation