Skip to main content
SpringerLink
Log in

Learning to Control a Quadcopter Qualitatively

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Qualitative modeling allows autonomous agents to learn comprehensible control models, formulated in a way that is close to human intuition. By abstracting away certain numerical information, qualitative models can provide better insights into operating principles of a dynamic system in comparison to traditional numerical models. We show that qualitative models, learned from numerical traces, contain enough information to allow motion planning and path following. We demonstrate our methods on the task of flying a quadcopter. A qualitative control model is learned through motor babbling. Training is significantly faster than training times reported in papers using reinforcement learning with similar quadcopter experiments. A qualitative collision-free trajectory is computed by means of qualitative simulation, and executed reactively while dynamically adapting to numerical characteristics of the system. Experiments have been conducted and assessed in the V-REP robotic simulator.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bratko, I.: An assessment of machine learning methods for robotic discovery. J. Comput. Inf. Technol. - CIT 16(4), 247–254 (2008). https://doi.org/10.2498/cit.1001392

    Article  Google Scholar 

  2. Bratko, I.: Autonomous discovery of abstract concepts by a robot. In: Adaptive and Natural Computing Algorithms, pp. 1–11. Springer Lecture Notes. https://doi.org/10.1007/978-3-642-20282-7_1 (2011)

  3. De Kleer, J.: Multiple representations of knowledge in a mechanics problem solver. In: Inproceedings of the International Joint Conference on Artificial Intelligence, pp. 299–304. https://doi.org/10.1016/b978-1-4832-1447-4.50009-2 (1977)

  4. De Kleer, J., Brown, J. S.: A qualitative physics confluences. Artif. Intell. 24(1-3), 7–83 (1984)

    Article  Google Scholar 

  5. Dierks, T., Jagannathan, S.: Output feedback control of a quadrotor uav using neural networks. IEEE Trans. Neural Netw. 21(1), 50–66 (2010). https://doi.org/10.1109/tnn.2009.2034145

    Article  Google Scholar 

  6. Domingos, D., Camargo, G., Gomide, F.: Autonomous fuzzy control and navigation of quadcopters. FAC-PapersOnLine 49(5), 73–78. https://doi.org/10.1016/j.ifacol.2016.07.092. 4th IFAC Conference on Intelligent Control and Automation SciencesICONS 2016 (2016)

  7. Došilović, F. K., Brčić, M., Hlupić, N.: Explainable Artificial Intelligence: a Survey. In: 2018 41St International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 210–215 (2018)

  8. Forbus, K. D.: Qualitative process theory. Artif. Intell. 24(1-3), 85–168 (1984). https://doi.org/10.21236/ada465743

    Article  Google Scholar 

  9. Forbus, K. D.: Introducing actions into qualitative simulation. In: Proceedings of the 11th International Joint Conference on Artificial Intelligence - Volume 2, IJCAI’89, pp. 1273–1278. Morgan Kaufmann Publishers Inc (1989)

  10. Forbus, K. D.: Qualitative Representations: How People Reason and Learn about the Continuous World, 1st edn. MIT Press. https://doi.org/10.7551/mitpress/11578.001.0001 (2019)

  11. Gandhi, D., Pinto, L., Gupta, A.: Learning to Fly by Crashing. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1344–1352. https://doi.org/10.1109/iros.2017.8206247 (2017)

  12. Garcia, A., Mittal, S., Kiewra, E., Ghose, K.: A Convolutional Neural Network Vision System Approach to Indoor Autonomous Quadrotor Navigation. In: 2019 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 1344–1352. https://doi.org/10.1109/icuas.2019.8798183 (2019)

  13. Goebel, R., Chander, A., Holzinger, K., Lecue, F., Akata, Z., Stumpf, S., Kieseberg, P., Holzinger, A.: Explainable AI: The new 42? In: Machine Learning and Knowledge Extraction. CD-MAKE 2018. Lecture Notes in Computer Science, vol. 11015, pp. 295–303. Springer, Cham . https://doi.org/10.1007/978-3-319-99740-7_21 (2018)

  14. Hoffmann, G. M., Huang, H., Wasl, S. L., Tomlin, E. C. J.: Quadrotor Helicopter Flight Dynamics and Control: Theory and Experiment. In: In Proc. of the AIAA Guidance, Navigation, and Control Conference. https://doi.org/10.2514/6.2007-6461 (2007)

  15. Hogge, J. C.: Compiling plan operators from domains expressed in qualitative process theory. In: Proceedings of the Sixth National Conference on Artificial Intelligence - Volume 1, AAAI’87, pp. 229–233. AAAI Press (1987)

  16. Hwangbo, J., Sa, I., Siegwart, R., Hutter, M.: Control of a quadrotor with reinforcement learning. IEEE Robot. Autom. Lett. 2(4), 2096–2103 (2017). https://doi.org/10.1109/lra.2017.2720851

    Article  Google Scholar 

  17. Koch, W., Mancuso, R., West, R., Bestavros, A.: Reinforcement learning for UAV attitude control. ACM Transactions on Cyber-Physical Systems, 3. https://doi.org/10.1145/3301273 (2018)

  18. Košmerlj, A., Bratko, I., žabkar, J.: Embodied concept discovery through qualitative aaction models. International Journal of Uncertainty. Fuzz. Knowl.-Based Syst. 19(03), 453–475 (2011). https://doi.org/10.1142/s0218488511007088

    Article  Google Scholar 

  19. Kouris, A., Bouganis, C.: Learning to Fly by Myself: a Self-Supervised Cnn-Based Approach for Autonomous Navigation. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). https://doi.org/10.1109/iros.2018.8594204 (2018)

  20. Kuipers, B.: Qualitative simulation. Artif. Intell. 29(3), 289–338 (1986). https://doi.org/10.1016/b978-1-4832-1447-4.50018-3

    Article  MathSciNet  Google Scholar 

  21. Kuipers, B.: Qualitative reasoning: Modeling and simulation with incomplete knowledge. MIT Press, Cambridge. https://doi.org/10.1063/1.4823391 (1994)

  22. Kunze, L., Beetz, M.: Envisioning the qualitative effects of robot manipulation actions using simulation-based projections. Artif. Intell. 247, 352–380 (2017). https://doi.org/10.1016/j.artint.2014.12.004

    Article  MathSciNet  Google Scholar 

  23. Leban, G., žabkar, J., Bratko, I.: An experiment in robot discovery with ILP. In: Proceedings of the 18th International Conference on Inductive Logic Programming, pp. 77–90. Springer, Berlin. https://doi.org/10.1007/978-3-540-85928-4_10 (2008)

  24. Loquercio, A., Maqueda, A. I., del Blanco, C. R., Scaramuzza, D.: Dronet: Learning to fly by driving. IEEE Robot. Autom. Lett. 3(2), 1088–1095 (2018). https://doi.org/10.1109/lra.2018.2795643

    Article  Google Scholar 

  25. Michie, D.: Machine learning in the next five years. In: Proceedings of the 3rd European Conference on European Working Session on Learning, EWSL’88, pp. 107–122 (1988)

  26. Mo, H., Farid, G.: Nonlinear and adaptive intelligent control techniques for quadrotor uav – a survey. Asian J. Control 21(2), 989–1008 (2019). https://doi.org/10.1002/asjc.1758

    Article  MathSciNet  Google Scholar 

  27. Mugan, J., Kuipers, B.: Autonomous learning of high-level states and actions in continuous environments. IEEE Trans. Auton. Ment. Dev. 4(1), 70–86 (2012). https://doi.org/10.1109/tamd.2011.2160943

    Article  Google Scholar 

  28. Quinlan, J. R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986). https://doi.org/10.1007/bf00116251

    Google Scholar 

  29. Raffo, G. V., Ortega, M., Rubio, F.: Backstepping/nonlinear H\(\infty \) control for path tracking of a quadrotor unmanned aerial vehicle. In: Proceedings of the American Control Conference, pp. 3356–3361. https://doi.org/10.1109/ACC.2008.4587010 (2008)

  30. Rohmer, E., Singh, S. P. N., Freese, M.: V-REP: a Versatile and Scalable Robot Simulation Framework. In: Proc. of the International Conference on Intelligent Robots and Systems (IROS), pp. 1321–1326. https://doi.org/10.1109/iros.2013.6696520 (2013)

  31. Rubí, B., Pérez, R., Morcego, B.: A survey of path following control strategies for uavs focused on quadrotors. Journal of Intelligent & Robotic Systems. https://doi.org/10.1007/s10846-019-01085-z(2019)

  32. Sammut, C., Yik, T.F.: Multistrategy Learning for Robot Behaviours, pp. 457–476. Springer, Berlin. https://doi.org/10.1007/978-3-642-05177-7_23 (2010)

  33. Sanchez-Lopez, J., Wang, M., Olivares-Mendez, M., Molina, M., Voos, H.: A real-time 3d path planning solution for collision-free navigation of multirotor aerial robots in dynamic environments. J. Intell. Robot. Syst. 93, 33–53 (2018). https://doi.org/10.1007/s10846-018-0809-5

    Article  Google Scholar 

  34. Sun, K., Liu, L., Qiu, J., Feng, G.: Fuzzy adaptive finite-time fault-tolerant control for strict-feedback nonlinear systems. IEEE Transactions on Fuzzy Systems. pp. 1–1. https://doi.org/10.1109/TFUZZ.2020.2965890 (2020)

  35. Sun, K., Mou, S., Qiu, J., Wang, T., Gao, H.: Adaptive fuzzy control for nontriangular structural stochastic switched nonlinear systems with full state constraints. IEEE Trans. Fuzzy Syst. 27(8), 1587–1601 (2019). https://doi.org/10.1109/TFUZZ.2018.2883374

    Article  Google Scholar 

  36. Sun, K., Qiu, J., Karimi, H. R., Gao, H.: A novel finite-time control for nonstrict feedback saturated nonlinear systems with tracking error constraint. IEEE Transactions on Systems, Man, and Cybernetics: Systems, pp. 1–12. https://doi.org/10.1109/TSMC.2019.2958072 (2019)

  37. Troha, M., Bratko, I.: Qualitative learning of object pushing by a robot. In: 25th International Workshop on Qualitative Reasoning, pp. 175–180. Barcelona, Spain (2011)

  38. Šoberl, D., Bratko, I.: Reactive motion planning with qualitative constraints. In: Advances in Artificial Intelligence: From Theory to Practice, Part I, pp. 41–50. Springer International Publishing. https://doi.org/10.1007/978-3-319-60042-0_5 (2017)

  39. Šoberl, D., žabkar, J., Bratko, I.: Qualitative planning of object pushing by a robot. In: Lecture Notes in Computer Science, vol. 9384, pp. 410–419. Springer, Berlin. https://doi.org/10.1007/978-3-319-25252-0_44 (2015)

  40. Šuc, D., Bratko, I: Induction of qualitative trees. In: Machine Learning: ECML 2001: 12th European Conference on Machine Learning Freiburg, Germany, 2001 Proceedings, pp. 442–453. Springer, Berlin. https://doi.org/10.1007/3-540-44795-4_38 (2001)

  41. Šuc, D., Bratko, I., Sammut, C.: Learning to fly simple and robust. In: Machine Learning: ECML 2004, vol. 3201, pp. 407–418. Springer, Berlin. https://doi.org/10.1007/978-3-540-30115-8_38 (2004)

  42. žabkar, J., Bratko, I., Demšar, J.: Learning qualitative models through partial derivatives by padé. In: Proceedings of the 21st Annual Workshop on Qualitative Reasoning, pp. 193–202 (2007)

  43. žabkar, J., Možina, M., Bratko, I., Demšar, J.: Learning qualitative models from numerical data. Artif. Intell. 175(9-10), 1604–1619 (2011). https://doi.org/10.1016/j.artint.2011.02.004

    Article  MathSciNet  Google Scholar 

  44. Waslander, S. L., Hoffmann, G. M., Tomlin, C. J.: Multi-Agent Quadrotor Testbed Control Design: Integral Sliding Mode Vs. Reinforcement Learning. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3712–3717. https://doi.org/10.1109/iros.2005.1545025 (2005)

  45. Wellman, M. P.: Qualitative Simulation with Multivariate Constraints. In: Second International Conference on Principles of Knowledge Representation and Reasoning, pp. 547–557. Morgan Kaufmann (1991)

  46. Wiley, T., Sammut, C., Bratko, I.: Qualitative planning with quantitative constraints for online learning of robotic behaviours. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence, pp. 2578–2584 (2014)

  47. Wiley, T., Sammut, C., Bratko, I.: A planning and learning hierarchy using qualitative reasoning for the on-line acquisition of robotic behaviors. Adv. Cogn. Syst. 4, 93–112 (2016)

    Google Scholar 

Download references

Acknowledgements

This work was partially funded by Slovenian Research Agency (ARRS) as part of research programme AI and Intelligent Systems.

Author information

Authors and Affiliations

  1. Faculty of Computer and Information Science, University of Ljubljana, Večna pot 113, SI-1000, Ljubljana, Slovenia

    Domen Šoberl, Ivan Bratko & Jure Žabkar

Authors
  1. Domen Šoberl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ivan Bratko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jure Žabkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Domen Šoberl.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Šoberl, D., Bratko, I. & Žabkar, J. Learning to Control a Quadcopter Qualitatively. J Intell Robot Syst 100, 1097–1110 (2020). https://doi.org/10.1007/s10846-020-01228-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-020-01228-7

Keywords

Search

Navigation