Abstract
The capacity of a robot to automatically adapt to new environments is crucial, especially in social robotics. Often, when these robots are deployed in home or office environments, they tend to fail because they lack the ability to adapt to new and continuously changing scenarios. In order to accomplish this task, robots must obtain new information from the environment, and then add it to their already learned knowledge. Deep learning techniques are often used to tackle this problem successfully. However, these approaches, complete retraining of the models, which is highly time-consuming. In this work, several strategies are tested to find the best way to include new knowledge in an already learned model in a deep learning pipeline, putting the spotlight on the time spent for this training. We tackle the localization problem in the long term with a deep learning approach and testing several retraining strategies. The results of the experiments are discussed and, finally, the best approach is deployed on a Pepper robot.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bui H M, Lech M, Cheng E, Neville K, Burnett I S (2016) Object recognition using deep convolutional features transformed by a recursive network structure. IEEE Access 4:10059–10066. https://doi.org/10.1109/ACCESS.2016.2639543
Dillmann R (2004) Teaching and learning of robot tasks via observation of human performance. Robot Auton Syst 47(2):109–116. https://doi.org/10.1016/j.robot.2004.03.005. http://www.sciencedirect.com/science/article/pii/S0921889004000417, robot Learning from Demonstration
Dillmann R, Rogalla O, Ehrenmann M, Zöliner R, Bordegoni M (2000) Learning robot behaviour and skills based on human demonstration and advice: The machine learning paradigm. In: Hollerbach JM, Koditschek DE (eds) Robotics research. Springer, London, pp 229–238
Eitel A, Springenberg J T, Spinello L, Riedmiller M A, Burgard W (2015) Multimodal deep learning for robust RGB-D object recognition. arXiv:1507.06821
Galindo C, Saffiotti A, Coradeschi S, Buschka P, Fernandez-Madrigal JA, Gonzalez J (2005) Multi-hierarchical semantic maps for mobile robotics. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, pp 2278–2283. https://doi.org/10.1109/IROS.2005.1545511
Goeddel R, Olson E (2016) Learning semantic place labels from occupancy grids using cnns. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3999–4004. https://doi.org/10.1109/IROS.2016.7759589
He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25, Curran Associates, Inc., pp 1097–1105. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
Li Z, Hoiem D (2016) Learning without forgetting. arXiv:1606.09282
Lowry S, Sünderhauf N, Newman P, Leonard J J, Cox D, Corke P, Milford M J (2016) Visual place recognition: A survey. IEEE Trans Robot 32(1):1–19. https://doi.org/10.1109/TRO.2015.2496823
Martinez-Gomez J, Garcia-Varea I, Cazorla M, Morell V (2015) Vidrilo: The visual and depth robot indoor localization with objects information dataset. Int J Robot Res 34(14):1681–1687. https://doi.org/10.1177/0278364915596058
Martínez-Gómez J, Morell V, Cazorla M, García-Varea I (2016) Semantic localization in the {PCL} library. Robot Auton Syst 75(Part B):641–648. https://doi.org/10.1016/j.robot.2015.09.006 https://doi.org/10.1016/j.robot.2015.09.006. http://www.sciencedirect.com/science/article/pii/S0921889015001943
Pronobis A, Mozos OM, Caputo B, Jensfelt P (2010) Multi-modal semantic place classification. Int J Robot Res 29(2-3):298–320. https://doi.org/10.1177/0278364909356483
Rangel JC, Cazorla M, García-Varea I, Martínez-Gómez J, Fromont É, Sebban M (2016) Scene classification based on semantic labeling. Adv Robot 30(11-12):758–769. https://doi.org/10.1080/01691864.2016.1164621
Romero-González C, Martínez-Gómez J, García-Varea I, Rodríguez-Ruiz L (2017) On robot indoor scene classification based on descriptor quality and efficiency. Expert Syst Appl 79:181–193. https://doi.org/10.1016/j.eswa.2017.02.040. http://www.sciencedirect.com/science/article/pii/S0957417417301318
Rubio F, Martínez-Gómez J, Flores MJ, Puerta JM (2016) Comparison between bayesian network classifiers and {SVMs} for semantic localization. Expert Syst Appl 64:434–443. https://doi.org/10.1016/j.eswa.2016.08.029. http://www.sciencedirect.com/science/article/pii/S0957417416304237
Ruiz-Sarmiento JR, Galindo C, Gonzalez-Jimenez J (2017) Building multiversal semantic maps for mobile robot operation. Knowl-Based Syst 119:257–272. http://www.sciencedirect.com/science/article/pii/S0950705116305184
Schwarz M, Schulz H, Behnke S (2015) Rgb-d object recognition and pose estimation based on pre-trained convolutional neural network features. In: 2015 IEEE international conference on robotics and automation (ICRA), pp 1329–1335. https://doi.org/10.1109/ICRA.2015.7139363
van der Maaten L, Hinton G (2008) Visualizing high-dimensional data using t-sne. J Mach Learn Res 9:2579–2605
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks?. arXiv:1411.1792
Zhou B, Lapedriza A, Khosla A, Oliva A, Torralba A (2017) Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence
Acknowledgments
This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. Edmanuel Cruz is funded by a Panamenian grant for PhD studies IFARHU & SENACYT 270-2016-207. Jose Carlos Rangel was supported by the National System of Research (SNI) of the SENACYT of Panama. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243. Thanks also to Nvidia for the generous donation of a Titan Xp and a Quadro P6000.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. Edmanuel Cruz is funded by a Panamenian grant for PhD studies IFARHU & SENACYT 270-2016-207. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243. Thanks to Nvidia also for the generous donation of a Titan Xp and a Quadro P6000.
Rights and permissions
About this article
Cite this article
Cruz, E., Rangel, J.C., Gomez-Donoso, F. et al. How to add new knowledge to already trained deep learning models applied to semantic localization. Appl Intell 50, 14–28 (2020). https://doi.org/10.1007/s10489-019-01517-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-019-01517-1