Skip to main content
SpringerLink
Log in

Object detection based on color and shape features for service robot in semi-structured indoor environment

  • Regular Paper
  • Published:
International Journal of Intelligent Robotics and Applications Aims and scope Submit manuscript

Abstract

Intelligent service robot is a challenging area of research that is rapidly expanding in our daily life. To meet robot’s object detection requirements in a messy surrounding, this paper provides a visual object detection method based on color and shape features. Firstly, a color hierarchical model and a multi-size filter are built to obtain initial object regions from scene image. Then a straight line-corner-arc strategy is presented to detect shape features. After comparing color and shape features with known-objects’ features stored in database, the detection scope is narrowed. So speeded up robust features algorithm is used to quickly match object features. The proposed method is tested by mobile robot in a semi-structured indoor environment. Finally, combined with the above steps, the total detection accuracy achieves 88.5% that confirms the feasibility of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Banerjee, J., Banerji, S., Ray, R., Shome, S.N.: Fuzzy based object shape recognition using translation, rotation and scale invariant parameters: an automatic approach. Springer, New Delhi (2016)

    Book  Google Scholar 

  • Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (SURF). Comput Vis Image Underst 110(3), 346–359 (2008)

    Article  Google Scholar 

  • Budiharto W, Gunawan AAS (2016) Development of coffee maker service robot using speech and face recognition systems using POMDP. In: First International Workshop on Pattern Recognition 2016, p. 1001110. International Society for Optics and Photonics

  • Busta M, Neumann L, Matas J (2017) Deep TextSpotter: an end-to-end trainable scene text localization and recognition framework. In: IEEE International Conference on Computer Vision 2017, pp. 2223–2231

  • Cheng MM, Liu Y, Hou Q, Bian J, Torr P, Hu SM, Tu Z (2016) HFS Hierarchical feature selection for efficient image segmentation.In: European Conference on Computer Vision 2016, Springer, Cham, pp. 867–882

    Chapter  Google Scholar 

  • Chung, H.-Y., Hou, C.-C., Chen, Y.-S.: Indoor intelligent mobile robot localization using fuzzy compensation and Kalman filter to fuse the data of gyroscope and magnetometer. IEEE Trans Industr Electron 62(10), 6436–6447 (2015)

    Article  Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on 2005, pp. 886–893

  • Ekvall, S., Kragic, D., Jensfelt, P.: Object detection and mapping for service robot tasks. Robotica 25(2), 175–187 (2007)

    Article  Google Scholar 

  • Fei-Fei, L., Fergus, R., Perona, P.: One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4), 594–611 (2006)

    Article  Google Scholar 

  • Ge L, Ju R, Ren T, Wu G (2015) Interactive RGB-D image segmentation using hierarchical graph cut and geodesic distance. In: Pacific Rim Conference on Multimedia 2015, Springer, Cham, pp. 114–124

    Google Scholar 

  • Harris C, Stephens M (1988) A combined corner and edge detector. In: Alvey vision conference 1988, vol. 50, p. 10.5244. Manchester, UK

  • Hernandez-Lopez, J.-J., Quintanilla-Olvera, A.-L., López-Ramírez, J.-L., Rangel-Butanda, F.-J., Ibarra-Manzano, M.-A., Almanza-Ojeda, D.-L.: Detecting objects using color and depth segmentation with Kinect sensor. Procedia Technol 3, 196–204 (2012)

    Article  Google Scholar 

  • Hossain, M.A., Kurnia, R., Nakamura, A., Kuno, Y.: Interactive object recognition through hypothesis generation and confirmation. IEICE TRANS Inf Syst 89(7), 2197–2206 (2006)

    Article  Google Scholar 

  • Jiang, L., Koch, A., Zell, A.: Object recognition and tracking for indoor robots using an RGB-D sensor, pp. 859–871. Springer, Cham (2016a)

    Google Scholar 

  • Jiang, L., Ye, Y., Xu, G.: An efficient curve detection algorithm. Optik Int J Light Electron Optics 127(1), 232–238 (2016b)

    Article  Google Scholar 

  • Jing, D., Xia, Y., Qifeng, Y.: Fast line segment detection based on edge connecting. Acta Optica Sinica 33(3), 0315003 (2013)

    Article  Google Scholar 

  • Karpathy A, Fei-Fei L (2015) Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2015, pp. 3128–3137

  • Kimoto M, Iio T, Shiomi M et al (2015) Improvement of object reference recognition through human robot alignment[C]//IEEE International Symposium on Robot and Human Interactive Communication.

  • Kumar V, Pandey S, Pal A, Sharma S (2016) Edge detection based shape identification arXiv:1604.02030.

  • Landsiedel, C., Rieser, V., Walter, M., Wollherr, D.: A review of spatial reasoning and interaction for real-world robotics. Adv Robot 31(5), 222–242 (2017)

    Article  Google Scholar 

  • Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. J Mach Learn Res 17(1), 1334–1373 (2016)

    MathSciNet  MATH  Google Scholar 

  • Liberda A, Lilja A, Langborn B, Lindström J, Kahl F, Larsson M (2016) Image segmentation and convolutional neural networks as tools for indoor scene understanding. Final report. Bachelor’s Thesis at Signals and systems - SSYX02-16-31

  • Maeyama, S., Takahashi, Y., Watanabe, K.: A solution to SLAM problems by simultaneous estimation of kinematic parameters including sensor mounting offset with an augmented UKF. Adv Robot 29(17), 1137–1149 (2015)

    Article  Google Scholar 

  • Mansur A, Sakata K, Rukhsana T, Kobayashi Y, Kuno Y (2008) Human robot interaction through simple expressions for object recognition. In: Robot and Human Interactive Communication, 2008. RO-MAN 2008. The 17th IEEE International Symposium on 2008, pp. 647–652

  • Matas J, Murino V, Rosenhahn B, Leal-Taixé L (2015) Holistic scene understanding (Dagstuhl Seminar 15081). In: Dagstuhl Reports 2015, vol. 2. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik

  • Mekhtiche, M., Bencherif, M., Algabri, M., Alsulaiman, M., Hedjar, R., Faisal, M., AlMutib, K.: Real time object detection tracking over a mobile platform. Indian J Sci Technol 8(22), 1–7 (2016)

    Google Scholar 

  • Muthugnanambika M, Padmavathi S (2017) Feature detection for color images using SURF. In: International Conference on Advanced Computing and Communication Systems 2017, pp. 1–4

  • Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vision 42(3), 145–175 (2001)

    Article  Google Scholar 

  • Oliva, A., Torralba, A.: The role of context in object recognition. Trends Cognit Sci 11(12), 520 (2007)

    Article  Google Scholar 

  • Pan, H.P., Li-Hua, H.U., Liu, Y.: A target recognition algorithm based on color clustering and seed filling. J Mech Elect Eng 2011, 7 (2011)

    Google Scholar 

  • Patraucean, V., Gurdjos, P., Gioi, R.G.V.: Joint A contrario ellipse and line detection. IEEE Trans Pattern Anal Mach Intell 39(4), 788–802 (2017)

    Article  Google Scholar 

  • Randelli, G., Bonanni, T.M., Iocchi, L., Nardi, D.: Knowledge acquisition through human–robot multimodal interaction. Intel Serv Robot 6(1), 19–31 (2013)

    Article  Google Scholar 

  • Rangel, J.C., Cazorla, M., García-Varea, I., Martínez-Gómez, J., Fromont, É., Sebban, M.: Scene classification based on semantic labeling. Adv Robot 30(11–12), 758–769 (2016)

    Article  Google Scholar 

  • Richtsfeld, A., Mörwald, T., Prankl, J., Zillich, M., Vincze, M.: Learning of perceptual grouping for object segmentation on RGB-D data. J Vis Commun Image Represent 25(1), 64–73 (2014)

    Article  Google Scholar 

  • Shen, L.L., Zhen, J.I.: Gabor wavelet selection and SVM classification for object recognition. Acta Autom Sin 35(4), 350–355 (2009)

    MathSciNet  MATH  Google Scholar 

  • Shi J (2002) Good features to track. In: Computer Vision and Pattern Recognition, 1994. Proceedings CVPR 1994, IEEE Computer Society Conference on 2002, pp. 593–600

  • Shotton, J., Blake, A., Cipolla, R.: Multiscale categorical object recognition using contour fragments. IEEE Trans Pattern Anal Mach Intell 30(7), 1270–1281 (2008)

    Article  Google Scholar 

  • Stark M, Goesele M, Schiele B (2009) A shape-based object class model for knowledge transfer. In: Computer Vision, 2009 IEEE 12th International Conference on 2009, pp. 373–380

  • Teo, C.L., Fermüller, C., Aloimonos, Y.: A Gestaltist approach to contour-based object recognition: combining bottom-up and top-down cues. Int J Robot Res 34(4–5), 627–652 (2015)

    Article  Google Scholar 

  • Turk, M.A., Morgenthaler, D.G., Gremban, K.D., Marra, M.: VITS-A vision system for autonomous land vehicle navigation. IEEE Trans Pattern Anal Mach Intell 10(3), 342–361 (1988)

    Article  Google Scholar 

  • Vincze M, Bajones M, Suchi M, Wolf D, Weiss A, Fischinger D, da la Puente P (2016) Learning and detecting objects with a mobile robot to assist older adults in their homes. In: European Conference on Computer Vision 2016, Springer, Cham, pp. 316–330

    Google Scholar 

  • Yang, C., Feinen, C., Tiebe, O., Shirahama, K., Grzegorzek, M.: Shape-based object matching using interesting points and high-order graphs. Pattern Recogn Lett 83(P3), 251–260 (2016)

    Article  Google Scholar 

  • Zaki, H.F.M., Shafait, F., Mian, A.: Learning a deeply supervised multi-modal RGB-D embedding for semantic scene and object category recognition[J]. Robot Auton Syst 92, 41–52 (2017)

    Article  Google Scholar 

  • Zhang Xd, Zhao QJ, Meng QX, Tu DW, Yi JG (2017) A new scene segmentation method based on color information for mobile robot in indoor environment. In: Wearable Sensors and Robots. Springer, Cham, pp. 353–363

    Google Scholar 

  • Zhang L, Jie-Xin PU, Fan QH (2008) Objects recognition based on genetic algorithm and BP neural network. Computer Engineering and Design

  • Zhao J, Liu H, Feng Y, Yuan S, Cai W (2015) BE-SIFT: A more brief and efficient sift image matching algorithm for computer vision. In: Computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing; pervasive intelligence and computing (CIT/IUCC/DASC/PICOM), 2015. IEEE International Conference on 2015, pp. 568–574

  • Zhao, Q., Li, X., Lu, J., Yi, J.: Monocular vision-based parameter estimation for mobile robotic painting. IEEE Trans Instrum Meas 68(10), 3589–3599 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant no. 61101177.

Author information

Authors and Affiliations

  1. School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China

    Haojie Li, Qijie Zhao, Xianfa Li & Xudong Zhang

Authors
  1. Haojie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qijie Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianfa Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xudong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qijie Zhao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Zhao, Q., Li, X. et al. Object detection based on color and shape features for service robot in semi-structured indoor environment. Int J Intell Robot Appl 3, 430–442 (2019). https://doi.org/10.1007/s41315-019-00113-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41315-019-00113-3

Keywords

Search

Navigation