Skip to main content
SpringerLink
Log in

Toward a computer vision-based wayfinding aid for blind persons to access unfamiliar indoor environments

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Independent travel is a well-known challenge for blind and visually impaired persons. In this paper, we propose a proof-of-concept computer vision-based wayfinding aid for blind people to independently access unfamiliar indoor environments. In order to find different rooms (e.g. an office, a laboratory, or a bathroom) and other building amenities (e.g. an exit or an elevator), we incorporate object detection with text recognition. First, we develop a robust and efficient algorithm to detect doors, elevators, and cabinets based on their general geometric shape, by combining edges and corners. The algorithm is general enough to handle large intra-class variations of objects with different appearances among different indoor environments, as well as small inter-class differences between different objects such as doors and door-like cabinets. Next, to distinguish intra-class objects (e.g. an office door from a bathroom door), we extract and recognize text information associated with the detected objects. For text recognition, we first extract text regions from signs with multiple colors and possibly complex backgrounds, and then apply character localization and topological analysis to filter out background interference. The extracted text is recognized using off-the-shelf optical character recognition software products. The object type, orientation, location, and text information are presented to the blind traveler as speech.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Arditi, A., Brabyn, J.: Signage and wayfinding. In: Silverstone, B., Lang, M.A., Rosenthal, B., Faye, E. (eds.) The Lighthouse Handbook on Visual Impairment and Vision Rehabilitation, Oxford University Press, New York (2000)

  2. Anguelov, D., Koller, D., Parker, E., Thrun, S.: Detecting and modeling doors with mobile robots. In: Proceedings of the IEEE international conference on robotics and automation (2004)

  3. Baker, A.: Blind Man is Found Dead in Elevator Shaft. The New York Times, City Room (2010)

  4. Biederman, I.: Recognition-by-components: a theory of human image understanding. Psychol. Rev. 94 (1987)

  5. Blind Sight: A camera for visually impaired people. http://accessability.blogspot.com/2008/10/blind-sight-camera-for-visually.html

  6. Canny J.: A computational approach to edge detection. IEEE Trans. Pattern Analy. Mach. Intell. PAMI 8, 679–698 (1986)

    Article  Google Scholar 

  7. Chen, X., Yuille, A.: Detecting and reading text in natural scenes, CVPR (2004)

  8. Chen, Z., Birchfield, S.: Visual detection of lintel-occluded doors from a single image. IEEE Computer Society workshop on visual localization for mobile platforms (2008)

  9. Chen, C., Tian, Y.: Door detection via signage context-based hierarchical compositional model. 2nd workshop on use of context in video processing (UCVP) (2010)

  10. Dakopoulos D., Bourbakis N.G.: Wearable obstacle avoidance electronic travel aids for blind: a survey. IEEE IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 40(1), 25–35 (2010)

    Article  Google Scholar 

  11. Dinh, V., Chun, S., Cha, S., Ryu, H., Sull, S.: An efficient method for text detection in video based on stroke width similarity. Asian conference on computer vision (ACCV) (2007)

  12. Divvala, S., Hoiem, D., Hays, J., Efros, A., Hebert, M.: An empirical study of context in object detection. In: Proceedings of IEEE CVPR (2009)

  13. Dubey, P.: Edge based text detection for multi-purpose application. Int. Conf. Signal Process. 4 (2006)

  14. Everingham, M., Thomas, B., Troscianko, T.: Wearable mobility aid for low vision using scene classification in a Markov random field model framework. Int. J. Hum. Comput. Interact. 15(2) (2003)

  15. Giudice N., Legge G.: Blind navigation and the role of technology. In: Helal, A.A., Mokhtari, M., Abdulrazak, B. (eds) The engineering handbook of smart technology for aging, disability, and independence., Wiley, Hoboken (2008)

    Google Scholar 

  16. He, X., Yung, N.: Corner detector based on global and local curvature properties. Opt. Eng. 47(5) (2008)

  17. Hensler, J., Blaich, M., Bittel, O.: Real-time door detection based on adaboost learning algorithm. International conference on research and education in robotics, Eurobot (2009)

  18. Ivanchenko, V., Coughlan J., Shen, H.: Crosswatch: a camera phone system for orienting visually impaired pedestrians at traffic intersections. 11th international conference on computers helping people with special needs (ICCHP ’08) (2008)

  19. Kasar, T., Kumar, J., Ramakrishnan, A.G.: Font and background color independent text binarization. Second international workshop on camera-based document analysis and recognition (2007)

  20. Kim, D., Nevatia, R.: A method for recognition and localization of generic objects for indoor navigation. In: ARPA image understanding workshop (1994)

  21. Kreiman, G.: Biological object recognition. Scholarpedia 3(6), 2667. http://www.scholarpedia.org/article/Biological_object_recognition (2008)

  22. Liu, C., Wang, C., Dai, R.: Text detection in images based on unsupervised classification of edge-based features. International conference on document analysis and recognition (2005)

  23. Liu, Q., Jung, C., Moon, Y.: Text segmentation based on stroke filter. In: Proceedings of international conference on multimedia (2006)

  24. Luo, J., Singhal, A., Zhu, W.: Natural object detection in outdoor scenes based on probabilistic spatial context models. International conference on multimedia and expo (2003)

  25. Manduchi, R., Coughlan, J., Ivanchenko, V.: Search strategies of visually impaired persons using a camera phone wayfinding system. 11th international conference on computers helping people with special needs (ICCHP ’08) (2008)

  26. Munoz-Salinas, R., Aguirre, E., Garcia-Silvente, M., Gonzalez, A.: Door-detection using computer vision and fuzzy logic. In: Proceedings of the 6th WSEAS international conference on mathematical methods and computational techniques in electrical engineering (2004)

  27. Murillo, A., Kosecka, J., Guerrero, J., Sagues, C.: Visual door detection integrating appearance and shape cues. Robot. Auton. Syst. (2008)

  28. National Research Council. Electronic travel aids: new directions for research. Working group on mobility aids for the visually impaired and blind, ed. C.o. vision. National Academy Press, Washington, DC, p. 107 (1986)

  29. Nikolaou, N., Papamarkos, N.: Color reduction for complex document images. Int. J. Imaging Syst. Technol. 19 (2009)

  30. Oliva A., Torralba A.: The role of context in object recognition. Trends Cognit. Sci. 11, 520–527 (2007)

    Article  Google Scholar 

  31. Paletta, L., Greindl, C.: Context based object detection from video. In: Proceedings of international conference on computer vision systems (2003)

  32. Pradeep, V., Medioni, G., Weiland, J.: Piecewise planar modeling for step detection using stereo vision. Workshop on computer vision applications for the visually impaired (2008)

  33. Shen, H., Coughlan, J.: Grouping using factor graphs: an approach for finding text with a camera phone. Workshop on graph-based representations in pattern recognition (2007)

  34. Shivakumara, P., Huang, W., Tan, C.: An efficient edge based technique for text detection in video frames. The eighth IAPR workshop on document analysis systems (2008)

  35. Stoeter, S., Mauff, F., Papanikolopoulos, N.: Realtime door detection in cluttered environments. In: Proceedings of the 15th IEEE international symposium on intelligent control (2000)

  36. Tian, Y., Yi, C., Arditi, A.: Improving computer vision-based indoor wayfinding for blind persons with context information. 12th international conference on computers helping people with special needs (ICCHP) (2010)

  37. Tian, Y., Yang, X., Arditi, A.: Computer vision-based door detection for accessibility of unfamiliar environments to blind persons. 12th international conference on computers helping people with special needs (ICCHP) (2010)

  38. Torralba A.: Contextual priming for object detection. Int. J. Comput. Vision 53(2), 169–191 (2003)

    Article  Google Scholar 

  39. Tran, H., Lux, A., Nguyen, H., Boucher, A.: A novel approach for text detection in images using structural features. The 3rd international conference on advances in pattern recognition (2005)

  40. Seeing with sound—the vOICe. http://www.seeingwithsound.com

  41. Wan, M., Zhang, F., Cheng, H., Liu, Q.: Text localization in spam image using edge features. International conference on communications, circuits and system (2008)

  42. Wong, E., Chen, M.: A new robust algorithm for video text extraction. Pattern Recognit. 36 (2003)

  43. Yang, X., Tian, Y.: Robust door detection in unfamiliar environments by combining edge and corner features. 3rd workshop on computer vision applications for the visually impaired (CVAVI) (2010)

  44. Zandifar, A., Duraiswami, R., Chahine, A., Davis, L.: A video based interface to textual information for the visually impaired. In: Proceedings of IEEE 4th international conference on multimodal interfaces (2002)

  45. Robust Reading Dataset. http://algoval.essex.ac.uk/icdar/Datasets.html

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, The City College, and Graduate Center, City University of New York, New York, NY, 10031, USA

    YingLi Tian

  2. Electrical Engineering Department, The City College, City University of New York, New York, NY, 10031, USA

    Xiaodong Yang

  3. The Graduate Center, City University of New York, New York, NY, 10036, USA

    Chucai Yi

  4. Visibility Metrics LLC, Chappaqua, NY, 10514, USA

    Aries Arditi

Authors
  1. YingLi Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chucai Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aries Arditi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to YingLi Tian.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tian, Y., Yang, X., Yi, C. et al. Toward a computer vision-based wayfinding aid for blind persons to access unfamiliar indoor environments. Machine Vision and Applications 24, 521–535 (2013). https://doi.org/10.1007/s00138-012-0431-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-012-0431-7

Keywords

Search

Navigation