Skip to main content
SpringerLink
Log in

TapCalculator: nonvisual touchscreen calculator for visually impaired people preliminary user study

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

TapCalculator is a novel, non-visual touchscreen calculator that uses simple gestures to represent digits and operations with the support of audio feedback and is designed specifically for visually impaired users. It enables users to do basic calculations without visually locating buttons on a screen or having to know braille or learning complex coding for digits. It uses a series of tapping and swiping to perform arithmetic operations. This paper lays the groundwork for devising, improving, and implementing intuitive, user-friendly application to meet the needs of individuals with visual impairments. This paper is supported by primary-source research conducted by the author, including a review of several touchscreen calculators, using a small sample of visually impaired users. Seven participants were asked to compare the TapCalculator to two other virtual calculators, BrailleTap and TalkBack. From this preliminary usability study, quantitative findings indicate that blind users enter digits and arithmetic operations on the TapCalculator faster and less power consumption than the others due to TapCalcultator’s emphasis on directional gestures and allows users to tap with a maximum of 5 fingers. BrailleTap was shown to be more accurate than both TapCalculator and TalkBack, suggesting areas for improvement and further study. Through interviews with participants, qualitative results suggest that, for blind users attuned to braille, directional gestures are more accessible than locating the position of a touchscreen button. This paper suggests improvements to enhance the user experience of the TapCalculator, which is still in its infancy, and other virtual calculators for the visually impaired users.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Total error rate (TER) is the rate of error occurred while typing equations including numbers, operations, and the equal sign.

References

  1. Tech Connections. (2004). Assistive technology quick reference series: Accessible calculators. Retrieved from http://www.catea.gatech. edu/quickrefguides/guides/Calculators.pdf

  2. Dick T, Kubiak E (1997) Issues and aids for teaching mathematics to the blind. Math Teach 90:344–349

    Article  Google Scholar 

  3. Bouck EC, Flanagan S, Joshi GS, Sheikh W, Schleppenbach D (2011) Speaking math—a voice input, speech output calculator for students with visual impairments. J Spec Educ Technol 26(4):1–14. https://doi.org/10.1177/016264341102600401

    Article  Google Scholar 

  4. National Federation of the Blind (2009) The Braille Literacy Crisis in America.

  5. Alnfiai M, Sampalli S (2017) BrailleTap: Developing a calculator based on braille using tap gestures. In: Universal Access in Human–Computer Interaction. Designing Novel Interactions. Springer International Publishing, Cambridge, 213–223. https://doi.org/10.1007/978-3-319-58703-5_16

  6. Chaudhuri B, Chaudhuri L, Petelka J, Garrison P, Fogarty J, Wobbrock JO, Ladner RE (2019) GestureCalc: An eyes-free calculator for touch screens. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, pp 112–123. https://doi.org/10.1145/3308561.3353783

  7. Ruamviboonsuk V, Azenkot S, Ladner RE (2012) Tapulator: A non-visual calculator using natural prefx-free codes. In Proceedings of the 14th international ACM SIGACCESS conference on computers and accessibility (ASSETS ’12). 221–222.

  8. Oliveira J, Guerreiro T, Nicolau H, Jorge J, Gonçalves D (2011) BrailleType: Unleashing braille over touch screen mobile phones. In IFIP conference on human-computer interaction. Springer, Berlin, pp 100–107. https://doi.org/10.1007/978-3-642-23774-4_10

  9. Mascetti S, Bernareggi C, Belotti M (2011) TypeInBraille: A braille-based typing application for touchscreen devices. In The proceedings of the 13th international ACM SIGACCESS conference on computers and accessibility (ASSETS ’11). ACM, pp 295–296. https://doi.org/10.1145/2049536.2049614

  10. Subash N, Nambiar S, Kumar V (2012) Braillekey: An alternative braille text input system: Comparative study of an innovative simplified text input system for the visually impaired. In Intelligent Human Computer Interaction (IHCI), 4th International Conference on, 1–4.

  11. Bonner MN, Brudvik JT, Abowd GD, Edwards WK (2010) No-look notes: Accessible eyes-free multi-touch text entry. In International Conference on Pervasive Computing (PerCom ’10). Springer, Berlin, pp 409–426. https://doi.org/10.1007/978-3-642-12654-3_24

  12. Frey B, Southern C, Romero M (2011) BrailleTouch: Mobile texting for the visually impaired. In: Universal access in human-computer interaction. Context diversity. Springer, Berlin, pp 19–25

  13. Alnfiai M, Sampalli S (2016) An evaluation of SingleTapBraille keyboard: A text entry method that utilizes braille patterns on touchscreen devices. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, New York, NY, USA, pp 161–169. https://doi.org/10.1145/2982142.2982161

  14. Siqueira et al (2016) BrailleÉcran: A Braille approach to text entry on smartphones," 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC) Atlanta, GA, 2016, pp 608–609. https://doi.org/10.1109/COMPSAC.2016.5

  15. Hesselmann T, Heuten W, Boll S, (2011) Tap2Count: Numerical input for interactive tabletops. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS ’11). ACM, pp 256–257. https://doi.org/10.1145/2076354.2076403

  16. Apple, Inc. (2021). iPhone Accessibility. Retrieved from https://www.apple.com/accessibility/iphone/vision/

  17. Google, LLC. 2021. Android Accessibility Overview. (2021). Retrieved from https://support.google.com/accessibility/android/answer/6006564?hl=en

  18. Shaun K. Kane, Jeffrey P. Bigham, and Jacob O. Wobbrock. (2008). Slide Rule: Making Mobile Touch Screens Accessible to Blind People Using Multi-touch Interaction Techniques. In Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’08). ACM, 73–80. : https://doi.org/10.1145/1414471.1414487

  19. Azenkot S, Wobbrock JO, Prasain S, Ladner RE (2012) Input finger detection for nonvisual touch screen text entry in Perkinput. Proc. GI ‘12, Canadian Information Processing Society, pp 121–129

  20. Bonner M, Brudvik J, Abowd G, Edwards K (2010) No-look notes: accessible eyes-free multi-touch text entry. IEEE Pervasive Computing ’10. Springer, Heidelberg, pp 409–426.

  21. Oliveira J, Guerreiro T, Nicolau H, Jorge J, Gonçalves D (2011) Blind people and mobile touch-based text-entry: acknowledging the need for different flavors. Proceedings of ASSETS '11. ACM Press, New York, pp 179–186

  22. World Health Organization (2019) Blindness and vision impairment. https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment. Accessed 23 November 2020

  23. Al-Hamid N (2013). Nearly 1 million in KSA are visually impaired. Arab News. https://www.arabnews.com/news/463054. Accessed 23 November 2020

  24. Azenkot S, Rector K, Ladner RE, Wobbrock JO (2012) PassChords: Secure multi-touch authentication for blind people. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’12). ACM, pp 159–166. https://doi.org/10.1145/2384916.2384945

  25. Kane SK, Wobbrock JO, Ladner RE (2011) Usable gestures for blind people: Understanding preference and performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). Association for Computing Machinery, New York, NY, USA, 413–422. https://doi.org/10.1145/1978942.1979001

  26. González-Cañete FJ, Rodríguez JL, Galdón PM, Díaz-Estrella A (2019) Improvements in the learnability of smartphone haptic interfaces for visually impaired users. PLoS ONE 14(11):e0225053. https://doi.org/10.1371/journal.pone.0225053

    Article  Google Scholar 

  27. Paisios N (2012) Mobile accessibility tools for the visually impaired. Ph.D. Thesis http:// cs.nyu.edu/web/Research/Theses/nektariosp.pdf. Accessed 19 Sep 2012

  28. Advanced Android. (Mar 30, 2021). Explore TalkBack and text-to-speech. Retrieved from https://developer.android.com/codelabs/advanced-android-training-explore-accessibility#1

  29. Wobbrock JO, Myers BA (2006) Analyzing the input stream for character-level errors in unconstrained text entry evaluations. TOCHI 13(4):458–489

    Article  Google Scholar 

  30. Wobbrock JO (2007) Measures of text entry performance. In: MacKenzie IS, Tanaka-Ishii K (eds) Text Entry Systems: Mobility, Accessibility, Universality. Morgan Kaufmann, San Francisco, pp 47–74

    Chapter  Google Scholar 

  31. Suresh K (2011) An overview of randomization techniques: An unbiased assessment of outcome in clinical research. Journal of human reproductive sciences 4(1):8–11. https://doi.org/10.4103/0974-1208.82352

    Article  Google Scholar 

  32. Yamada H (1980) A historical study of typewriters and typing methods: from the position of planning Japanese parallels. J Inform Process 2:175–202

    Google Scholar 

  33. Profile you App. (2021). Measure app performance with Android Profiler. Retrieved from https://developer.android.com/studio/profile/android-profiler?gclid=CjwKCAjwjuqDBhAGEiwAdX2cj8DhnvRHM4tnkCgLrMAA7iMVANAvOXJVXzdEgYh2waaNn2oqv5N7bBoCmMAQAvD_BwE&gclsrc=aw.ds

  34. Kurihara S, Fukuda S, Oguchi M, Yamaguch S (2017) Estimation of Power Consumption of Each Application Caused by Device Lock Considering Software Dependency in Smartphones. Fifth International Symposium on Computing and Networking (CANDAR) 2017:560–564. https://doi.org/10.1109/CANDAR.2017.56

    Article  Google Scholar 

Download references

Acknowledgements

I would like to thank all participants from Taif University accessibility center for devoting their time and effort in the study.

If any of the sections are not relevant to your manuscript, please include the heading and write 'Not applicable' for that section.

Author information

Authors and Affiliations

  1. Department of IT, College of Computer and Information Technology, Taif University, Al-Haweiah, P. O. Box 888, Taif, 21974, Kingdom of Saudi Arabia

    Mrim Alnfiai

Authors
  1. Mrim Alnfiai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mrim Alnfiai.

Ethics declarations

Conflicts of interest

None.

Ethics approval

None.

Consent to participate

None.

Availability of data and material

None.

Code availability

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alnfiai, M. TapCalculator: nonvisual touchscreen calculator for visually impaired people preliminary user study. J Multimodal User Interfaces 16, 143–154 (2022). https://doi.org/10.1007/s12193-021-00379-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-021-00379-7

Keywords

Search

Navigation