Hostname: page-component-8448b6f56d-42gr6 Total loading time: 0 Render date: 2024-04-24T16:48:05.900Z Has data issue: false hasContentIssue false
  • English
  • Français

A machine translation mechanism of Brazilian Portuguese to Libras with syntactic-semantic adequacy

Published online by Cambridge University Press:  01 February 2021

Manuella Aschoff C. B. Lima ,
Tiago Maritan U. de Araújo ,
Rostand E. O. Costa Open the ORCID record for Rostand E. O. Costa [Opens in a new window]  and
Erickson S. Oliveira
Manuella Aschoff C. B. Lima
Affiliation:
Digital Video Aplicattions Lab (LAVID), Computer Science Institute (CI), Federal University of Paraíba (UFPB), João Pessoa, Brazil
Tiago Maritan U. de Araújo
Affiliation:
Digital Video Aplicattions Lab (LAVID), Computer Science Institute (CI), Federal University of Paraíba (UFPB), João Pessoa, Brazil
Rostand E. O. Costa*
Affiliation:
Digital Video Aplicattions Lab (LAVID), Computer Science Institute (CI), Federal University of Paraíba (UFPB), João Pessoa, Brazil
Erickson S. Oliveira
Affiliation:
Digital Video Aplicattions Lab (LAVID), Computer Science Institute (CI), Federal University of Paraíba (UFPB), João Pessoa, Brazil
*
*Corresponding author. E-mail: rostand@lavid.ufpb.br
Get access Rights & Permissions [Opens in a new window]

Abstract

Deaf people communicate naturally using visual-spatial languages, called sign languages (SL). Although SLs are recognized as a language in many countries, the problems faced by Deaf people for accessing information remain. As a result, they have difficulties exercising their citizenship and access information in SLs, which usually leads to linguistic and knowledge acquisition delays. Some scientific works have been developed to address these problems related to the machine translation of spoken languages to sign languages. However, the existing machine translation platforms have some limitations, especially in syntactic and lexical nature. Thus, this work aims to develop a mechanism for machine translation to Libras, the Brazilian Sign Language, with syntactic-semantic adequacy. It consists of an automatic translation component for Libras based on syntactic-semantic translation rules and a formal syntactic-semantic rule description language. As proof of concept of the proposed approach, we created a specific grammar for Libras translation exploring these aspects and integrating these elements into VLibras Suite, a service for machine translation of digital content in Brazilian Portuguese (BP) to Libras. We performed several tests using this modified version of VLibras to measure the level of comprehension of the output generated by the new translator mechanism. In the computational experiments, as well as in the actual tests with Deaf and hearing users, the proposed approach was able to improve the results of the current VLibras version.

Keywords

Machine TranslationMultimedia PlatformsFormal Rule Description LanguageSign Language
Type
Article
Information
Natural Language Engineering , Volume 28 , Issue 3 , May 2022 , pp. 271 - 294
Check for updates
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alencar, L.F. (2010). Aelius: uma ferramenta para anotação automática de corpora usando o nltk. (Aelius: a tool for automatic corpora annotation using nltk). In IX Encontro de Linguistica de Corpus. Campinas, SP, Brasil: Mercado das Letras, pp. 216–219.Google Scholar
Araujo, T.M.U. (2012). Uma solução para geração automática de trilhas em Língua Brasileira de Sinais em conteudos multimídia (A Solution for Automatic Generation of Tracks in Brazilian Sign Language in Multimedia Content). PhD Thesis, Universidade Federal do Rio Grande do Norte, Natal, RN, Brazil.Google Scholar
Araujo, T.M.U., Ferreira, F.L.S., Silva, D.A.N.d.S., Lemos, F.H., Neto, G.P., Omaia, D., de Souza Filho, G.L. and Tavares, T.A. (2013). Automatic generation of brazilian sign language windows for digital tv systems. Journal of the Brazilian Computer Society 19(2), 107–25, Springer, New York, NY, USA.Google Scholar
Araujo, T.M.U., Ferreira, F.L.S., Silva, D.A.N.d.S., Oliveira, L.D., Falcóo, E.L., Domingues, L.A., Martins, V.F., Portela, I.A.C., Nobrega, Y.S., de Souza Filho, G.L., Tavares, T.A. and Duarte, A.N. (2014). An approach to generate and embed sign language video tracks into multimedia contents. Information Sciences 281, 762–780, Elsevier, New York,, NY, EUA.Google Scholar
Bacardit, J., Burke, E.K. and Krasnogor, N. (2009). Improving the scalability of rule-based evolutionary learning. Memetic Computing 01(01), 55–67, Springer, New York, NY, USA.Google Scholar
Boulares, M. and Jemni, M. (2013). Toward a mobile service for hard of hearing people to make information accessible anywhere. In International Conference on Electrical Engineering and Software Applications (ICEESA). Nova Jersey, USA: IEEE, pp. 1–5.Google Scholar
Bonham, M.E. (2015). English to ASL Gloss Machine Translation. Master Thesis, Brigham Young University, Provo, USA.Google Scholar
Choi, D., Vieira, M.I.d.S., de Oliveira, P.R.G. and Nakasato, R. (2011). Libras: conhecimento além dos sinais (Libras: Knowledge Beyond the Signs). São Paulo, Brazil: Person Prentice Hall.Google Scholar
Costa, W.C.O. (2005). O texto traduzido como re-textualização (The text translated as re-textualization). Cadernos de Tradução 2(16), 25–54, UFSC, Florianopólis, SC, Brasil.Google Scholar
Dreuw, P., Forster, J., Gweth, Y., Stein, D., Ney, H., Martinez, G., Llahi, J.V., Crasborn, O., Ormel, E., Du, W., Hoyoux, T., Piater, J., Moya, J.M. and Wheatley, M. (2010). SignSpeak: scientific understanding and vision-based technological development for continuous sign language recognition and translation. In Proceedings of the 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies. Valletta, Malta: Universität Hamburg, pp. 65–72.Google Scholar
Felipe, T. (2007). Descrição da língua de sinais: Desafios teãricos e práticos (Sign language description: Theoretical and practical challenges). In Anais do Congresso Internacional do INES. Rio de Janeiro, Brasil: Instituto Nacional de Educação de Surdos.Google Scholar
Felipe, T. (2006). Os processos de formaçáo de palavra na Libras (The processes of word formation in Libras). Educação Temática Digital 7(2), 199–216, Unicamp, Campinas, SP, Brazil.Google Scholar
Freitas, C., Rocha, P. and Bick, E. (2008). Floresta sinta(c)tica: bigger, thicker and easier. In Computational Processing of the Portuguese Language. New York, NY, USA: Springer, pp. 216–219.CrossRefGoogle Scholar
Guimarães, A.P.N. (2014). Recomendações para avaliação da experiência de usuário em aplicativos móveis para surdos (Recommendations for Evaluating the User Experience in Mobile Applications for the Deaf). Master Dissertation, Universidade Federal da Paraíba, João Pessoa, Brazil.Google Scholar
Huenerfauth, M. (2004). A multi-path architecture for machine translation of english text into american sign language animation. In Proceedings of the Student Research Workshop at HLTNAACL, vol. 4. Boston, USA: Association for Computational Linguistics, pp. 25–30.Google Scholar
Huenerfauth, M. (2005a). American sign language generation: multimodal nlg with multiple linguistic channels. In Proceedings of the ACL Student Research Workshop. Ann Arbor, USA: Association for Computational Linguistics, pp. 37–42.Google Scholar
Huenerfauth, M. (2005b). Representing coordination and non-coordination in an american sign language animation. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility, vol. 7, New York, NY, USA: ACM, pp. 44–51.Google Scholar
Huenerfauth, M. (2008). Generating american sign language animation: overcoming misconceptions and technical challenges. Universal Access in the Information Society 6(4), 419–434, Springer, New York, NY, USA.CrossRefGoogle Scholar
Huenerfauth, M., Zhao, L., Gu, E. and Allbeck, J. (2008). Evaluating american sign language generation through the participation of native asl signers. ACM Transactions on Accessible Computing 1(1), 211–218, ACM, New York, NY, USA.CrossRefGoogle Scholar
Kim, J. and O’Neill-Brown, P. (2019). Improving American sign language recognition with synthetic data. In Proceedings of Machine Translation Summit XVII Volume 1: Research Track, vol. 19. Dublin, Ireland: European Association for Machine Translation, pp. 151–161.Google Scholar
Lopez-Ludena, V., Gonzalez-Morcillo, C., Lopez, J., Ferreiro, E., Ferreiros, J. and San-Segundo, R. (2014a). Methodology for developing an advanced communications system for the deaf in a new domain. Knowledge-Based Systems 52, 240–252, Elsevier, New York, NY, USA.CrossRefGoogle Scholar
Lopez-Ludena, V., Gonzalez-Morcillo, C., Lopez, J., Ferreiro, E., Ferreiros, J. and San-Segundo, R. (2014b). Translating bus information into sign language for deaf people. Engineering Applications of Artificial Intelligence 32, 258–269, Elsevier, New York, NY, USA.CrossRefGoogle Scholar
Melo, F.R., Matos, H.C.d.O. and Dias, E.R.B. (2014). Aplicação da métrica bleu para avaliação comparativa dos tradutores automáticos bing tradutor e google tradutor (Application of the metric bleu for comparative evaluation of automatic translators bing translator and google translator). E-Scrita 5(3), 33–45, UNIABEU, Belford Roxo, Brasil.Google Scholar
Morrissey, S. and Way, A. (2013). Manual labour: tackling machine translation for sign languages. Machine Translation 27(1), 25–64, Springer, New York, NY, USA.CrossRefGoogle Scholar
Niessen, S., Och, F.J., Leusch, G. and Ney, H. (2000). An evaluation tool for machine translation: fast evaluation for machine translation research. In Proceedings of the Second International Conference on Language Resources & Evaluation (LREC), Athens, Greece. Paris: France European Language Resources Association (ELRA), pp. 39–45.Google Scholar
Othero, G.A. (2009) A gramática da Frase em Português: algumas reflexões para a formalização da estrutura frasal em português (The Grammar of the Portuguese Phrase: Some Reflections for the Formalization of the Phrasal Structure in Portuguese). Porto Alegre, Brazil: EdiPUCRS.Google Scholar
Papineni, K., Roukos, S., Ward, T. and Zhu, W.J. (2002). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Boston, USA: ACL, pp. 311–318.Google Scholar
Post, M. (2018). A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers. Boston, USA: Association for Computational Linguistics, pp. 186–191. doi: 10.18653/v1/W18-6319.Google Scholar
Quadros, R.M. (1997). Educação de Surdos - Aquisição da Linguagem (Deaf Education - Language Acquisition). Porto Alegre, Brazil: Editora Artes Medicas.Google Scholar
Quadros, R.M. and Karnopp, L.B. (2004). Língua de sinais brasileira: estudos linguísticos (Brazilian Sign Language: Language Studies). Porto Alegre, Brazil: Artmed Editora.Google Scholar
Rayner, E., Bouillon, P., Gerlach, J., Strasly, I., Tsourakis, N. and Ebling, S. (2016) An OpenWeb platform for rule-based speech-to-sign translation. In 54th Annual Meeting of the Association for Computational Linguistics (ACL). Boston, USA: Association for Computational Linguistics. Retrieved from https://archive-ouverte.unige.ch/unige:87596.Google Scholar
San-Segundo, R., Montero, J.M., Macas-Guarasa, J., Cordoba, R., Ferreiros, J. and Pardo, J.M. (2008). Proposing a speech to gesture translation architecture for spanish deaf people. Journal of Visual Languages and Computing 19(5), 523–538, Elsevier, Boston: USA.CrossRefGoogle Scholar
Silva, M.P.M. (2006). A Semântica como Negociação dos Significados em Libras (Semantics as Negotiation of Meanings in Libras). Trabalhos em Linguística Aplicada 45(2), 255–269, Unicamp, Campinas, SP, Brasil.CrossRefGoogle Scholar
Stein, D., Forster, J., Zelle, U., Dreuw, P. and Ney, H. (2010). Rwthphoenix: analysis of the German sign language corpus. In Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, vol. 4. Valletta, Malta: Universität Hamburg, pp. 225–230.Google Scholar
Stokoe, W.C. (1980). Sign language structure. Annual Review of Anthropology 9, 365–3690, JSTOR, Redwood City, USA.Google Scholar
Su, H.Y. and Wu, A.H. (2009). Improving structural statistical machine translation for sign language with small corpus using thematic role templates as translation memory. IEEE Transactions on Machine Translation, Audio, Speech, and Language Processing 17, 1305–1315, IEEE, Nova Jersey, EUA.CrossRefGoogle Scholar