Skip to main content
SpringerLink
Log in

A short survey on end-to-end simple question answering systems

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Searching for a specific and meaningful piece of information in the humongous textual data volumes found on the internet and knowledge repositories is a very challenging task. This problem is usually constrained to answering simple, factoid questions by resorting to a question answering (QA) system built on top of complex approaches such as heuristics, information retrieval, and machine learning. More precisely, deep learning methods became into sharp focus of this research field because such purposes can realize the benefits of the vast amounts of data to boost the practical results of QA systems. In this paper, we present a systematic survey on deep learning-based QA systems concerning factoid questions, with particular focus on how each existing system addresses their critical features in terms of learning end-to-end models. We also detail the evaluation process carried out on these systems and discuss how each approach differs from the others in terms of the challenges tackled and the strategies employed. Finally, we present the most prominent research problems still open in the field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://project-hobbit.eu/challenges/qald-8-challenge/

  2. https://rajpurkar.github.io/SQuAD-explorer/

References

  • Abualigah LM, Khader AT, Hanandeh ES (2018) A combination of objective functions and hybrid krill herd algorithm for text document clustering analysis. Eng Appl Artif Intell 73:111–125

    Article  Google Scholar 

  • Abualigah LMQ (2019) Feature selection and enhanced krill herd algorithm for text document clustering. Springer, Berlin

    Book  Google Scholar 

  • Abualigah LMQ, Hanandeh ES (2015) Applying genetic algorithms to information retrieval using vector space model. Int J Comput Sci Eng Appl 5(1):19

    Google Scholar 

  • ACM (2013) Acm digital library. http://dl.acm.org

  • Aghaebrahimian A, Jurčíček F (2016) Open-domain factoid question answering via knowledge graph search. In: Proceedings of the workshop on human-computer question answering, pp 22–28

  • Bast H, Haussmann E (2015) More accurate question answering on freebase. In: Proceedings of the 24th ACM international on conference on information and knowledge management, ACM, pp 1431–1440

  • Berant J, Chou A, Frostig R, Liang P (2013) Semantic parsing on freebase from question-answer pairs. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 1533–1544

  • Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, AcM, pp 1247–1250

  • Boné R, Crucianu M, de Beauville JPA (2002) Learning long-term dependencies by the selective addition of time-delayed connections to recurrent neural networks. Neurocomputing 48(1–4):251–266

    Article  Google Scholar 

  • Bordes A, Chopra S, Weston J (2014) Question answering with subgraph embeddings. arXiv preprint arXiv:14063676

  • Bordes A, Usunier N, Chopra S, Weston J (2015) Large-scale simple question answering with memory networks. arXiv preprint arXiv:150602075

  • Buchholz S, Daelemans W (2001) Complex answers: a case study using a www question answering system. Natl Lang Eng 7(4):301–323

    Article  Google Scholar 

  • Buzaaba H, Amagasa T (2019) A modular approach for efficient simple question answering over knowledge base. In: International conference on database and expert systems applications, Springer, pp 237–246

  • Camacho-Collados J, Pilehvar MT (2018) From word to sense embeddings: a survey on vector representations of meaning. J Artif Intell Res 63:743–788. https://doi.org/10.1613/jair.1.11259

    Article  MathSciNet  MATH  Google Scholar 

  • Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:14061078

  • Cimiano P, Minock M (2009) Natural language interfaces: what is the problem?–a data-driven quantitative analysis. In: International conference on application of natural language to information systems, Springer, pp 192–206

  • Cimiano P, Lopez V, Unger C, Cabrio E, Ngomo ACN, Walter S (2013) Multilingual question answering over linked data (qald-3): Lab overview. In: International conference of the cross-language evaluation forum for european languages, Springer, pp 321–332

  • Dai Z, Li L, Xu W (2016) Cfo: Conditional focused neural question answering with large-scale knowledge bases. arXiv preprint arXiv:160601994

  • Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805

  • Diefenbach D, Lopez V, Singh K, Maret P (2018) Core techniques of question answering systems over knowledge bases: a survey. Knowl Inf Syst 55(3):529–569

    Article  Google Scholar 

  • Direct S (2013) Science direct. http://www.sciencedirect.com/

  • Goldberg Y (2017) Neural network methods for natural language processing. Synth Lect Human Lang Technol 10(1):1–309

    Article  Google Scholar 

  • Golub D, He X (2016) Character-level question answering with attention. arXiv preprint arXiv:160400727

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge, http://www.deeplearningbook.org

  • Graves A, Mohamed Ar, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE international conference on acoustics, speech and signal processing, IEEE, pp 6645–6649

  • Hakimov S, Unger C, Walter S, Cimiano P (2015) Applying semantic parsing to question answering over linked data: Addressing the lexical gap. In: International conference on applications of natural language to information systems, Springer, pp 103–109

  • Hannun A, Case C, Casper J, Catanzaro B, Diamos G, Elsen E, Prenger R, Satheesh S, Sengupta S, Coates A, et al. (2014) Deep speech: scaling up end-to-end speech recognition. arXiv preprint arXiv:14125567

  • He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  • Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J, et al. (2001) Gradient flow in recurrent nets: the difficulty of learning long-term dependencies

  • Höffner K, Walter S, Marx E, Usbeck R, Lehmann J, Ngonga Ngomo AC (2017) Survey on challenges of question answering in the semantic web. Semantic Web 8(6):895–920

    Article  Google Scholar 

  • Hu B, Lu Z, Li H, Chen Q (2014) Convolutional neural network architectures for matching natural language sentences. In: Advances in neural information processing systems, pp 2042–2050

  • IEEXplore (2013) Ieeexplore digital library. http://ieeexplore.ieee.org/Xplore/home.jsp

  • Jain S (2016) Question answering over knowledge base using factual memory networks. In: Proceedings of the NAACL student research workshop, pp 109–115

  • Joulin A, Grave E, Bojanowski P, Nickel M, Mikolov T (2017) Fast linear model for knowledge graph embeddings. arXiv preprint arXiv:171010881

  • Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv preprint arXiv:14042188

  • Kumar A, Irsoy O, Ondruska P, Iyyer M, Bradbury J, Gulrajani I, Zhong V, Paulus R, Socher R (2016) Ask me anything: Dynamic memory networks for natural language processing. In: International conference on machine learning, pp 1378–1387

  • Lafferty J, McCallum A, Pereira FC (2001) Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning, Morgan Kaufmann, pp 282–289

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  • Lopez V, Unger C, Cimiano P, Motta E (2013) Evaluating question answering over linked data. Web Semant Sci Serv Agents World Wide Web 21:3–13

    Article  Google Scholar 

  • Lukovnikov D, Fischer A, Lehmann J, Auer S (2017) Neural network-based question answering over knowledge graphs on word and character level. In: Proceedings of the 26th international conference on World Wide Web, international world wide web conferences steering committee, pp 1211–1220

  • Lukovnikov D, Fischer A, Lehmann J (2019) Pretrained transformers for simple question answering over knowledge graphs. In: International semantic web conference, Springer, pp 470–486

  • Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:13013781

  • Mohammed S, Shi P, Lin J (2017) Strong baselines for simple question answering over knowledge graphs with and without neural networks. arXiv preprint arXiv:171201969

  • Pai M, McCulloch M, Gorman JD, Pai N, Enanoria W, Kennedy G, Tharyan P, Colford JJ (2004) Systematic reviews and meta-analyses: an illustrated, step-by-step guide. Natl Med J India 17(2):86–95

    Google Scholar 

  • Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), pp 1532–1543. http://www.aclweb.org/anthology/D14-1162

  • Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. arXiv preprint arXiv:180205365

  • Petrochuk M, Zettlemoyer L (2018) Simplequestions nearly solved: a new upperbound and baseline approach. arXiv preprint arXiv:180408798

  • Qin P, Xu W, Guo J (2016) An empirical convolutional neural network approach for semantic relation classification. Neurocomputing 190:1–9

    Article  Google Scholar 

  • Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:160605250

  • Rao Y, Lu J, Zhou J (2017) Attention-aware deep reinforcement learning for video face recognition. In: Proceedings of the IEEE international conference on computer vision, pp 3931–3940

  • Rush AM, Chopra S, Weston J (2015) A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:150900685

  • Sak H, Senior A, Rao K, Beaufays F (2015) Fast and accurate recurrent neural network acoustic models for speech recognition. arXiv preprint arXiv:150706947

  • Schaefer AM, Udluft S, Zimmermann HG (2008) Learning long-term dependencies with recurrent neural networks. Neurocomputing 71(13–15):2481–2488

    Article  Google Scholar 

  • Scopus (2013) Scopus. http://www.scopus.com/

  • Severyn A, Moschitti A (2015) Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, ACM, New York, NY, USA, SIGIR ’15, pp 373–382, 10.1145/2766462.2767738,

  • Sharma Y, Gupta S (2018) Deep learning approaches for question answering system. Procedia Computer Science 132:785–794

    Article  Google Scholar 

  • Suen CY (1979) N-gram statistics for natural language understanding and text processing. IEEE Trans Pattern Anal Mach Intell PAMI 1(2):164–172

    Article  Google Scholar 

  • Sukhbaatar S, Weston J, Fergus R, et al. (2015) End-to-end memory networks. In: Advances in neural information processing systems, pp 2440–2448

  • Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112

  • Ture F, Jojic O (2017) No need to pay attention: Simple recurrent neural networks work!(for answering“ simple” questions). arXiv preprint arXiv:160605029

  • Unger C, Forascu C, Lopez V, Ngomo ACN, Cabrio E, Cimiano P, Walter S (2014) Question answering over linked data (qald-4). In: Working Notes for CLEF 2014 Conference

  • Wang P, Xu B, Xu J, Tian G, Liu CL, Hao H (2016) Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification. Neurocomputing 174:806–814

    Article  Google Scholar 

  • Wani MA, Bhat FA, Afzal S, Khan AI (2020) Advances in deep learning. Springer, Berlin

    Book  Google Scholar 

  • Weston J, Chopra S, Bordes A (2014) Memory networks. arXiv preprint arXiv:14103916

  • Yao X, Van Durme B (2014) Information extraction over structured data: Question answering with freebase. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, vol 1, pp 956–966

  • Yin W, Yu M, Xiang B, Zhou B, Schütze H (2016) Simple question answering by attentive convolutional neural network. arXiv preprint arXiv:160603391

  • Zhou J, Xu W (2015) End-to-end learning of semantic role labeling using recurrent neural networks. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, vol 1, Long Papers, Association for computational linguistics, Beijing, China, pp 1127–1137, https://doi.org/10.3115/v1/P15-1109. https://www.aclweb.org/anthology/P15-1109

  • Zhu S, Cheng X, Su S, Lang S (2017) Knowledge-based question answering by jointly generating, copying and paraphrasing. In: Proceedings of the 2017 ACM on conference on information and knowledge management, ACM, pp 2439–2442

Download references

Author information

Authors and Affiliations

  1. Universidade Federal do Ceará, Crateús, Brazil

    José Wellington Franco da Silva & Amanda Drielly Pires Venceslau

  2. University of Passau, Germany, Passau, Germany

    Juliano Efson Sales

  3. Universidade Federal do Ceará, Campus do Pici, Fortaleza, Ceará, Brazil

    José Gilvan Rodrigues Maia & Vânia Maria Ponte Vidal

  4. University of Fortaleza, Fortaleza, Ceará, Brazil

    Vládia Célia Monteiro Pinheiro

Authors
  1. José Wellington Franco da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amanda Drielly Pires Venceslau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juliano Efson Sales
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Gilvan Rodrigues Maia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vládia Célia Monteiro Pinheiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Vânia Maria Ponte Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Wellington Franco da Silva.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Silva, J.W.F., Venceslau, A.D.P., Sales, J.E. et al. A short survey on end-to-end simple question answering systems. Artif Intell Rev 53, 5429–5453 (2020). https://doi.org/10.1007/s10462-020-09826-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-020-09826-5

Keywords

Search

Navigation