Skip to main content
SpringerLink
Log in

Document structure model for survey generation using neural network

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Survey generation aims to generate a summary from a scientific topic based on related papers. The structure of papers deeply influences the generative process of survey, especially the relationships between sentence and sentence, paragraph and paragraph. In principle, the structure of paper can influence the quality of the summary. Therefore, we employ the structure of paper to leverage contextual information among sentences in paragraphs to generate a survey for documents. In particular, we present a neural document structure model for survey generation. We take paragraphs as units, and model sentences in paragraphs, we then employ a hierarchical model to learn structure among sentences, which can be used to select important and informative sentences to generate survey. We evaluate our model on scientific document data set. The experimental results show that our model is effective, and the generated survey is informative and readable.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mohammad S, Dorr B, Egan M, Hassan A, Muthukrishan P, Qazvinian V, Radev D R, Zajic D. Using citations to generate surveys of scientific paradigms. In: Proceedings of Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2009, 584–592

  2. Jha R, Finegan-Dollak C, King B, Coke R, Radev D R. Content models for survey generation: a factoid-based evaluation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015, 441–450

  3. Jha R, Coke R, Radev D R. Surveyor: a system for generating coherent survey articles for scientific topics. In: Proceedings of the 29th AAAI Conference on Artificial Intelligence. 2015, 2167–2173

  4. Teufel S, Moens M. Summarizing scientific articles: experiments with relevance and rhetorical status. Computational Linguistics, 2002, 28(4): 409–445

    Article  Google Scholar 

  5. Teufel S. Argumentative zoning: information extraction from scientific text. Dissertation, University of Edinburgh, 1999

  6. Guo Y, Korhonen A, Poibeau T. A weakly-supervised approach to argumentative zoning of scientific documents. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2011, 273–283

  7. Tbahriti I, Chichester C, Lisacek F, Ruch P. Using argumentation to retrieve articles with similar citations: an inquiry into improving related articles search in the medline digital library. International Journal of Medical Informatics, 2006, 75(6): 488–495

    Article  Google Scholar 

  8. Liakata M, Dobnik S, Saha S, Batchelor C, Rebholz-Schuhmann D. A discourse-driven content model for summarising scientific articles evaluated in a complex question answering task. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2013, 747–757

  9. Widyantoro D H, Khodra M L, Riyanto B, Aziz E A. A multiclass-based classification strategy for rhetorical sentence categorization from scientific papers. Journal of ICT Research and Applications, 2013, 7(3): 235–249

    Article  Google Scholar 

  10. Liu H. Automatic argumentative-zoning using Word2vec. 2017, arXiv preprint arXiv: 1703.10152

  11. Chen J Q, Zhuge H. Automatic generation of related work through summarizing citations. Concurrency and Computation: Practice and Experience, 2019, 31(3): e4261

    Article  Google Scholar 

  12. Teufel S, Siddharthan A, Batchelor C. Towards discipline-independent argumentative zoning: evidence from chemistry and computational linguistics. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2009, 1493–1502

  13. Guo Y F, Korhonen A, Silins I, Stenius U. Weakly supervised learning of information structure of scientific abstracts — is it accurate enough to benefit real-world tasks in biomedicine? Bioinformatics, 2011, 27(22): 3179–3185

    Article  Google Scholar 

  14. Cohan A, Goharian N. Scientific document summarization via citation contextualization and scientific discourse. International Journal on Digital Libraries, 2018, 19(2–3): 287–303

    Article  Google Scholar 

  15. Namboodiri A M, Jain A K. Document Structure and Layout Analysis. Digital Document Processing. Springer, London, 2007

    Book  Google Scholar 

  16. Lüngen H, Bärenfänger M, Hilbert M, Lobin H, Puskás C. Discourse Relations and Document Structure. Linguistic Modeling of Information and Markup Languages. Springer, London, 2010

    Book  Google Scholar 

  17. Mao S, Rosenfeld A, Kanungo T. Document structure analysis algorithms: a literature survey. Proceedings of the SPIE, 2003, 5010: 197–207

    Article  Google Scholar 

  18. Yang C C, Wang F L. Fractal summarization: summarization based on fractal theory. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 2003, 391–392

  19. Wang F L, Yang C C. Impact of document structure on hierarchical summarization. In: Proceedings of the 9th Conference on Asian Digital Libraries. 2006, 459–469

  20. Wang F L, Yang C C, Shi X. Multi-document summarization for terrorism information extraction. In: Proceedings of the International Conference on Intelligence and Security Informatics. 2006, 602–608

  21. Yang C C, Wang F L. Hierarchical summarization of large documents. Journal of the American Society for Information Science and Technology, 2008, 59(6): 887–902

    Article  Google Scholar 

  22. Glaser B, Strauss A L. The discovery of grounded theory: strategies for qualitative research. Nursing Research, 1968, 17(4): 377–380

    Article  Google Scholar 

  23. Endres-Niggemeyer B, Maier E, Sigel A. How to implement a naturalistic model of abstracting: four core working steps of an expert abstractor. Information Processing and Management, 1995, 31(5): 631–674

    Article  Google Scholar 

  24. Contractor D, Guo Y F, Korhonen A. Using argumentative zones for extractive summarization of scientific articles. In: Proceedings of the 24th International Conference on Computational Linguistics. 2012, 663–678

  25. Collins E, Augenstein I, Riedel S. A supervised approach to extractive summarisation of scientific papers. In: Proceedings of the 21st Conference on Computational Natural Language Learning. 2017, 195–205

  26. Xiao W, Carenini G. Extractive summarization of long documents by combining global and local context. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 3009–3019

  27. Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2013, 3111–3119

  28. Graves A. Supervised sequence labelling with recurrent neural networks. Disseration, Technical University of Munich, Germany, 2008

    MATH  Google Scholar 

  29. Gers F A, Schmidhuber J, Cummins F A. Learning to forget: continual prediction with LSTM. Neural Computation, 2000, 12(10): 2451–2471

    Article  Google Scholar 

  30. Gers F A, Schraudolph N N, Schmidhuber J. Learning precise timing with LSTM recurrent networks. Journal of Machine Learning Research, 2002, 3: 115–143

    MathSciNet  MATH  Google Scholar 

  31. Ruben Z, Alicia L D, Javier G D, Doroteo T T, Joaquin G R, Ian M. Language identification in short utterances using long short-term memory (LSTM) recurrent neural networks. PLoS ONE, 2016, 11(1): e0146917

    Article  Google Scholar 

  32. Bengio Y, Schwenk H, Senécal J S, Morin F, Gauvain J L. Neural probabilistic language models. Journal of Machine Learning Research, 2003, 3(6): 1137–1155

    Google Scholar 

  33. Mikolov T, Zweig G. Context dependent recurrent neural network language model. In: Proceedings of IEEE Spoken Language Technology Workshop. 2012, 234–239

  34. Le Q V, Mikolov T. Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning. 2014, 1188–1196

  35. Lin C Y. ROUGE: a package for automatic evaluation of summaries. In: Proceedings of ACL Workshop on Text Summarization Branches Out. 2004, 74–81

  36. Nallapati R, Zhai F, Zhou B W. SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 3075–3081

  37. Yao J G, Wan X J, Xiao J G. Recent advances in document summarization. Knowledge and Information Systems, 2017, 53(2): 297–336

    Article  Google Scholar 

  38. Chen Y C, Bansal M. Fast abstractive summarization with reinforce-selected sentence rewriting. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018, 675–686

  39. Kryscinski W, Keskar N S, Mccann B, Xiong C, Socher R. Neural text summarization: a critical evaluation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 540–551

  40. Halliday M A K, Hasan R. Cohesion in English. London: Longman, 1976

    Google Scholar 

Download references

Acknowledgements

We would like to extend our sincere thanks to Dr. Qingying Sun for her beneficial suggestions. We also thank our anonymous reviewers for their insightful and valuable comments. This work was supported by the Fundamental Research Funds for the Central Universities (2018B678X14 and 2016B44414), Postgraduate Research Practice Innovation Program of Jiangsu Province of China (KYCX18_0553 and KYLX16_0722), the National Natural Science Foundation of China (Grant Nos. 61806137 and 61976146), Project of Natural Science Research of the Universities of Jiangsu Province (18KJB520043).

Author information

Authors and Affiliations

  1. College of Computer and Information, Hohai University, Nanjing, 210098, China

    Huiyan Xu, Xiaolan Weng & Zhijian Wang

  2. School of Computer Science and Technology, Huaiyin Normal University, Huai’an, 223300, China

    Huiyan Xu & Xiaolan Weng

  3. Natural Language Processing Lab, Soochow University, Suzhou, 215006, China

    Zhongqing Wang, Yifei Zhang & Guodong Zhou

Authors
  1. Huiyan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhongqing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yifei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaolan Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhijian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guodong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhongqing Wang.

Additional information

Huiyan Xu is a PhD candidate in the College of Computer and Information, Hohai University, China. Her major research interests include text mining and natural language processing. She is interested in document summarization, text generation, and semantic computing.

Zhongqing Wang received his PhD degree in 2016 from the School of Computer Science and Technology, Soochow University, China. Since April 2016, he has been a postdoctoral research fellow at Singapore University of Technology and Design, Singapore. He is a lecturer in the School of Computer Science and Technology, Soochow University, China. His current research interests include natural language processing, sentiment analysis and social computing.

Yifei Zhang is a master candidate in the School of Computer Science and Technology, Soochow University, China. Her major research interests include text mining and natural language processing. She is interested in document summarization, text generation, and semantic computing.

Xiaolan Weng is a PhD candidate in the College of Computer and Information, Hohai University, China. Her major research interests include text mining and natural language processing. She is interested in document recommendation and social computing.

Zhijian Wang received his PhD degree from the Nanjing University, China. He is a professor in the College of Computer and Information, Hohai University, China. His major research interests include computer software and text mining. He is interested in software system integration and distributed computing.

Guodong Zhou received his PhD degree in 1999 from the National University of Singapore, Singapore. He is a full professor in the School of Computer Science and Technology, and the Director of the Natural Language Processing Laboratory from Soochow University, China. His research interests include information retrieval, natural language processing.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, H., Wang, Z., Zhang, Y. et al. Document structure model for survey generation using neural network. Front. Comput. Sci. 15, 154325 (2021). https://doi.org/10.1007/s11704-020-9366-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-020-9366-8

Keywords

Search

Navigation