Skip to main content
SpringerLink
Log in

Improving neural sentence alignment with word translation

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Sentence alignment is a basic task in natural language processing which aims to extract high-quality parallel sentences automatically. Motivated by the observation that aligned sentence pairs contain a larger number of aligned words than unaligned ones, we treat word translation as one of the most useful external knowledge. In this paper, we show how to explicitly integrate word translation into neural sentence alignment. Specifically, this paper proposes three cross-lingual encoders to incorporate word translation: 1) Mixed Encoder that learns words and their translation annotation vectors over sequences where words and their translations are mixed alternatively; 2) Factored Encoder that views word translations as features and encodes words and their translations by concatenating their embeddings; and 3) Gated Encoder that uses gate mechanism to selectively control the amount of word translations moving forward. Experimentation on NIST MT and Opensubtitles Chinese-English datasets on both non-monotonicity and monotonicity scenarios demonstrates that all the proposed encoders significantly improve sentence alignment performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Proceedings of International Conference on Learning Representations. 2015

  2. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A, Kaiser L, Polosukhin I. Attention is all you need. In: Proceedings of the 31st Conference on Neural Information Processing Systems. 2017, 6000–6010

  3. Hermann K M, Blunsom P. Multilingual models for compositional distributed semantics. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014, 58–68

  4. Nie J Y, Simard M, Isabelle P, Durand R. Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the web. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 1999, 74–81

  5. Martino G D S, Romeo S, Barroón-Cedeno A, Joty S, Marquez L, Moschitti A, Nakov P. Cross-language question re-ranking. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, 1145–1148

  6. Wu D. Alignment. Handbook of Natural Language Processing. CRC Press. 2010

  7. Gregoire F, Langlais P. A deep neural network approach to parallel sentence extraction. 2017, arXiv preprint arXiv:1709.09783

  8. Grover J, Mitra P. Bilingual word embeddings with bucketed CNN for parallel sentence extraction. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics-Student Research Workshop. 2017, 11–16

  9. Ding Y, Li J, Zhou G. Word-pair relevance network for sentence alignment. Journal of Chinese Information Processing, 2019

  10. Ding Y, Li J H, Gong Z X, Zhou G D. Word-pair relevance modeling with multi-view neural attention mechanism for sentence alignment. Journal of Computer Science and Technology, 2019

  11. Liu L, Utiyama M, Finch A, Sumita E. Neural machine translation with supervised attention. In: Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers. 2016, 3093–3102

  12. Mi H, Wang Z, Ittycheriah A. Supervised attentions for neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 2283–2288

  13. Arthur P, Neubig G, Nakamura S. Incorporating discrete translation lexicons into neural machine translation. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016, 1557–1567

  14. Gale W A, Church K W. A program for aligning sentences in bilingual corpora. In: Proceedings of the Meeting on the Association for Computational Linguistics. 1991, 177–184

  15. Chen S F. Aligning sentences in bilingual corpora using lexical information. Computer Knowledge & Technology, 1993, 46(3): 9–16

    Google Scholar 

  16. Wu D. Aligning a parallel English-Chinese corpus statistically with lexical criteria. Computer Science, 1994, 4(4): 80–87

    Google Scholar 

  17. Moore R C. Fast and accurate sentence alignment of bilingual corpora. In: Processing of the 5th Conference of the Association for Machine Translation in the Americas. 2002, 135–144

  18. Brown P F, Pietra V J D, Pietra S A D, Mercer R L. The mathematics of statistical machine translation: parameter estimation. Computational linguistics, 1993, 19(2): 263–311

    Google Scholar 

  19. Braune F, Fraser A. Improved unsupervised sentence alignment for symmetrical and asymmetrical parallel corpora. In: Processings of the 23rd International Conference on Computational Linguistics. 2010, 81–89

  20. Ma X. Champollion: a robust parallel text sentence aligner. In: Processings of the 5th International Conference on Language Resources and Evaluation. 2006, 489–492

  21. Li P, Sun M, Xue P. Fast-Champollion: a fast and robust sentence alignment algorithm. In: Proceedings of the 23rd International Conference on Computational Linguistics. 2010, 710–718

  22. Quan X, Kit C, Song Y. Non-monotonic sentence alignment via semisupervised learning. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. 2013, 622–630

  23. Chatterjee R, Negri M, Turchi M, Federico M, Specia L, Blain F. Guiding neural machine translation decoding with external knowledge. In: Proceedings of the 2nd Conference on Machine Translation. 2017, 157–168

  24. Nguyen T Q, Chiang D. Improving lexical choice in neural machine translation. 2017, arXiv preprint arXiv:1710.01329

  25. Wang X, Tu Z, Xiong D, Zhang M. Translating phrases in neural machine translation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 1421–1431

  26. Wang X, Lu Z, Tu Z, Li H, Xiong D, Zhang M. Neural machine translation advised by statistical machine translation. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017, 3330–3336

  27. Chen K, Wang R, Utiyama M, Liu L, Tamura A, Sumita E, Zhao T. Neural machine translation with source dependency representation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017, 2846–2852

  28. Li J, Xiong D, Tu Z, Zhu M, Zhang M, Zhou G. Modeling source syntax for neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 688–697

  29. Eriguchi A, Hashimoto K, Tsuruoka Y. Tree-to-sequence attentional neural machine translation. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016, 823–833

  30. Sennrich R, Haddow B. Linguistic input features improve neural machine translation. In: Proceedings of the 1st Conference on Machine Translation. 2016, 83–91

  31. Aharoni R, Goldberg Y. Towards string-to-tree neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Short Papers). 2017, 132–140

  32. Chen H, Huang S, Chiang D, Chen J. Improved neural machine translation with a syntax-aware encoder and decoder. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017, 1936–1945

  33. Han D, Li J, Li Y, Zhang M, Zhou G. Explicitly modeling word translations in neural machine translation. ACM Transactions on Asian and Low-Resource Language Information Processing, 2019, 19(1): 1–17

    Article  Google Scholar 

  34. Langlais P, Simard M, Veronis J. Methods and practical issues in evaluating alignment techniques. In: Processings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics. 1998, 711–717

  35. Kit C, Webster J J, Sin K K, Pan H, Li H. Clause alignment for bilingual Hong Kong legal texts: a lexicalbased approach. International Journal of Corpus Linguistics, 2004, 9(1): 29–52

    Article  Google Scholar 

  36. Och F J, Ney H. A systematic comparison of various statistical alignment models. Computational Linguistics, 2003, 29(1): 19–51

    Article  Google Scholar 

  37. Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014, 1724–1734

  38. Sutskever I, Salakhutdinov R, Tenenbaum J B. Modelling relational data using bayesian clustered tensor factorization. In: Proceedings of the 22nd International Conference on Neural Information Processing Systems. 2009, 1821–1828

  39. Jenatton R, Roux N L, Bordes A, Obozinski G. A latent factor model for highly multi-relational data. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012, 3167–3175

  40. Collobert R, Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning. 2008, 160–167

  41. Zou W Y, Socher R, Cer D, Manning C D. Bilingual word embeddings for phrase-based machine translation. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 2013, 1393–1398

Download references

Acknowledgements

We thank the anonymous reviewers for their insightful comments and suggestions. This work was supported by the National Natural Science Foundation of China (Grant Nos. 61876120, 61673290).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Soochow University, Suzhou, 215006, China

    Ying Ding, Junhui Li, Zhengxian Gong & Guodong Zhou

Authors
  1. Ying Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junhui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengxian Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guodong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengxian Gong.

Additional information

Ying Ding received her BS degree in computer science from Huaiyin Normal University, China in 2016. She is now a Master student in computer science at Soochow University, China. Her current research interests include natural language processing, machine translation.

Junhui Li received his PhD degree in computer science from Soochow University, China in 2010. He is an associate professor in Soochow University, China. His main research interests include natural language processing, machine translation.

Zhengxian Gong received her PhD degree in computer science from Soochow University, China in 2014. She is an associate professor in Soochow University, China. Her main research interests include natural language processing, machine translation.

Guodong Zhou received his PhD degree in computer science from the National University of Singapore, Singapore in 1999. He is a distinguished professor in Soochow University, China. His research interests include natural language processing, information extraction and machine learning.

Supporting information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, Y., Li, J., Gong, Z. et al. Improving neural sentence alignment with word translation. Front. Comput. Sci. 15, 151302 (2021). https://doi.org/10.1007/s11704-019-9164-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-019-9164-3

Keywords

Search

Navigation