Skip to main content
SpringerLink
Log in

Aspect-based sentiment analysis for online reviews with hybrid attention networks

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Aspect-based sentiment analysis has received considerable attention in recent years because it can provide more detailed and specific user opinion information. Most existing methods based on recurrent neural networks usually suffer from two drawbacks: information loss for long sequences and a high time consumption. To address such issues, a hybrid attention model is proposed for aspect-based sentiment analysis in this paper, which utilizes only attention mechanisms rather than recurrent or convolutional structures. In this model, a self-attention mechanism and an aspect-attention mechanism are designed for generating the semantic representation at the word and sentence levels respectively. Two auxiliary features of word location and part-of-speech are also explored for the proposed models to enhance the semantic representation of sentences. A series of experiments are conducted on three benchmark datasets for aspect-based sentiment analysis. Experimental results demonstrate the advantage of the proposed models for both efficiency and execution effectiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://stanfordnlp.github.io/CoreNLP/

  2. https://tensorow.google.org/

  3. http://alt.qcri.org/semeval2014/task4/index.php

  4. https://www.datafountain.cn/projects/2018CCF/index.html

  5. http://goo.gl/5Enpu7

  6. https://nlp.stanford.edu/projects/glove/

  7. https://radimrehurek.com/gensim/models/word2vec.html

  8. https://github.com/google-research/bert

  9. https://www.tensorflow.org/

References

  1. Baccianella, S., Esuli, A., Sebastiani, F.: Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: Lrec, vol. 10, pp. 2200–2204 (2010)

  2. Bafna, K., Toshniwal, D.: Feature based summarization of customers’ reviews of online products. In: 17Th International Conference in Knowledge Based and Intelligent Information and Engineering Systems, pp. 142–151 (2013)

  3. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR 2015 (2015)

  4. Chen, P., Sun, Z., Bing, L., Yang, W.: Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 452–461 (2017)

  5. Cho, H., Kim, S., Lee, J., Lee, J.: Data-driven integration of multiple sentiment dictionaries for lexicon-based sentiment classification of product reviews. Knowl. Based Syst. 71, 61–71 (2014)

    Article  Google Scholar 

  6. Cho, K., van Merrienboer, B., Gülçehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1724–1734 (2014)

  7. Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning (2014)

  8. Do, H.H., Prasad, P.W.C., Maag, A., Alsadoon, A.: Deep learning for aspect-based sentiment analysis: A comparative review. Expert Syst. Appl. 118, 272–299 (2019)

    Article  Google Scholar 

  9. Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K.: Adaptive recursive neural network for target-dependent twitter sentiment classification. In: ACL 2014, pp. 49–54 (2014)

  10. Du, H., Xu, X., Cheng, X., Wu, D., Liu, Y., Yu, Z.: Aspect-specific sentimental word embedding for sentiment analysis of online reviews. In: Proceedings of the 25th International Conference on World Wide Web, pp. 29–30 (2016)

  11. Gamon, M., Aue, A.: Automatic identification of sentiment vocabulary: exploiting low association with known sentiment terms. In: Proceedings of the ACL Workshop on Feature Engineering for Machine Learning in Natural Language Processing, pp. 57–64 (2005)

  12. Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.N.: Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning, pp. 1243–1252 (2017)

  13. Guerini, M., Gatti, L., Turchi, M.: Sentiment analysis: How to derive prior polarities from sentiwordnet. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1259–1269 (2013)

  14. Hatzivassiloglou, V., McKeown, K.R.: Predicting the semantic orientation of adjectives. In: 35Th Annual Meeting of the Association for Computational Linguistics and 8Th Conference of the European Chapter of the Association for Computational Linguistics, pp. 174–181 (1997)

  15. Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 168–177 (2004)

  16. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization ICLR 2015 (2015)

  17. Kobayashi, N., Inui, K., Matsumoto, Y.: Extracting aspect-evaluation and aspect-of relations in opinion mining. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp. 1065–1074 (2007)

  18. Lazaridou, A., Titov, I., Sporleder, C.: A bayesian model for joint unsupervised induction of sentiment, aspect and discourse representations. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pp. 1630–1639 (2013)

  19. Li, F., Pan, S.J., Jin, O., Yang, Q., Zhu, X.: Cross-domain co-extraction of sentiment and topic lexicons. In: The 50Th Annual Meeting of the Association for Computational Linguistics - Volume 1: Long Papers, pp. 410–419 (2012)

  20. Lin, Y., Jiang, X., Li, Y., Zhang, J., Cai, G.: Semi-supervised collective extraction of opinion target and opinion word from online reviews based on active labeling. J. Intell. Fuzzy Syst. 33(6), 3949–3958 (2017)

    Article  Google Scholar 

  21. Liu, B., Zhang, L.: A survey of opinion mining and sentiment analysis. In: Mining Text Data, pp. 415–463 (2012)

  22. Liu, B., Zhang, L.: A survey of opinion mining and sentiment analysis. In: Mining Text Data, pp. 415–463 (2012)

  23. Liu, Q., Zhang, H., Zeng, Y., Huang, Z., Wu, Z.: Content attention model for aspect based sentiment analysis. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 1023–1032 (2018)

  24. Lu, B., Ott, M., Cardie, C., Tsou, B.K.: Multi-aspect sentiment analysis with topic models. In: 2011 IEEE 11Th International Conference on Data Mining Workshops, pp. 81–88 (2011)

  25. Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: EMNLP 2015, pp. 1412–1421 (2015)

  26. Ma, D., Li, S., Zhang, X., Wang, H.: Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp. 4068–4074 (2017)

  27. Manek, A.S., Shenoy, P.D., Mohan, M.C., Venugopal, K.R.: Aspect term extraction for sentiment analysis in large movie reviews using gini index feature selection method and SVM classifier. World Wide Web 20(2), 135–154 (2017)

    Article  Google Scholar 

  28. Marcheggiani, D., Täckström, O., Esuli, A., Sebastiani, F.: Hierarchical multi-label conditional random fields for aspect-oriented opinion mining Advances in Information Retrieval - 36Th European Conference on IR Research, pp. 273–285 (2014)

  29. Mikolov, T., Yih, W., Zweig, G.: Linguistic regularities in continuous space word representations. In: Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, pp. 746–751 (2013)

  30. Miller, G. A.: Wordnet: An electronic lexical database. MIT press (1998)

  31. Negi, S., Buitelaar, P.: INSIGHT galway: Syntactic and lexical features for aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation, pp. 346–350 (2014)

  32. Nguyen, T. H., Shirai, K.: Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2509–2514 (2015)

  33. Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Ret. 2(1–2), 1–135 (2008)

    Article  Google Scholar 

  34. Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: EMNLP 2014, pp. 1532–1543 (2014)

  35. Rao, Y., Lei, J., Wenyin, L., Li, Q., Chen, M.: Building emotional dictionary for sentiment analysis of online news. World Wide Web 17(4), 723–742 (2014)

    Article  Google Scholar 

  36. Samha, A.K., Li, Y., Zhang, J.: Aspect-based opinion mining from product reviews using conditional random fields. In: Thirteenth Australasian Data Mining Conference, pp. 119–128 (2015)

  37. Severyn, A., Moschitti, A.: Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 959–962 (2015)

  38. Shuang, K., Ren, X., Yang, Q., Li, R., Loo, J.: Aela-dlstms: Attention-enabled and location-aware double lstms for aspect-level sentiment classification. Neurocomputing 334, 25–34 (2019)

    Article  Google Scholar 

  39. Sun, C., Huang, L., Qiu, X.: Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, pp. 380–385 (2019)

  40. Sun, C., Shrivastava, A., Singh, S., Gupta, A.: Revisiting unreasonable effectiveness of data in deep learning era. In: IEEE ICCV 2017, pp. 843–852 (2017)

  41. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37(2), 267–307 (2011)

    Article  Google Scholar 

  42. Tang, D., Qin, B., Feng, X., Liu, T.: Effective Lstms for target-dependent sentiment classification. In: COLING 2016, pp. 3298–3307 (2016)

  43. Turney, P. D.: Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 417–424 (2002)

  44. Vaswani, A., Bengio, S., Brevdo, E., Chollet, F., Gomez, A.N., Gouws, S., Jones, L., Kaiser, L., Kalchbrenner, N., Parmar, N., Sepassi, R., Shazeer, N., Uszkoreit, J.: Tensor2tensor for neural machine translation. In: AMTA 2018, pp. 193–199 (2018)

  45. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: NIPS 2017, pp. 6000–6010 (2017)

  46. Wang, W., Pan, S.J., Dahlmeier, D., Xiao, X.: Recursive neural conditional random fields for aspect-based sentiment analysis. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 616–626 (2016)

  47. Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-Based LSTM for aspect-level sentiment classification. In: EMNLP 2016, pp. 606–615 (2016)

  48. Xu, H., Liu, B., Shu, L., Yu, P.: BERT post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2324–2335 (2019)

  49. Xu, H., Shu, L., Yu, P., Liu, B.: Understanding pre-trained BERT for aspect-based sentiment analysis. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 244–250 (2020)

  50. Xue, W., Li, T.: Aspect based sentiment analysis with gated convolutional networks. In: ACL 2018, Volume 1: Long Papers, pp. 2514–2523 (2018)

  51. Yuan, M., Ouyang, Y., Sheng, H.: Investigating association rules for sentiment classification of web reviews. J. Intell. Fuzzy Syst. 27(4), 2055–2065 (2014)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the editor and anonymous reviewers for their valuable comments and feedbacks. This work was supported by National Natural Science Foundation of China (No. 62062027 and U1711263), Guangxi Natural Science Foundations(No. 2018GXNSFDA281049, 2020GXNSFAA159012 and 2018GXNSFAA281326), Science and Technology Major Project of Guangxi Province (No. AA19046004), Guangxi Key Laboratory of Trusted Software (No. kx202021), and Innovation Project of Guangxi Graduate Education (No. YCBZ2021069).

Author information

Authors and Affiliations

  1. Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin, 541004, China

    Yuming Lin, Yu Fu, You Li & Guoyong Cai

  2. School of Data Science and Engineering, East China Normal University, Shanghai, 200062, China

    Aoying Zhou

Authors
  1. Yuming Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. You Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoyong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Aoying Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to You Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, Y., Fu, Y., Li, Y. et al. Aspect-based sentiment analysis for online reviews with hybrid attention networks. World Wide Web 24, 1215–1233 (2021). https://doi.org/10.1007/s11280-021-00898-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00898-z

Keywords

Search

Navigation