Skip to main content
SpringerLink
Log in

Is position important? deep multi-task learning for aspect-based sentiment analysis

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The position information of aspect is essential and useful for aspect-based sentiment analysis, while how to model the position of the aspect effectively during aspect-based sentiment analysis has not been well studied. Inspired by the intuition that the position prediction can help boost the performance of aspect-based sentiment analysis, we propose a D eep M ulti-T ask L earning (DMTL) model, which handles sentiment prediction (SP) and position prediction (PP) simultaneously. In particular, we first use a shared layer to learn the common features of the two tasks. Then, two task-specific layers are utilized to learn the features specific to the tasks and perform position prediction and sentiment prediction in parallel. Inspired by autoencoder structure, we design a position-aware attention and a deep bi-directional LSTM (DBi-LSTM) model for sentiment prediction and position prediction respectively to capture the position information better. Extensive experiments on four benchmark datasets show that our approach can effectively improve the performance of aspect-based sentiment analysis compared with the strong baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Available at: http://alt.qcri.org/semeval2014/task4/

  2. Available at: http://alt.qcri.org/semeval2015/task12/

  3. Available at: http://alt.qcri.org/semeval2016/task5/

  4. Available at: https://pytorch.org/

  5. Available at: https://www.yelp.com/dataset/challenge

  6. Available at: http://jmcauley.ucsd.edu/data/amazon/

  7. http://www.yorku.ca/adersim

  8. http://brainalliance.ca

References

  1. Abualigah L (2020) Multi-verse optimizer algorithm: a comprehensive survey of its results, variants, and applications. Neural Comput Appl 1–21

  2. Abualigah L, Diabat A (2020) A novel hybrid antlion optimization algorithm for multi-objective task scheduling problems in cloud computing environments. Clust Comput 1–19

  3. Abualigah LM, Khader AT (2017) Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering. J Supercomput 73:4773–4795

    Article  Google Scholar 

  4. Abualigah LM, Khader AT, Hanandeh ES (2018) A combination of objective functions and hybrid krill herd algorithm for text document clustering analysis. Eng Appl Artif Intell 73:111–125

    Article  Google Scholar 

  5. Abualigah LM, Khader AT, Hanandeh ES (2018) Hybrid clustering analysis using improved krill herd algorithm. Appl Intell 48:4047–4071

    Article  Google Scholar 

  6. Abualigah LM, Khader AT, Hanandeh ES (2018) A new feature selection method to improve the document clustering using particle swarm optimization algorithm. J Computat Sci 25:456–466

    Article  Google Scholar 

  7. Abualigah LMQ (2019). Feature selection and enhanced krill herd algorithm for text document clustering

  8. Abualigah LMQ, Hanandeh ES (2015) Applying genetic algorithms to information retrieval using vector space model. Int J Comput Scie Eng Appl 5:19

    Google Scholar 

  9. Appel O, Chiclana F, Carter J, Fujita H (2016) A hybrid approach to the sentiment analysis problem at the sentence level. Knowl-Based Syst 108:110–124

    Article  Google Scholar 

  10. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473

  11. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: EMNLP, pp 452–461

  12. Cheng J, Zhao S, Zhang J, King I, Zhang X, Wang H (2017) Aspect-level sentiment classification with heat (hierarchical attention) network. In: CIKM, pp 97–106

  13. Collobert R, Weston J (2008) A unified architecture for natural language processing: Deep neural networks with multitask learning. In: ICML, pp 160–167

  14. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537

    MATH  Google Scholar 

  15. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: ACL, vol 2, pp 49–54

  16. Gu S, Zhang L, Hou Y, Song Y (2018) A position-aware bidirectional attention network for aspect-level sentiment analysis. In: COLING, pp 774–784

  17. He R, Lee WS, Ng HT, Dahlmeier D (2018) Exploiting document knowledge for aspect-level sentiment classification. In: ACL, pp 579–585

  18. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780

    Article  Google Scholar 

  19. Jiang L, Yu M, Zhou M, Liu X, Zhao T (2011) Target-dependent twitter sentiment classification. In: ACL, pp 151–160

  20. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: ICLR, vol 5

  21. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) NRC-canada-2014: Detecting aspects and sentiment in customer reviews. In: Semeval, pp 437–442

  22. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In: ACL, pp 946–956

  23. Li X, Lam W (2017) Deep multi-task learning for aspect term extraction with memory interaction. In: EMNLP, pp 2886–2892

  24. Liu B (2012) Sentiment analysis and opinion mining. Synth. Lect. Hum. Lang. Technol. 5:1–167

    Article  Google Scholar 

  25. Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. In: IJCAI, pp 912–921

  26. Liu X, Gao J, He X, Deng L, Duh K, Wang Y (2015) Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: NAACL, pp 912–921

  27. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: IJCAI, pp 4068–4074

  28. Ma Y, Peng H, Cambria E (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In: AAAI, pp 5876–5883

  29. Nguyen TH, Shirai K (2015) Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In: EMNLP, pp 2509–2514

  30. Pang B, Lee L, Vaithyanathan S (2002) Thumbs up?: sentiment classification using machine learning techniques. In: EMNLP, pp 79–86

  31. Pang B, Lee L, et al. (2008) Opinion mining and sentiment analysis. Foundations and Trends®, in Information Retrieval 2:1–135

    Article  Google Scholar 

  32. Pennington J, Socher R, Manning C (2014) Glove: Global vectors for word representation. In: EMNLP, pp 1532–1543

  33. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Mohammad AS, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O et al (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: Semeval, pp 19–30

  34. Pota M, Marulli F, Esposito M, De Pietro G, Fujita H (2019) Multilingual pos tagging by a composite deep architecture based on character-level features and on-the-fly enriched word embeddings. Knowl-Based Syst 164:309–323

    Article  Google Scholar 

  35. Schouten K, Frasincar F (2016) Survey on aspect-level sentiment analysis. IEEE TKDE 28:813–830

    Google Scholar 

  36. Seo M, Kembhavi A, Farhadi A, Hajishirzi H (2016) Bidirectional attention flow for machine comprehension. arXiv:1611.01603

  37. Sukhbaatar S, Weston J, Fergus R, et al. (2015) End-to-end memory networks. In: NIPS, pp 2440–2448

  38. Tang D, Qin B, Feng X, Liu T (2016) Effective lstms for target-dependent sentiment classification. In: COLING, pp 3298– 3307

  39. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: EMNLP, pp 214–224

  40. Tran VC, Nguyen NT, Fujita H, Hoang DT, Hwang D (2017) A combination of active learning and self-learning for named entity recognition on twitter using conditional random fields. Knowl-Based Syst 132:179–187

    Article  Google Scholar 

  41. Vo DT, Zhang Y (2015) Target-dependent twitter sentiment classification with rich automatic features. In: IJCAI, pp 1347–1353

  42. Wallaart O, Frasincar F (2019) A hybrid approach for aspect-based sentiment analysis using a lexicalized domain ontology and attentional neural models. In: European semantic web conference, pp 363–378

  43. Wang Y, Huang M, Zhao L, et al. (2016) Attention-based lstm for aspect-level sentiment classification. In: EMNLP, pp 606– 615

  44. Yu J, Zha ZJ, Wang M, Chua TS (2011) Aspect ranking: Identifying important product aspects from online consumer reviews. In: ACL, pp 1496–1505

  45. Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: COLING, pp 2335–2344

  46. Zhang M, Zhang Y, Vo DT (2016) Gated neural networks for targeted sentiment analysis. In: AAAI, pp 3087–3093

  47. Zhang Y, Yang Y, Li T, Fujita H (2019) A multitask multiview clustering algorithm in heterogeneous situations based on lle and le. Knowl-Based Syst 163:776–786

    Article  Google Scholar 

  48. Zhou J, Chen Q, Huang JX, Hu QV, He L (2020) Position-aware hierarchical transfer model for aspect-level sentiment classification. Inf Sci 513:1–16

    Article  Google Scholar 

  49. Zhou J, Huang JX, Chen Q, Hu QV, Wang T, He L (2019) Deep learning for aspect-level sentiment classification: survey, vision and challenges IEEE Access

Download references

Acknowledgments

We greatly appreciate anonymous reviewers and the associate editor for their valuable and high quality comments that greatly helped to improve the quality of this article. This research is funded by the Science and Technology Commission of Shanghai Municipality (No. 18511105502) and by Xiaoi Research. The computation is performed in ECNU Multifunctional Platform for Innovation (001). This research is also supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada, an NSERC CREATE award in ADERSIM,Footnote 7 the York Research Chairs (YRC) program and an ORF-RE (Ontario Research Fund-Research Excellence) award in BRAIN Alliance.Footnote 8

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, East China Normal University, Shanghai, 200241, China

    Jie Zhou

  2. Information Retrieval and Knowledge Management Research Lab, York University, Toronto, Ontario, M3J 1P3, Canada

    Jimmy Xiangji Huang

  3. The School of Computer Science, Ryerson University, Toronto, Ontario, M5B 2K3, Canada

    Qinmin Vivian Hu

  4. School of Computer Science and Technology, East China Normal University, Shanghai, 200241, China

    Liang He

Authors
  1. Jie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jimmy Xiangji Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinmin Vivian Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Huang, J.X., Hu, Q.V. et al. Is position important? deep multi-task learning for aspect-based sentiment analysis. Appl Intell 50, 3367–3378 (2020). https://doi.org/10.1007/s10489-020-01760-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01760-x

Keywords

Search

Navigation