Skip to main content
SpringerLink
Log in

Hybrid Deep Learning Models for Thai Sentiment Analysis

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Many people use social media in their daily life for entertainment, business, personal communication, and catching up with friends. In social media marketing, sentiment analysis is one of the most popular research topics because it can be employed to perform brand or market research monitoring and to keep an eye on the competitors. Machine learning algorithms have been utilized to carry out the task. In addition, sentiment analysis is essential in cognitive computing. Currently, there are still a limited number of Thai sentiment analysis research. This paper proposes a framework for sentiment analysis in Thai along with Thai-SenticNet5 corpus. The framework employs different types of features, namely, word embedding, part-of-speech, sentic features, and all combinations of these features. Furthermore, we fused deep learning algorithms—convolutional neural network (CNN) and bidirectional long short-term memory (BLSTM)—in different ways and compare it to several other fused combinations. Three datasets in Thai were used in this work: ThaiTales, ThaiEconTwitter, and Wisesight datasets. The experimental results show that combining all three features and fusing deep learning algorithms were able to improve overall performance. The best hybrid deep learning was BLSTM-CNN that achieved F1-scores of 0.7436, 0.7707, and 0.5521, on ThaiTales, ThaiEconTwitter, and Wisesight datasets, respectively. According to the experimental results, we conclude that feature combination and hybrid deep learning algorithms can improve the overall performances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://wisesight.com/zocialeye

  2. https://evolve24.com/

  3. A group of English words that have synonymous meaning with Thai words.

References

  1. Bright J, Margetts H, Hale S, Yasseri T. 2014. The use of social media for research and analysis: a feasibility study. In: DWP ad hoc research report, no. 13. London: Department for Work and Pensions.

  2. Herhold K. 2014. Why digital marketing is an essential investment. Accessed 7 Jan 2019. https://www.business2community.com/digital-marketing/why-digital-marketing-is-an-essential-investment-02129487.

  3. Cambria E, Grassi M, Hussain A, Havasi C. Sentic computing for social media marketing. Multimed Tools Appl 2012;59(2):557–577.

    Article  Google Scholar 

  4. Cambria E. Affective computing and sentiment analysis. IEEE Intell Sys 2016;31(2):102–107.

    Article  Google Scholar 

  5. Kim Y. Convolutional neural networks for sentence classification. Proceedings of the international conference on empirical methods in natural language processing, (EMNLP 2014), 25-29 October 2014, Doha, Qatar. p. 1746–1751; 2014.

  6. Wang Y, Huang M, Zhu X, Zhao L. Attention-based LSTM for aspect-level sentiment classification. Proceedings of the 2016 conference on empirical methods in natural language processing (EMNLP 2016), 1-4 November 2016, Austin, Texas, USA. p. 606–615; 2016.

  7. Plank B, Søgaard A, Goldberg Y . Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. Proceedings of the 54th annual meeting of the association for computational linguistics (ACL 2016), 7-12 August 2016, Berlin, Germany. p. 412–418; 2016.

  8. Wang P, Li Z, Hou Y, Li W. 2016. Combining convnets with hand-crafted features for action recognition based on an HMM-SVM classifier. arXiv:160200749.

  9. Chavaltada C, Pasupa K, Hardoon DR. Combining multiple features for product categorisation by multiple kernel learning. Proceedings of the 14th international conference on computing and information technology (IC2IT2018), 5-6 July 2018, Chiang Mai, Thailand. p. 3–12; 2018.

  10. Pasupa K, Sunhem W, Loo CK. A hybrid approach to build face shape classifier for hairstyle recommender system. Expert Sys Appl 2019;120:14–32.

    Article  Google Scholar 

  11. Pasupa K, Seneewong Na Ayutthaya T. Thai sentiment analysis with deep learning techniques: a comparative study based on word embedding, POS-tag, and sentic features. Sustain Cities Soc 2019;50:101615.

    Article  Google Scholar 

  12. Seneewong Na Ayutthaya T, Pasupa K. Thai sentiment analysis via bidirectional LSTM-CNN model with embedding vectors and sentic features. Proceedings of the 13th International joint symposium on artificial intelligence and natural language processing (iSAI-NLP 2018), 15-17 November 2018, Pattaya, Thailand. p. 84–89; 2018.

  13. Wang J, Yu LC, Lai KR, Zhang X. Dimensional sentiment analysis using a regional CNN-LSTM model. Proceedings of the 54th annual meeting of the association of computational linguistics (ACL 2016), 7-12 August 2016, Berlin, Germany. p. 225–230; 2016.

  14. Lin S, Xie H, Yu LC, Lai KR. SentiNLP at IJCNLP-2017 task 4: Customer feedback analysis using a bi-LSTM-CNN model. Proceedings of the 8th international joint conference on natural language processing (IJCNLP 2017), 27 November-1 December 2017, Taipei, Taiwan. p. 149–154; 2017.

  15. Minaee S, Azimi E, Abdolrashidi AA. 2019. Deep-sentiment: Sentiment analysis using ensemble of CNN and Bi-LSTM models. arXiv:190404206.

  16. Lertsuksakda R, Netisopakul P, Pasupa K. Thai Sentiment terms construction using the hourglass of emotions. Proceedings of the 6th international conference on knowledge and smart technology (KST 2014), 30-31 January 2014, Chonburi, Thailand. p. 46–50; 2014.

  17. Surin B. 2019. Volubilis 9.5 (2019.1)–107,000 Entries. Accessed 2 Jan 2019. http://belisan-volubilis.blogspot.com.

  18. Bird S, Klein E, Loper E. 2009. Natural language processing with python. 1st ed. O’Reilly Media Inc.

  19. Pasupa K, Netisopakul P, Lertsuksakda R. Sentiment analysis on thai children stories. Artif Life Robot 2016;21(3):357–364.

    Article  Google Scholar 

  20. Ofek N, Poria S, Rokach L, Cambria E, Hussain A, Shabtai A. Unsupervised commonsense knowledge enrichment for domain-specific sentiment analysis. Cognit Comput 2016;8(3):467–477.

    Article  Google Scholar 

  21. Oneto L, Bisio F, Cambria E, Anguita D. Semi-supervised learning for affective common-sense reasoning. Cognit Comput 2017;9(1):18–42.

    Article  Google Scholar 

  22. Wang J, Sun C, Li S, Wang J, Si L, Zhang M, et al. Human-like decision making: Document-level aspect sentiment classification via hierarchical reinforcement learning. Proceedings of the 2019 Conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP 2019), 3-7 November 2019, Hong Kong, China. Association for Computational Linguistics. p. 5580-5589. In: Inui K, Jiang J, Ng V, and Wan X, editors; 2019.

  23. Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R. Sentiment analysis of twitter data. Proceedings of the workshop on language in social media (LSM 2011), 23 June 2011, Portland, Oregon, USA. p. 30-38; 2011.

  24. Phienthrakul T, Kijsirikul B, Takamura H, Okumura M. Sentiment classification with support vector machines and multiple kernel functions. Proceedings of the 16th international conference on neural information processing (ICONIP 2009), 1-5 December, 2009, Bangkok, Thailand. vol. 5864 of Lecture Notes in Computer Science. p. 583–592; 2009.

  25. Flender M, Gips C. Sentiment analysis of a german twitter-corpus. Proceedings of the lernen, wissen, daten, analysen conference (LWDA 2017), 11-13 September 2017, Rostock, Germany. vol. 1917 of CEUR Workshop Proceedings. p. 25; 2017.

  26. Abdaoui A. 2016. French social media mining: expertise and sentiment université Montpellier.

  27. Peng H, Cambria E, Hussain A. A review of sentiment analysis research in chinese language. Cognit Computat 2017;9(4):423–435.

    Article  Google Scholar 

  28. Hussein DMEDM. A survey on sentiment analysis challenges. J King Saud Univ Eng Sci 2018;30 (4):330–338.

    Google Scholar 

  29. Vilares D, Peng H, Satapathy R, Cambria E. Babelsenticnet: A commonsense reasoning framework for multilingual sentiment analysis. Proceedings of the IEEE symposium series on computational intelligence (SSCI 2018), 18-21 November 2018, Bangalore, India. p. 1292–1298; 2018.

  30. Sriphaew K, Takamura H, Okumura M. Sentiment analysis for thai natural language processing. Proceedings of the 2nd thailand-japan international academic conference (TJIA 2009), 20 November 2009, Kyoto, Japan. p. 123–124; 2009.

  31. Boonkwan P. 2017. Personal communication.

  32. Inrak P, Sinthupinyo S. Applying latent semantic analysis to classify emotions in Thai text. Proceedings of the 2nd international conference on computer engineering and technology (ICCET 2010), 16-18 April 2010, Chengdu, China. p. 450–454; 2010.

  33. Haruechaiyasak C, Kongthon A, Palingoon P, Sangkeettrakarn C. Constructing Thai opinion mining resource: a case study on hotel reviews. Proceedings of the 8th workshop on asian language resources, 21-22 August 2010, Beijing, China. p. 64–71; 2010.

  34. Haruechaiyasak C, Kongthon A, Palingoon P, Trakultaweekoon K. S-Sense: A sentiment analysis framework for social media sensing. Proceedings of the IJCNLP workshop on natural language processing for social media (SocialNLP 2013), 14-18 October, 2013, Nagoya, Japan. p. 6–13; 2013.

  35. Damdoung W, Chanlekha H, Kawtrakul A. A context-induced bootstrapping approach for constructing contextual-dependent Thai sentiment lexicon. Proceedings of the 10th international symposium on natural language processing (SNLP 2013), 28 October-30 October 2013, Phuket, Thailand. p. 225–230; 2013.

  36. Sarakit P, Theeramunkong T, Haruechaiyasak C, Okumura M. Classifying emotion in Thai youtube comments. Proceedings of the 6th international conference of information and communication technology for embedded systems (IC-ICTES 2015), 22-24 March 2015, Hua-hin, Thailand. p. 1–5; 2015.

  37. Netisopakul P, Chattupan A. Thai stock news sentiment classification using wordpair features. Proceedings of the 29th pacific asia conference on language, information and computation, (PACLIC 2015), 30 October-1 November 2015, Shanghai, China. p. 188–195; 2015.

  38. Vateekul P, Koomsubha T. A study of sentiment analysis using deep learning techniques on Thai Twitter data. Proceedings of the 13th international joint conference on computer science and software engineering (JCSSE 2016), 13-15 July 2016, Khon Kaen, Thailand. p. 1–6; 2016.

  39. Cambria E, Havasi C, Hussain A. Senticnet 2: A semantic and affective resource for opinion mining and sentiment analysis. Proceedings of the 25 international florida artificial intelligence research society conference (FLAIRS 2012), 23-25 May 2012, Florida, USA. p. 202–207; 2012.

  40. Netisopakul P, Pasupa K, Lertsuksakda R. Hypothesis testing based on observation from Thai sentiment classification. Artif Life Robot 2017;22(2):184–190.

    Article  Google Scholar 

  41. Zhang L, Wang S, Liu B. Deep learning for sentiment analysis: a survey. Data Min Knowl Discov 2018;8(4):e1253.

    Google Scholar 

  42. Li Y, Pan Q, Yang T, Wang S, Tang J, Cambria E. Learning word representations for sentiment analysis. Cognit Comput 2017;9(6):843–851.

    Article  Google Scholar 

  43. Severyn A, Moschitti A. Twitter sentiment analysis with deep convolutional neural networks. Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval (SIGIR 2015), 9-13 August 2015, Santiago, Chile. p. 959–962; 2015.

  44. Graves A, Fernández S, Schmidhuber J. Bidirectional LSTM networks for improved phoneme classification and recognition. Proceedings of the 15th international conference on artificial neural networks (ICANN 2005), 11-15 September 2005, Warsaw, Poland. p. 799-804; 2005.

  45. Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw 2005;18(5-6):602–610.

    Article  Google Scholar 

  46. Xu K, Xie L, Yao K. Investigating LSTM for punctuation prediction. Proceedings of the 10th international symposium on chinese spoken language processing (ISCSLP 2016), 17-20 October 2016, Tianjin, China p. 1–5; 2016.

  47. Pang B, stars LEEL. Seeing exploiting class relationships for sentiment categorization with respect to rating scales. Proceedings of the 43rd annual meeting on association for computational linguistics (ACL 2005), 25-30 June 2005, Michigan, USA. p. 115–124; 2005.

  48. Ouyang X, Zhou P, Li CH, Liu L. Sentiment analysis using convolutional neural network. Proceedings of the IEEE international conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing. pervasive intelligence and computing, (CIT/IUCC/DASC/PICom 2015), 26-28 October 2015, Liverpool, UK. p. 2359–2364; 2015.

  49. Nowak J, Taspinar A, Scherer R. LSTM Recurrent neural networks for short text and sentiment classification. Proceedings of the international conference on artificial intelligence and soft computing (ICAISC 2017), 11-15 June 2017, Zakopane, Poland. p. 553–562; 2017.

  50. Socher R, Perelygin A, Wu J, Chuang J, Manning C, Ng A, et al. Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the international conference on empirical methods in natural language processing (EMNLP 2013), 18-21 October 2013, Washington, USA. p. 1631–1642; 2013.

  51. Yu LC, Lee LH, Hao S, Wang J, He Y, Hu J, et al. Building Chinese affective resources in valence-arousal dimensions. Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies (NAACL HLT 2016), 12-17 June 2016, San Diego, California, USA. p. 540–545; 2016.

  52. Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C. Learning word vectors for sentiment analysis. Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies (ACL HLT 2011), 19-24 June 2011, Oregon, USA. p. 142–150; 2011.

  53. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Proceedings of the advances in neural information processing systems (NIPS 2017), 4-9 December 2017, CA, USA. p. 5998–6008; 2017.

  54. Ma D, Li S, Zhang X, Wang H. Interactive attention networks for aspect-level sentiment classification. Proceedings of the 26th international joint conference on artificial intelligence (IJCAI 2017), 19-25 August 2017, Melbourne, Australia. p. 4068–4074; 2017.

  55. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S. SemEval-2014 Task 4: aspect based sentiment analysis. Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), August 2014, Dublin, Ireland. p. 27–35; 2014.

  56. PyThaiNLP. 2019. PyThaiNLP: Thai natural language processing in python. GitHub. Accessed 2 Jan 2019. https://github.com/PyThaiNLP/pythainlp.

  57. Meknavin S, Charoenpornsawat P, Kijsirikul B. Feature-based Thai word segmentation. Proceedings of the natural language processing pacific rim symposium (NLPRS 1997), 2-4 December 1997, Phuket, Thailand. p. 1–6; 1997.

  58. Aroonmanakun W. Collocation and Thai word segmentation. Proceedings of the 5th symposium on natural language processing (SNLP) & 5th Oriental COCOSDA Workshop, 9-11 May 2002 Huahin, Thailand. p. 68–75; 2002.

  59. Haruechaiyasak C, Kongyoung S, Dailey M. A comparative study on Thai word segmentation approaches. Proceedings of the 5th international conference on electrical engineering/electronics, computer, telecommunications and information technology (ECTI-CON 2008), 14-17 May 2008, Krabi, Thailand. p. 125–128; 2008.

  60. Theeramunkong T, Sornlertlamvanich V, Tanhermhong T, Chinnan W. Character cluster based thai information retrieval. Proceedings of the 5th international workshop information retrieval with asian languages (IRAL 2002), 30 September-1 October 2000, Hong Kong, China. p. 75–80; 2000.

  61. Norvig P. 2016. How to write a spelling corrector. Accessed 2 Jan 2019. https://norvig.com/spell-correct.html.

  62. Sornlertlamvanich V, Takahashi N, Isahara H. 1998. Thai part-of-speech tagged corpus: ORCHID.

  63. Nivre J, De Marneffe MC, Ginter F, Goldberg Y, Hajic J, Manning CD, et al. Universal dependencies v1: A multilingual treebank collection’. Proceedings of the 10th international conference on language resources and evaluation (LREC 2016). p. 1659–1666; 2016.

  64. Eisner B, Rocktäschel T, Augenstein I, Bošnjak M, Riedel S. Emoji2vec: Learning Emoji representations from their description. Proceedings of the 4th international workshop on natural language processing for social media, Austin, TX, USA. p. 48–54; 2016.

  65. Kim T, Wurster K. 2015. Emoji terminal output for Python. GitHub. Accessed 2 Jan 2019. https://github.com/carpedm20/emoji.

  66. Cambria E, Speer R, Havasi C, Hussain A. Senticnet: A publicly available semantic resource for opinion mining. Proceedings of the 2010 AAAI fall symposium: commonsense knowledge, 11-13 November 2010, Arlington, Virginia, USA. vol. FS-10-02 of AAAI Technical Report. p. 14–18; 2010.

  67. Wang J, Oard DW. 2006. Combining bidirectional translation and synonymy for cross-language information retrieval.

  68. Thoongsup S, Robkop K, Mokarat C, Sinthurahat T, Charoenporn T, Sornlertlamvanich V, et al. Thai wordnet construction. Proceedings of the 7th workshop on asian language resources (ALR 2009), 6-7 August 2009, Singapore. p. 139–144; 2009.

  69. Netisopakul P, Thong-iad K. Thai sentiment resource using Thai WordNet. Proceedings of the 12th international conference on complex, intelligent, and software intensive systems (CISIS 2018), 4-6 July 2018, Matsue, Japan. p. 329–340 ; 2018.

  70. Cambria E, Poria S, Hazarika D, Kwok K. Senticnet 5: Discovering conceptual primitives for sentiment analysis by means of context embeddings. Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI-18), 2-7 February 2018, Louisiana, USA. p. 1795–1802; 2018.

  71. Mikolov T, Chen K, Corrado G, Dean J. 2013. Efficient estimation of word representations in vector space. arXiv:13013781.

  72. Pennington J, Socher R, Manning C. Glove: global vectors for word representation. Proceedings of the international conference on empirical methods in natural language processing, (EMNLP 2014), 25-29 October 2014, Doha, Qatar. p. 1532-1543; 2014.

  73. Howard J, Ruder S. 2018. Universal language model fine-tuning for text classification. arXiv:180106146.

  74. Polpanumas C. 2019. ULMFit language modeling, text feature extraction, and text classification in Thai language. GitHub. Accessed 2 Jan 2019. https://github.com/cstorm125/thai2fit.

  75. Merity S, Keskar NS, Socher R. 2017. Regularizing and optimizing LSTM language models. arXiv:170802182.

  76. Cambria E, Livingstone A, Hussain A. 2012. The hourglass of emotions: Springer, Berlin.

  77. Susanto Y, Livingstone A, Ng BC, Cambria E. The hourglass model revisited. IEEE Intelligent Systems 2020;35(5):96–102.

    Article  Google Scholar 

  78. Sundermeyer M, Schlüter R, Ney H. 2012. LSTM neural networks for language modeling.

  79. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014;15(1):1929–1958.

    MathSciNet  MATH  Google Scholar 

  80. Agarap AF. 2018. Deep learning using rectified linear units (ReLU). arXiv:170802182.

  81. Khunkwang P. 2017. A dictionary-based sentiment classification approach for business news on Twitter [B.Sc. Thesis]. Faculty of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Bangkok, hailand.

  82. Kingma DP, Ba J. 2014. Adam: A method for stochastic optimization. arXiv:14126980.

  83. Siegel S, Castellian NJ. Nonparametric statistics for the behavioral sciences, 2nd ed. Singapore: McGraw-Hill; 1988.

    Google Scholar 

  84. Siegel A.F. Multiple t tests: some practical considerations. TESOL Q 1990;24(4):773–775.

    Article  Google Scholar 

  85. Abdi A, Shamsuddin SM, Hasan S, Piran J. Deep learning-based sentiment classification of evaluative text based on multi-feature fusion. Inform Process Manag 2019;56:1245–1259.

    Article  Google Scholar 

  86. Pasupa K, Vatathanavaro S, Tungjitnob S. 2020. Convolutional neural networks based focal loss for class imbalance problem: a case study of canine red blood cells morphology classification. Journal of Ambient Intelligence and Humanized Computing.

  87. Cambria E, Li Y, Xing FZ, Poria S, Kwok K. SenticNet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis. Proceedings of the 29th ACM conference on information and knowledge management (CIKM 2020), 19-23 October 2020, Virtual Event, Ireland. Association for Computing Machinery; in press. p. 1–9; 2020.

Download references

Funding

This research was supported by the Faculty of Information Technology, King Mongkut’s Institute of Technology Ladkrabang.

Author information

Authors and Affiliations

  1. Faculty of Information Technology, King Mongkut’s Institute of Technology Ladkrabang, Bangkok, 10520, Thailand

    Kitsuchart Pasupa & Thititorn Seneewong Na Ayutthaya

Authors
  1. Kitsuchart Pasupa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thititorn Seneewong Na Ayutthaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kitsuchart Pasupa.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: A Decade of Sentic Computing

Guest Editors: Erik Cambria, Amir Hussain

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pasupa, K., Seneewong Na Ayutthaya, T. Hybrid Deep Learning Models for Thai Sentiment Analysis. Cogn Comput 14, 167–193 (2022). https://doi.org/10.1007/s12559-020-09770-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-020-09770-0

Keywords

Search

Navigation