Abstract
Huge amount of unstructured data is posted on the cloud from various sources for the purpose of feedback and reviews. These review needs require classification for many a reasons and sentiment classification is one of them. Sentiment classification of these reviews quite difficult as they are arriving from many sources. A robust classifier is needed to deal with different data distributions. Traditional supervised machine learning approaches not works well as they require retraining when domain is changed. Deep learning techniques perform well to handle these situations, but they are more data hungry and computationally expensive.
Transfer learning is a feature in the cross-domain sentiment classification where features are transferred from one domain to another without any training. Moreover, transfer learning allows the domains, tasks, and distributions used in training and testing to be different. Therefor transfer learning mechanism is required to transfer the sentiment features across the domains.
This paper presents a transfer learning approach using pretrained language model, ELMO which helps in transferring sentiment features across domains. This model has been tested on text reviews posted on twitter data set and compared with deep learning methods with and without pretraining process, also our model delivers promising results. This model permits flexibility to plug and play parameters with target models with easier domain adaptivity and transfer sentiment features. Also, model enables sentiment classifiers by using the transferred features from an already trained domain and reuse the sentiment features by saving the time and training cost.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blitzer J, Dredze M and Pereira F, (2007) Biographies, bollywood, boom-boxes and blenders, In: association of computational linguistics, 440–447
Bollegala D, Weir D and Carroll J (2011) Using multiple sources to construct a sentiment sensitive thesaurus for cross-domain sentiment classification,” In: association of computational linguistics, pages 132–141, June 2011
Chen H, Sun M, Tu C, Lin Y and Liu Z, (2016) Neural sentiment classification with user and product attention, In: association of computational linguistics, 1650–1659, Nov 2016
Alexandra Chronopoulou, Christos Baziotis, Alexandros Potamianos, Wei Wang (2019) An embarrassingly simple approach for transfer learning from pretrained language models. In: Proceedings of 2019 association for computational linguistics, p 2089–2095, Jun 2019
Wanyun Cui, Guangyu Zheng, Zhiqiang Shen, Sihang Jiang, Wei Wang (2019) Transfer learning for sequences via learning to collocate. In: Proceedings of the 2019 conference paper at ICLR, Feb 2019
Dahou A, Xiong S, Zhou J, Houcine Haddoud M, Duan P (2016) Word embeddings and convolutional neural network for arabic sentiment classification. In: COLING 2016, the 26th international conference on computational linguistics, 2418–2427, Dec 2016
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of Oreilly AI conference, Apr 2019.
Glorot X, Bordes A and Bengio Y, (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach, in proceedings of the 28th international conference on machine learning, 2011, 513–520
Hai Z, Chang K, Kim J-J, Yang CC (2014) Identifying Features in Opinion Mining via Intrinsic and Extrinsic Domain Relevance. IEEE Transactions On Knowledge And Data Engineering 26(3):623–634
He Y, Liny C and Alani H, (2011) Automatically extracting polarity-bearing topics for cross-domain sentiment classification, In: association of computational linguistics, 123–131, June 2011
Howard J, Ruder S, “Universal Language Model Fine-tuning for Text Classification,” Proceedings of conference of ACL, Vol 1, page 328–339, Jul 2018.
Iyyer M, Manjunatha V, Boyd-Graber J, Daume H (2015) Deep unordered composition rivals syntactic methods for text classification. In: association of computational linguistics, 1681–1691, July 2015
Y Kim (2014) Convolutional neural networks for sentence classification. In: association of computational linguistics, 1746–1751, Oct 2014
Li F, Pan SJ, Jin O, Yang Q and Zhu X, (2012) Cross-domain co-extraction of sentiment and topic lexicons, In: association of computational linguistics, 410–419, July 2012
McCann B, Bradbury J, Xiong C, Socher R (2017) Learned in translation: contextualized word vectors. In: Proceedings of neural information processing systems, Vol 1, Aug 2017
Pan SJ, Yang Q (2010a) A Survey on Transfer Learning. IEEE Trans Knowl Data Eng 22(10):1345–1359
Pan S J, Ni X, Sun J-T, Yang Q and Chen Z, Cross-domain sentiment classification via spectral feature alignment, In: ACM 2010, 751–760
Pan C, Huang J, Gong J, Yuan X (2019) Few-shot transfer learning for text classification with lightweight word embedding based models. IEEE J 7:53296–53304
ME. Peters, Ammar W, Bhagavatula C, Power R (2017) Semi-supervised sequence tagging with bidirectional language models. In: Proceedings of conference of ACL, Apr 2017
Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of conference of acl: human language technologies, vol 1, pp 2227–2237
Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment Treebank. In: association of computational linguistics, 1631–1642, Oct 2013
Song Y, Wang J, Liang Z, Liu Z, and Jiang T, “Utilizing BERT intermediate layers for aspect based sentiment analysis and natural language inference,” arXiv preprint, 2020.
Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: EMNLP 2016
D Tang, B Qin, X Feng, T Liu (2016) Effective LSTMs for target-dependent sentiment classification. In: COLING 2016, the 26th international conference on computational linguistics, 3298–3307, Dec 2016
Tu Z, He Y, Foster J, Genabith J, Liu Q and Lin S, (2012) Identifying high-impact sub-structures for convolution kernels in document-level sentiment classification, In: association of computational linguistics, 338–343, July 2012.
Xia R, Zong C, Hu X and Cambria E, Feature ensemble plus sample selection: domain adaptation for sentiment classification, Proceedings of the twenty-fourth international joint conference on artificial intelligence (IJCAI 2015), 4229–4233.
Xiao Y, Kyunghyun Cho (2016) Efficient character-level document classification by combining convolution and recurrent layers. In: computational language, Feb 2016
Xu J, Chen D, Qiu X and Huang X, (2016) Cached long short-term memory neural networks for document-level sentiment classification,” In: association of computational linguistics, 1660–1669, Nov 2016
Yang W, Lu W, Zheng V (2017) A simple regularization-based algorithm for learning cross-domain word embeddings. In: Proceedings of the 2017 conference on empirical methods in natural language processing, 2898–2904, Sep 2017
Yi Yang, J Eisenstein (2015) Unsupervised multi-domain adaptation with feature embeddings. In: association of computational linguistics, 672–682, June 2015
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Krishna, B.V., Pandey, A.K. & Kumar, A.P.S. Universally domain adaptive algorithm for sentiment classification using transfer learning approach. Int J Syst Assur Eng Manag 12, 542–552 (2021). https://doi.org/10.1007/s13198-021-01113-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13198-021-01113-y