Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter Mouton September 18, 2020

The linguistic construction of sentiment expressions in student opinionated content: A corpus-based study

  • Aleksandar Kovačević , Olivera Grljević EMAIL logo , Zita Bošnjak and Gordana Svilengaćin

Abstract

Motivated by an increasing use of social media for the expression of personal stance towards a certain target, we analyse the language used to produce such opinionated content with expressions of sentiment, which represents the main data source for sentiment analysis. We use the first manually annotated corpus for sentiment analysis of the Serbian language developed for the service sector of higher education. Our study focuses on how various linguistic constructions, used in different context, influence the sentiment polarity of a text. Our findings indicate that sentiment expressions and negation have a most significant role in determining whether the text conveys positive, neutral, or negative sentiment, while intensifiers (words which either increase or decrease sentiment) have a considerable influence on sentiment intensity. We also present an analysis of the impact of conjunctions, conditional sentences, comparative and modal verbs, and pronouns on sentiment polarity and intensity. Based on the derived observations, we propose a set of rules that could be integrated with machine learning algorithms into an automated sentiment analysis system for the Serbian language. Our findings also make a much-needed contribution to the few currently available resources for natural language processing of Serbian.


Olivera Grljević Faculty of Economics in Subotica University of Novi Sad Segedinski put 9–11 Subotica, 24000 Serbia

References

Altrabsheh, N., M. Cocea and S. Fallahkha. 2014. “Learning sentiment from students’ feedback for real-time interventions in classrooms”. Paper presented at the Adaptive and Intelligent Systems – Third International Conference, volume 8779, Bournemouth, UK, 8–9 September. 40–49.Search in Google Scholar

Altrabsheh, N., M.G. Medhat and M. Cocea. 2013. “SA-E: Sentiment analysis for education”. Paper presented at the 5th KES International Conference on Intelligent Decision Technologies, Sesimbra, Portugal, June. 255.Search in Google Scholar

Balahur, A., J.M. Hermida and A. Montoyo. 2011. “Detecting implicit expressions of sentiment in text based on commonsense knowledge”. Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis, ACL-HLT 2011, Portland, Oregon, USA. 53–60.Search in Google Scholar

Blanco, E. and D. Moldovan. 2013. “Retrieving implicit positive meaning from negated statements”. Natural Language Engineering 20. 501–535.10.1017/S1351324913000041Search in Google Scholar

Bollen, J., H. Mao and X.J. Zeng. 2011. “Twitter mood predicts the stock market”. Journal of Computational Science 2(1). 1–8.10.1016/j.jocs.2010.12.007Search in Google Scholar

Bošnjak, Z., O. Grljević and M. Dimitrijević. 2018. “Primena inteligentnih tehnologija u visokom obrazovanju” [Application of intelligent technologies in higher education]. Anali Ekonomskog fakulteta u Subotici 54(39/2018). 291–303.10.5937/AnEkSub1839291BSearch in Google Scholar

Boucher, J. and C.E. Osgood. 1969. “The Pollyanna hypothesis”. Journal of Verbal Learning and Verbal Behaviour 8. 1–8.10.1016/S0022-5371(69)80002-2Search in Google Scholar

Broß, J. 2013. Aspect-oriented sentiment analysis of customer reviews using distant supervision techniques. (PhD dissertation, Freie Universität Berlin.)Search in Google Scholar

Caffi, C. and R.W. Janney. 1994. “Towards a pragmatics of emotive communication”. Journal of Pragmatics 22. 325–373.10.1016/0378-2166(94)90115-5Search in Google Scholar

Calders, T. and M. Pechenizkiy. 2012. “Introduction to the special section on educational data mining”. SIGKDD Explorations 13(2). 3–6.10.1145/2207243.2207245Search in Google Scholar

Carrillo de Albornoz, J. and L. Plaza. 2013. “An emotion-based model of negation, intensifiers, and modality for polarity and intensity classification”. Journal of the American Society for Information Science and Technology 64. 1618–1633.10.1002/asi.22859Search in Google Scholar

Chapman, W., B. Will, H. Paul, C.F. Gregory and B.G. Buchanan. 2001. “A simple algorithm for identifying negated findings and diseases in discharge summaries”. Journal of Biomedical Informatics 34(5). 301–310.10.1006/jbin.2001.1029Search in Google Scholar

Choi, Y. and C. Cardie. 2008. “Learning with compositional semantics as structural inference for subsentential sentiment analysis”. Proceedings of EMNLP 200810.3115/1613715.1613816Search in Google Scholar

Chung, S.F. 2011. “Uses of ter- in Malay: A corpus-based study”. Journal of Pragmatics 43(3). 799–813.10.1016/j.pragma.2010.10.004Search in Google Scholar

Collins, C. 2016. “Not even”. Natural Language Semantics 24(4). 291–303.10.1007/s11050-016-9124-5Search in Google Scholar

Dalal, M.K. and M.A. Zaveri. 2014. “Opinion mining from online user reviews using fuzzy linguistic hedges”. Applied Computational Intelligence and Soft Computing 2014, Article ID 735942.Search in Google Scholar

D’Avanzo, E. and G. Pilato. 2015. “Mining social network users opinions’ to aid buyers’ shopping decisions”. Computers in Human Behavior 51. 1284–1294.10.1016/j.chb.2014.11.081Search in Google Scholar

Dave, K., S. Lawrence and D.M. Pennock. 2003. “Mining the Peanut Gallery: Opinion extraction and semantic classification of product reviews”. Paper presented at the 12th International Conference on World Wide Web, Budapest, Hungary, 20–24 May.10.1145/775152.775226Search in Google Scholar

Fleiss, J. L. 1971. “Measuring nominal scale agreement among many raters”. Psychological Bulletin 76(5). 378–382.10.1037/h0031619Search in Google Scholar

Fleiss, J.L., B. Levin and M.C. Paik. 2003. Statistical methods for rates and proportions Oxford: Wiley.10.1002/0471445428Search in Google Scholar

Ganu, G., N. Elhadad and A. Marian. 2009. “Beyond the stars: Improving rating predictions using Review Text Content”. Paper presented at the 12th International Workshop on the Web and Databases (WebDB 2009), Providence, RI, USA, 28 June.Search in Google Scholar

Garrote, M., C. Kimura, K. Matsui, M.A. Sandoval and E. Takamori. 2013. “C-ORAL-JAPON: Corpus of spontaneous spoken Japanese”. Corpus Linguistics and Linguistic Theory 11(2). 373–392.10.1515/cllt-2013-0004Search in Google Scholar

González, A.O. and A.M. Ramos. 2013. “A comparative study of collocations in a native corpus and a learner corpus of Spanish”. Procedia – Social and Behavioral Sciences 95. 563–570.10.1016/j.sbspro.2013.10.683Search in Google Scholar

Grafsgaard, J.F., J.B. Wiggins, E.K. Boyer, E.N. Wiebe and J.C. Lester. 2013. “Embodied affect in tutorial dialogue: Student gesture and posture”. Paper presented at the16th International Conference on Artificial Intelligence in Education AIED Memphis, TN, USA, 9–13 July.10.1007/978-3-642-39112-5_1Search in Google Scholar

Grimmer, J. and B.M. Stewart. 2013. “Text as data: The promise and pitfalls of automatic content analysis methods for political texts”. Political Analysis. 1–31.10.1093/pan/mps028Search in Google Scholar

Grefenstette, G., Q. Yan, S.G. James and D.A. Evans. 2004. “Coupling niche browsers and affect analysis for an opinion mining application”. Proceedings of RIAO 2004Search in Google Scholar

Grljević, O. 2016. Sentiment u sadržajima sa društvenih mreža kao instrument unapređenja poslovanja visokoškolskih institucija [Sentiment in social networks as means of business improvement of higher education institutions]. (PhD dissertation, University of Novi Sad, Faculty of Economics in Subotica.)Search in Google Scholar

Grljević, O. and Z. Bošnjak. 2015. “Development of Serbian Higher Education Corpus”. Paper presented at the 16th IEEE International Symposium on Computational Intelligence and Informatics (CINTI), Budapest, Hugary, 19–21 November.10.1109/CINTI.2015.7382918Search in Google Scholar

Grljević, O., Z. Bošnjak and A. Kovačević. Forthcoming. “Opinion mining in higher education: A corpus-based approach”. Enterprise Information Systems, SI: The Artificial Intelligence-enabled Enterprise Information SystemsSearch in Google Scholar

Hammond, M. 2015. “Predicting the gender of Welsh nouns”. Corpus Linguistics and Linguistic Theory 12(2). 221–261.10.1515/cllt-2015-0001Search in Google Scholar

Henricson, S. and M. Nelson. 2017. “Giving and receiving advice in higher education. Comparing Sweden-Swedish and Finland-Swedish supervision”. Journal of Pragmatics 109. 105–120.10.1016/j.pragma.2016.12.013Search in Google Scholar

Hu, M. and B. Liu. 2004. “Mining and summarizing customer reviews”. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04 Seattle, WA, USA, 22-25 August. 168–177.Search in Google Scholar

Huang, T.H., Y. Ho-Cheng and H.H. Chen. 2012. “Modeling Polyanna phenomena in Chinese sentiment analysis”. Proceedings of COLING 2012: Demonstration papers Mumbai, India. 231–238.Search in Google Scholar

Hunston, S. 2011. Corpus approaches to evaluation: Phraseology and evaluative language New York: Routledge.10.4324/9780203841686Search in Google Scholar

Hutto, C.J. and E. Gilbert. 2014. “VADER: A parsimonious rule-based model for sentiment analysis of social media text”. Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media Ann Arbor, MI.10.1609/icwsm.v8i1.14550Search in Google Scholar

Iosif, E., I. Klasinas, G. Athanasopoulou, E. Palogiannidi, S. Georgiladakis, K. Louka and A. Potamianos. 2018. “Speech understanding for spoken dialogue systems: From corpus harvesting to grammar rule induction”. Computer Speech and Language 47. 272–297.10.1016/j.csl.2017.08.002Search in Google Scholar

Iruskieta, M., A. Ilarraza deDiaz and M. Lersundi. 2013. “Establishing criteria for RST-based discourse segmentation and annotation for texts in Basque”. Corpus Linguistics and Linguistic Theory 11(2). 303–334.10.1515/cllt-2013-0008Search in Google Scholar

Jia, L., C. Yu and W. Meng. 2009. “The effect of negation on sentiment analysis and retrieval effectiveness”. Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009 Hong Kong, China. 1827–1830.Search in Google Scholar

Jing-Schmidt, Z. 2007. “Negativity bias in language: A cognitive-affective model of emotive intensifiers”. Cognitive Linguistics 18. 417–443.10.1515/COG.2007.023Search in Google Scholar

Kayabaşi, D. and M. Özgen. 2018. “A phase-based account on NPI-licensing in Turkish”. Poznań Studies in Contemporary Linguistics 54(1). 83–113.10.1515/psicl-2018-0003Search in Google Scholar

Kennedy, A. and D. Inkpen. 2006. “Sentiment classification of movie reviews using contextual valence shifters”. Computational Intelligence 22. 110–125.10.1111/j.1467-8640.2006.00277.xSearch in Google Scholar

Kessler, J.S. and N. Nicolov. 2009. “Targeting sentiment expressions through supervised ranking of linguistic configurations”. Proceedings of the Third International AAAI Conference on Weblogs and Social Media 90–97.10.1609/icwsm.v3i1.13948Search in Google Scholar

Kessler, J. S., M. Eckert, L. Clark and N. Nicolov. 2010. “The 2010 ICWSM JDPA sentiment corpus for the automotive domain”. Paper presented at the 4th International AAAI Conference on Weblogs and Social Media Data Workshop Challenge (ICWSM-DWC 2010), Washington, DC, USA, 23–26 May.Search in Google Scholar

Kim, E. and R. Klinger. 2018. “Who feels what and why? Annotation of a literature corpus with semantic roles of emotions”. Proceedings of the 27th International Conference on Computational Linguistics Santa Fe, NM, USA. 1345–1359.Search in Google Scholar

Kim, J., E. Shaw, S. Wyner, T. Kim and J. Li. 2010. “Discerning affect in student discussions”. Paper presented at the Annual Meeting of the Cognitive Science Society, Portland, Oregon, USA, 11–14 August.Search in Google Scholar

Kim, J., D. Choi, M. Hwang and P. Kim. 2014. “Analysis on smartphone related Twitter reviews by using opinion mining techniques”. Advanced Approaches to Intelligent Information and Database Systems, Studies in Computational Intelligence 551. 205–212.10.1007/978-3-319-05503-9_20Search in Google Scholar

Landis, R. J. and G. G. Koch. 1977. “The measurement of observer agreement for categorical data”. Biometrics 33(1). 159–174.10.2307/2529310Search in Google Scholar

Lin, Y.L. 2017. “Co-occurrence of speech and gestures: A multimodal corpus linguistic approach to intercultural interaction”. Journal of Pragmatics 117. 155–167.10.1016/j.pragma.2017.06.014Search in Google Scholar

Litman, D.J. and K. Forbes-Riley. 2004. “Annotating student emotional states in spoken tutoring dialogues”. Paper presented at the 5th SIGdial Workshop on Discourse and Dialogue, Cambridge, MA, USA, April 30–May 1. 144–153.Search in Google Scholar

Liu, B. 2012. Sentiment analysis and opinion mining. Morgan & Claypool Publishers.10.2200/S00416ED1V01Y201204HLT016Search in Google Scholar

Liu, Y., X. Huang, A. An and X. Yu. 2007. “ARSA: A sentiment-aware model for predicting sales performance using blogs”. Paper presented at the SIGIR’07, Amsterdam, Netherlands, 23–27 July. 607–614.Search in Google Scholar

Martin, J.R. and P.R.R. White. 2005. The language of evaluation: Appraisal in English London & New York: Palgrave Macmillan.10.1057/9780230511910Search in Google Scholar

Medhat, W., A. Hassan and H. Korashy. 2014. “Sentiment analysis algorithms and applications: A survey”. Ain Shams Engineering Journal 5. 1093–1113.10.1016/j.asej.2014.04.011Search in Google Scholar

Morsy, A.S. and A. Rafea. 2012. “Improving document-level sentiment classification using contextual valence shifters”. In Bouma, G., A. Ittoo, E. Métais and H. Wortmann (eds.), Natural language processing and information systems: Proceedings of the 17th International Conference on Applications of Natural Language to Information Systems Groningen, The Netherlands: Springer. 253–258.10.1007/978-3-642-31178-9_30Search in Google Scholar

Nikolić, M. 2014. Kategorija stepena u srpskom jeziku (složena rečenica) [Category of degrees in Serbian (complex sentence)]. Beograd: Institut za srpski jezik SANU.Search in Google Scholar

Pang, B., L. Lee and S. Vaithyanathan. 2002. “Thumbs up?: Sentiment classification using machine learning techniques”. Paper presented at the ACL-02 Conference on Empirical Methods in Natural Language Processing, University of Pennsylvania, Philadelphia, PA, USA, 6–7 July.10.3115/1118693.1118704Search in Google Scholar

Pang, B. and L. Lee. 2008. “Opinion mining and sentiment analysis”. Foundations and Trends in Information Retrieval 2(1–2). 1–135.10.1561/9781601981516Search in Google Scholar

Piper, P., I. Antonić, V. Ružić, S. Tanasić, Lj. Popović and B. Tošović. 2005. Sintaksa savremenog srpskog jezika – Prosta rečenica [Syntax of contemporary Serbian – The simple sentence]. Beograd: Institut za srpski jezik SANU, Beogradska knjiga, Matica srpska.Search in Google Scholar

Polanyi, L. and A. Zaenen. 2006. “Contextual valence shifters”. In Shanahan, J.G., Y. Qu and J. Wiebe (eds.), Computing attitude and affect in text: Theory and applications Dordrecht: Springer. 1–10.Search in Google Scholar

Poria, S., E. Cambria, L.W. Ku, C. Gui and A. Gelbukh. 2014. “A rule-based approach to aspect extraction from product reviews”. Proceedings of the Second Workshop on Natural Language Processing for Social Media (SocialNLP) Association for Computational Linguistics and Dublin City University. 28–37.10.3115/v1/W14-5905Search in Google Scholar

Poulos, A. and M.J. Mahony. 2008. “Effectiveness of feedback: The students perspective”. Assessment & Evaluation in Higher Education 33(2). 143–154.10.1080/02602930601127869Search in Google Scholar

Pustejovsky, J. and A. Stubbs. 2013. Natural language annotation for machine learning. O’Reilly Media, Inc.Search in Google Scholar

Rumenapp, J.C. 2016. “Analyzing discourse analysis: Teachers’ views of classroom discourse and student identity”. Linguistics and Education 35. 26–36.10.1016/j.linged.2016.04.002Search in Google Scholar

Sagum, R.A., G.J.M. de Vera, J.P.S. Lansang, S.D.R. Narciso and J.K. Respeto. 2015. “Application of language modelling in sentiment analysis for faculty comment evaluation”. Paper presented at the International Multi Conference of Engineers and Computer Scientists 2015 IMECS 2015 I, Hong Kong, 18–20 March.Search in Google Scholar

Saurí, R. 2008. A factuality profiler for eventualities in text. (PhD dissertation thesis, Brandeis University, Waltham.)Search in Google Scholar

Sayeed, A.B., J. Boyd-Graber, B. Rusk and A. Weinberg. 2012. “Grammatical structures for word-level sentiment detection”. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Montreal, Canada, June 3–8, 2012. 667–676.Search in Google Scholar

Schoenmueller, V., O. Netzer, F. Stahl. 2018. “The extreme distribution of online reviews: Prevalence, drivers and implications”. Columbia Business School Research Paper No. 18-10. 68.10.2139/ssrn.3100217Search in Google Scholar

Stanojčić, Ž. and Lj. Popović. 2002. Gramatika srpskog jezika [A grammar of the Serbian language]. Beograd: Zavod za udžbenike i nastavna sredstva.Search in Google Scholar

Su, Q., K. Xiang, H. Wang, B. Sun and S. Yu. 2006. “Using pointwise mutual information to identify implicit features in customer reviews”. In: Matsumoto Y., R.W. Sproat, K.F. Wong, M. Zhang (eds.), Computer processing of Oriental languages. Beyond the Orient: The research challenges ahead. ICCPOL 2006 (Lecture Notes in Computer Science, vol 4285.) Berlin: Springer.10.1007/11940098_3Search in Google Scholar

Taboada, M. 2016. “Sentiment analysis: An overview from linguistics”. Annual Review of Linguistics 2. 325–347.10.1146/annurev-linguistics-011415-040518Search in Google Scholar

Taboada, M., J. Brooke, M. Tofiloski, K. Voll and M. Stede. 2011. “Lexicon-based methods for sentiment analysis”. Computational Linguistics 37. 267–307.10.1162/COLI_a_00049Search in Google Scholar

Tang, X. 2016. “Lexeme-based collexeme analysis with DepCluster”. Corpus Linguistics and Linguistic Theory 13(1). 165–202.10.1515/cllt-2015-0007Search in Google Scholar

Tomasello, M. 2006. “Acquiring linguistic constructions”. In Siegler, R. and D. Kuhn (eds.), Handbook of child psychology: Cognitive development Oxford: Wiley. 255– 299.Search in Google Scholar

Van de Kauter, M., D. Breesch and V. Hoste. 2015. “Fine-grained analysis of explicit and implicit sentiment in financial news articles”. Expert Systems with Applications 42: 4999–5010.10.1016/j.eswa.2015.02.007Search in Google Scholar

Vass, H. 2017. “Lexical verb hedging in legal discourse: The case of law journal articles and Supreme Court majority and dissenting opinions”. English for Specific Purposes 48. 17–31.10.1016/j.esp.2017.07.001Search in Google Scholar

Wachsmuth, H., M. Trenkmann, B. Stein, G. Engels and T. Palakarska. 2014. “A review corpus for argumentation analysis”. Paper presented at the15th International Conference on Computational Linguistics and Intelligent Text Processing, Kathmandu, Nepal, 06–12 April.10.1007/978-3-642-54903-8_10Search in Google Scholar

Wen, M., D. Yang and C.R. Penstein. 2014. “Sentiment analysis in MOOC discussion forums: What does it tell us?” Paper presented at the 7th International Conference on Educational Data Mining, London, UK, 4–7 July. 130–137.Search in Google Scholar

Wiebe, J., T. Wilson and C. Cardie. 2005. “Annotating expressions of opinions and emotions in language”. Language Resources and Evaluation 39(2–3). 165–210.10.1007/s10579-005-7880-9Search in Google Scholar

Wyner, S., E. Shaw, T. Kim, J. Li and J. Kim. 2009. “Sentiment analysis of a student Q&A board for computer science”. Paper presented at the IJCAI workshop on Computational Models of Natural Argument, Pasadena, CA, USA, 13 July.Search in Google Scholar

Yang, C.S., C.H. Chen and P.C. Chang. 2015. “Harnessing consumer reviews for marketing intelligence: a domain-adapted sentiment classification approach”. Information Systems and e-Business Management 13(3). 403–419.10.1007/s10257-014-0266-zSearch in Google Scholar

Yang, K. 2008. “WIDIT in TREC 2008 blog track: Leveraging multiple sources of opinion evidence”. Proceedings of The Seventeenth Text REtrieval Conference, TREC 2008 Gaithersburg, Maryland, USA, November 18–21, 2008.Search in Google Scholar

Yoo, J. and J. Kim. 2014. “Capturing difficulty expressions in student online Q&A discussions”. Paper presented at the Twenty-Eighth {AAAI} Conference on Artificial Intelligence, Québec City, Québec, Canada, 27–31 July. 208–214.Search in Google Scholar

Published Online: 2020-09-18
Published in Print: 2020-06-25

© 2020 Faculty of English, Adam Mickiewicz University, Poznań, Poland

Downloaded on 28.3.2024 from https://www.degruyter.com/document/doi/10.1515/psicl-2020-0006/html
Scroll to top button