Skip to main content
Log in

Recent trends in deep learning based personality detection

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Recently, the automatic prediction of personality traits has received a lot of attention. Specifically, personality trait prediction from multimodal data has emerged as a hot topic within the field of affective computing. In this paper, we review significant machine learning models which have been employed for personality detection, with an emphasis on deep learning-based methods. This review paper provides an overview of the most popular approaches to automated personality detection, various computational datasets, its industrial applications, and state-of-the-art machine learning models for personality detection with specific focus on multimodal approaches. Personality detection is a very broad and diverse topic: this survey only focuses on computational approaches and leaves out psychological studies on personality detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The Great Hack, a recent documentary about the Cambridge Analytica data scandal

References

  • Agarwal B (2014) Personality detection from text: a review. Int J Comput Syst 1:1–4

    Google Scholar 

  • Al Moubayed N, Vazquez-Alvarez Y, McKay A, Vinciarelli A (2014) Face-based automatic personality perception. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 1153–1156

  • An G, Levitan R (2018) Lexical and acoustic deep learning model for personality recognition. In: Proceedings of interspeech, pp 1761–1765. https://doi.org/10.21437/Interspeech.2018-2263

  • Baltrusaitis T, Robinson P, Morency LP (2016) Openface: an open source facial behavior analysis toolkit. In: IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1–10

  • Biel JI, Teijeiro-Mosquera L, Gatica-Perez D (2012) Facetube: predicting personality from facial expressions of emotion in online conversational video. In: Proceedings of the 14th ACM international conference on multimodal interaction. ACM, pp 53–56

  • Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Lear Res 3:993–1022

    MATH  Google Scholar 

  • Briggs-Myers I (1993) Introduction to type: a guide to understanding your results on the Myers-Briggs type indicator (revised by lk kirby & kd myers), Palo alto

  • Bruce V, Young A (1986) Understanding face recognition. Br J Psychol 77(3):305–327

    Google Scholar 

  • Busso C, Bulut M, Lee CC, Kazemzadeh A, Mower E, Kim S, Chang JN, Lee S, Narayanan SS (2008) IEMOCAP: interactive emotional dyadic motion capture database. Lang Resour Eval 42(4):335

    Google Scholar 

  • Cambria E, Poria S, Hazarika D, Kwok K (2018) SenticNet 5: discovering conceptual primitives for sentiment analysis by means of context embeddings. In: AAAI, pp 1795–1802

  • Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Int Syst 32(6):74–80

    Google Scholar 

  • Caprara GV, Schwartz S, Capanna C, Vecchione M, Barbaranelli C (2006) Personality and politics: values, traits, and political choice. Polit Psychol 27(1):1–28

    Google Scholar 

  • Carletta J (2007) Unleashing the killer corpus: experiences in creating the multi-everything AMI meeting corpus. Lang Resour Eval 41(2):181–190

    Google Scholar 

  • Carletta J, Ashby S, Bourban S, Flynn M, Guillemot M, Hain T, Kadlec J, Karaiskos V, Kraaij W, Kronenthal M et al (2005) The ami meeting corpus: a pre-announcement. In: International workshop on machine learning for multimodal interaction. Springer, pp 28–39

  • Cattell HE, Mead AD (2008) The sixteen personality factor questionnaire (16pf). The SAGE handbook of personality theory and assessment 2:135–178

    Google Scholar 

  • Celli F (2012) Unsupervised personality recognition for social network sites. In: Proceedings of sixth international conference on digital society

  • Celli F, Poesio M (2014) Pr2: a language independent unsupervised tool for personality recognition from text. arXiv preprint arXiv:1402.2796

  • Celli F, Di Lascio FML, Magnani M, Pacelli B, Rossi L (2010) Social network data and practices: the case of friendfeed. In: International conference on social computing, behavioral modeling, and prediction. Springer, pp 346–353

  • Chen B, Escalera S, Guyon I, Ponce-López V, Shah N, Simón MO (2016) Overcoming calibration problems in pattern labeling with pairwise ratings: application to personality traits. In: European conference on computer vision. Springer, pp 419–432

  • Chittaranjan G, Blom J, Gatica-Perez D (2011) Who’s who with big-five: analyzing and classifying personality traits with smartphones. In: 15th Annual international symposium on wearable computers (ISWC). IEEE, pp 29–36

  • Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

  • Cristani M, Vinciarelli A, Segalin C, Perina A (2013) Unveiling the multimedia unconscious: implicit cognitive processes and multimedia content analysis. In: Proceedings of the 21st ACM international conference on multimedia. ACM, pp 213–222

  • CVPR (2017) Chalearn dataset looking at people challenge. http://chalearnlap.cvc.uab.es/dataset/24/description/. Accessed 09 Oct 2019

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, CVPR 2005, vol 1. IEEE, pp 886–893

  • Davis SB, Mermelstein P (1990) Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. In: Readings in speech recognition. Elsevier, pp 65–74

  • Digman JM (1990) Personality structure: emergence of the five-factor model. Ann Rev Psychol 41(1):417–440

    Google Scholar 

  • Eddine Bekhouche S, Dornaika F, Ouafi A, Taleb-Ahmed A (2017) Personality traits and job candidate screening via analyzing facial videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 10–13

  • Escalante HJ, Guyon I, Escalera S, Jacques J, Madadi M, Baró X, Ayache S, Viegas E, Güçlütürk Y, Güçlü U et al (2017) Design of an explainable machine learning challenge for video interviews. In: International joint conference on neural networks (IJCNN). IEEE, pp 3688–3695

  • Escalante HJ, Kaya H, Salah AA, Escalera S, Gucluturk Y, Guclu U, Baró X, Guyon I, Junior JJ, Madadi M et al (2018) Explaining first impressions: modeling, recognizing, and explaining apparent personality from videos. arXiv preprint arXiv:1802.00745

  • Escalera S, Baró X, Guyon I, Escalante HJ (2018) Guest editorial: apparent personality analysis. IEEE Trans Affect Comput 9(3):299–302

    Google Scholar 

  • Eyben F, Wollmer M, Schuller B (2010) Opensmile: the munich versatile and fast open-source audio feature extractor. In: Proceedings of the 18th ACM international conference on multimedia. ACM, pp 1459–1462

  • Eysenck HJ (2012) A model for personality. Springer, Berlin

    Google Scholar 

  • Freund Y, Schapire RE et al (1996) Experiments with a new boosting algorithm. ICML, Bari, Italy 96:148–156

    Google Scholar 

  • Furnham A (1990) Can people accurately estimate their own personality test scores? Eur J Personal 4(4):319–327

    Google Scholar 

  • Furnham A (1996) The big five versus the big four: the relationship between the Myers-Briggs type indicator (MBTI) and NEO-PI five factor model of personality. Personal Individ Differ 21(2):303–307

    Google Scholar 

  • Golbeck J (2016) Predicting personality from social media text. AIS Trans Replication Res 2(1):2

    Google Scholar 

  • Gonzalez-Gallardo CE, Montes A, Sierra G, Nunez-Juarez JA, Salinas-Lopez AJ, Ek J (2015) Tweets classification using corpus dependent tags, character and pos n-grams. In: CLEF (working notes)

  • Gorbova J, Lusi I, Litvin A, Anbarjafari G (2017) Automated screening of job candidate based on multimodal video processing. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 29–35

  • Gucluturk Y, Guclu U, Perez M, Balderas HJE, Baro X, Guyon I, Andujar C, Junior J, Madadi M, Escalera S et al (2017) Visualizing apparent personality analysis with deep residual networks. In: International conference on computer vision-ICCV 2017

  • Gurpinar F, Kaya H, Salah AA (2016) Combining deep facial and ambient features for first impression estimation. In: European conference on computer vision. Springer, pp 372–385

  • Gurpinar F, Kaya H, Salah AA (2016) Multimodal fusion of audio, scene, and face features for first impression estimation. In: 23rd International conference on pattern recognition (ICPR). IEEE, pp 43–48

  • Haghighat M, Abdel-Mottaleb M, Alhalabi W (2016) Discriminant correlation analysis: real-time feature level fusion for multimodal biometric recognition. IEEE Trans Inf Forensics Secur 11(9):1984–1996

    Google Scholar 

  • Hernandez RK, Scott I (2017) Predicting Myers-Briggs type indicator with text

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Hirsch HG, Pearce D (2000) The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In: ASR2000-Automatic speech recognition: challenges for the new Millenium ISCA tutorial and research workshop (ITRW)

  • Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67

    MATH  Google Scholar 

  • Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529

    Google Scholar 

  • Ilmini K, Fernando TL (2016) Persons’ personality traits recognition using machine learning algorithms and image processing techniques

  • Ireland ME, Pennebaker JW (2010) Language style matching in writing: Synchrony in essays, correspondence, and poetry. J Personal Soc Psychol 99(3):549

    Google Scholar 

  • Jaynes ET (1982) On the rationale of maximum-entropy methods. Proc IEEE 70(9):939–952

    Google Scholar 

  • Ji S, Xu W, Yang M, Yu K (2013) 3d convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35(1):221–231

    Google Scholar 

  • Judge TA, Higgins CA, Thoresen CJ, Barrick MR (1999) The big five personality traits, general mental ability, and career success across the life span. Pers Psychol 52(3):621–652

    Google Scholar 

  • Junior J, Jacques C, Güçlütürk Y, Pérez M, Güçlü U, Andujar C, Baró X, Escalante HJ, Guyon I, van Gerven MA et al (2018) First impressions: a survey on computer vision-based apparent personality trait analysis. arXiv preprint arXiv:1804.08046

  • Kalghatgi MP, Ramannavar M, Sidnal NS (2015) A neural network approach to personality prediction based on the big-five model. Int J Innov Res Adv Eng (IJIRAE) 2(8):56–63

    Google Scholar 

  • Kamenskaya E, Kukharev G (2008) Recognition of psychological characteristics from face. Metody Inform Stosow 1(1):59–73

    Google Scholar 

  • Kampman O, Barezi EJ, Bertero D, Fung P (2018) Investigating audio, video, and text fusion methods for end-to-end automatic personality prediction. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: short papers), vol 2, pp 606–611

  • Kindiroglu AA, Akarun L (2017) Aran O (2017) Multi-domain and multi-task prediction of extraversion and leadership from meeting videos. EURASIP J Image Video Proc 1:77

    Google Scholar 

  • Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Google Scholar 

  • Levitan SI, An G, Wang M, Mendels G, Hirschberg J, Levine M, Rosenberg A (2015) Cross-cultural production and detection of deception from speech. In: Proceedings of the 2015 ACM on workshop on multimodal deception detection. ACM, pp 1–8

  • Levitan SI, Levitan Y, An G, Levine M, Levitan R, Rosenberg A, Hirschberg J (2016) Identifying individual differences in gender, ethnicity, and personality from dialogue for deception detection. In: Proceedings of the second workshop on computational approaches to deception detection, pp 40–44

  • Liem CC, Langer M, Demetriou A, Hiemstra AM, Wicaksana AS, Born MP, König CJ (2018) Psychology meets machine learning: Interdisciplinary perspectives on algorithmic job candidate screening. In: Explainable and interpretable models in computer vision and machine learning. Springer, pp 197–253

  • Ling W, Luis T, Marujo L, Astudillo RF, Amir S, Dyer C, Black AW, Trancoso I (2015) Finding function in form: Compositional character models for open vocabulary word representation. arXiv preprint arXiv:1508.02096

  • Liu L, Preotiuc-Pietro D, Samani ZR, Moghaddam ME, Ungar LH (2016) Analyzing personality through social media profile picture choice. In: ICWSM, pp 211–220

  • Madzlan NA, Han J, Bonin F, Campbell N (2014) Automatic recognition of attitudes in video blogs—prosodic and visual feature analysis. In: Fifteenth annual conference of the international speech communication association

  • Majumder N, Poria S, Gelbukh A, Cambria E (2017) Deep learning-based document modeling for personality detection from text. IEEE Intell Syst 32(2):74–79

    Google Scholar 

  • Majumder N, Poria S, Peng H, Chhaya N, Cambria E, Gelbukh A (2019) Sentiment and sarcasm classification with multitask learning. IEEE Intel Syst 34(3):38–43

    Google Scholar 

  • Matthews G, Deary IJ, Whiteman MC (2003) Personality traits. Cambridge University Press, Cambridge

    Google Scholar 

  • Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

  • Miles J, Hempel S (2004) The Eysenck personality scales: the Eysenck personality questionnaire-revised (EPQ-R) and the Eysenck personality profiler (EPP). Comprehensive handbook of psychological assessment vol 2, pp 99–107

  • Mukherjee S, Kumar U (2016) Ethical issues in personality assessment. In: The Wiley handbook of personality assessment, pp 415–426

  • Oosterhof NN, Todorov A (2008) The functional basis of face evaluation. Proc Natl Acad Sci 105(32):11087–11092

    Google Scholar 

  • Palaz D, Magimai-Doss M, Collobert R (2015) Analysis of cnn-based speech recognition system using raw speech as input. Technical report, Idiap

  • Papurt MJ (1930) A study of the woodworth psychoneurotic inventory with suggested revision. J Abnorm Soc Psychol 25(3):335

    Google Scholar 

  • Park G, Schwartz HA, Eichstaedt JC, Kern ML, Kosinski M, Stillwell DJ, Ungar LH, Seligman ME (2015) Automatic personality assessment through social media language. J Personal Soc Psychol 108(6):934

    Google Scholar 

  • Pennebaker JW, King LA (1999) Linguistic styles: language use as an individual difference. J Personal Soc Psychol 77(6):1296

    Google Scholar 

  • Pennebaker JW, Francis ME, Booth RJ (2001) Linguistic inquiry and word count: LIWC 2001, vol 71. Lawrence Erlbaum Associates, Mahway

    Google Scholar 

  • Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  • Perez-Rosas V, Mihalcea R, Morency LP (2013) Utterance-level multimodal sentiment analysis. In: Proceedings of the 51st annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 973–982

  • Pervin LA, John OP (1999) Handbook of personality: theory and research. Elsevier, Amsterdam

    Google Scholar 

  • Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16(5):295–306

    Google Scholar 

  • Polzehl T, Moller S, Metze F (2010) Automatically assessing personality from speech. In: IEEE fourth international conference on semantic computing (ICSC). IEEE, pp 134–140

  • Ponce-López V, Chen B, Oliu M, Corneanu C, Clapés A, Guyon I, Baró X, Escalante HJ, Escalera S (2016) Chalearn lap 2016: first round challenge on first impressions-dataset and results. In: European conference on computer vision. Springer, pp 400–418

  • Poria S, Gelbukh A, Agarwal B, Cambria E, Howard N (2013) Common sense knowledge based personality recognition from text. In: Mexican international conference on artificial intelligence. Springer, pp 484–496

  • Poria S, Chaturvedi I, Cambria E, Bisio F (2016) Sentic LDA: Improving on LDA with semantic similarity for aspect-based sentiment analysis. In: International joint conference on neural networks (IJCNN), pp 4465–4473

  • Poria S, Cambria E, Hazarika D, Majumder N, Zadeh A, Morency LP (2017) Context-dependent sentiment analysis in user-generated videos. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 873–883

  • Rai N (2016) Bi-modal regression for apparent personality trait recognition. In: 23rd International conference on pattern recognition (ICPR). IEEE, pp 55–60

  • Roberto B, Poggio T (1993) Face recognition: features versus templates. IEEE Tran Pattern Anal Mach Intell 15(10):1042–1052

    Google Scholar 

  • Rojas M, Masip D, Todorov A, Vitria J (2011) Automatic prediction of facial trait judgments: appearance vs. structural models. PLoS ONE 6(8):e23323

  • Saez Y, Navarro C, Mochon A, Isasi P (2014) A system for personality and happiness detection. IJIMAI 2(5):7–15

    Google Scholar 

  • Sanchez-Cortes D, Aran O, Gatica-Perez D (2011) An audio visual corpus for emergent leader analysis. In: ICMI-MLMI, Multimodal corpora for machine learning, pp 14–18

  • Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168

    MATH  Google Scholar 

  • Shaver PR, Brennan KA (1992) Attachment styles and the “big five” personality traits: their connections with each other and with romantic relationship outcomes. Personal Soc Psychol Bull 18(5):536–545

    Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  • Su MH, Wu CH, Zheng YT (2016) Exploiting turn-taking temporal evolution for personality trait perception in dyadic conversations. IEEE/ACM Trans Audio Speech Lang Process 24(4):733–744

    Google Scholar 

  • Subramaniam A, Patel V, Mishra A, Balasubramanian P, Mittal A (2016) Bi-modal first impressions recognition using temporally ordered deep audio and stochastic visual features. In: European conference on computer vision. Springer, pp 337–348

  • Sun X, Liu B, Cao J, Luo J, Shen X (2018) Who am I? Personality detection based on deep learning for texts. In: IEEE international conference on communications (ICC). IEEE, pp 1–6

  • Tausczik YR, Pennebaker JW (2010) The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 29(1):24–54

    Google Scholar 

  • Tseng SC (2004) Processing mandarin spoken corpora. Traitement Automatique des Langes, pp 89–108

  • Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86

    Google Scholar 

  • Valente F, Kim S, Motlicek P (2012) Annotation and recognition of personality traits in spoken conversations from the ami meetings corpus. In: Thirteenth annual conference of the international speech communication association

  • Ventura C, Masip D, Lapedriza A (2017) Interpreting cnn models for apparent personality trait regression. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 1705–1713

  • Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Google Scholar 

  • Vo NN, Liu S, He X, Xu G (2018) Multimodal mixture density boosting network for personality mining. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 644–655

  • Walker M, Vetter T (2016) Changing the personality of a face: Perceived big two and big five personality factors modeled in real photographs. J Personal Soc Psychol 110(4):609

    Google Scholar 

  • Whissell C, Fournier M, Pelland R, Weir D, Makarec K (1986) A dictionary of affect in language: IV Reliability, validity, and applications. Percept Motor Skills 62(3):875–888

    Google Scholar 

  • Willis J, Todorov A (2006) First impressions: making up your mind after a 100-ms exposure to a face. Psychol Sci 17(7):592–598

    Google Scholar 

  • Yang K, Mall S, Glaser N (2017) Prediction of personality first impressions with deep bimodal LSTM

  • Yang HC, Huang ZR (2019) Mining personality traits from social messages for game recommender systems. Knowl Based Syst 165:157–168

    Google Scholar 

  • Vinciarelli A, Mohammadi G (2014) A survey of personality computing. IEEE Trans Affect Comput 5(3):273–291

    Google Scholar 

  • Yin H, Wang Y, Li Q, Xu W, Yu Y, Zhang T (2018) A network-enhanced prediction method for automobile purchase classification using deep learning

  • Yu J, Markov K (2017) Deep learning based personality recognition from facebook status updates. In: IEEE 8th international conference on awareness science and technology (iCAST). IEEE, pp 383–387

  • Zadeh A, Zellers R, Pincus E, Morency LP (2016) Mosi: multimodal corpus of sentiment intensity and subjectivity analysis in online opinion videos. arXiv preprint arXiv:1606.06259

  • Zhang W, Shan S, Gao W, Chen X, Zhang H (2005) Local gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Tenth IEEE international conference on computer vision, ICCV 2005, vol 1. IEEE, pp 786–791

  • Zhang CL, Zhang H, Wei XS, Wu J (2016) Deep bimodal regression for apparent personality analysis. In: European conference on computer vision. Springer, pp 311–324

  • Zhao S, Gholaminejad A, Ding G, Gao Y, Han J, Keutzer K (2019) Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Tran Multimed Comput Commun Appl (TOMM) 15(1s):14

    Google Scholar 

  • Zuo X, Feng B, Yao Y, Zhang T, Zhang Q, Wang M, Zuo W (2013) A weighted ML-kNN model for predicting users’ personality traits. In: Proceedings of international conference on information science and computer application (ISCA), pp 345–350

Download references

Acknowledgements

We would like to thank Prof. Bharat M Deshpande for his valuable guidance. A. Gelbukh recognizes the support of the Instituto Politecnico Nacional via the Secretaria de Investigacion y Posgrado projects SIP 20196437 and SIP 20196021.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Erik Cambria.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mehta, Y., Majumder, N., Gelbukh, A. et al. Recent trends in deep learning based personality detection. Artif Intell Rev 53, 2313–2339 (2020). https://doi.org/10.1007/s10462-019-09770-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-019-09770-z

Keywords

Navigation