Skip to main content

Advertisement

SpringerLink
Log in

Multimodal depression detection on instagram considering time interval of posts

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Depression is a common and serious mental disorder that causes a person to have sad or hopeless feelings in his/her daily life. With the rapid development of social media, people tend to express their thoughts or emotions on the social platform. Different social platforms have various formats of data presentation, which makes huge and diverse data available for analysis by researchers. In our study, we aim to detect users with depressive tendency on Instagram. We create a depression dictionary for automatically collecting data of depressive and non-depressive users. In terms of the prediction model, we construct a multimodal system, which utilizes image, text and behavior features to predict the aggregated depression score of each post on Instagram. Considering the time interval between posts, we propose a two-stage detection mechanism for detecting depressive users. Experimental results demonstrate that our proposed methods can achieve up to 0.835 F1-score for detecting depressive users. It can therefore serve as an early depression detector for a timely treatment before it becomes severe.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  • Andalibi, N., Ozturk, P., & Forte, A. (2015). Depression-related imagery on instagram. In Inproceedings of the 18th ACM Conference Companion on Computer Supported Cooperative work and social computing (pp. 231–234).

  • Borth, D., Ji, R., Chen, T., Breuel, T., & Chang, S.-F. (2013). Large-scale visual sentiment ontology and detectors using adjective noun pairs. In proceedings of the 21st ACM international conference on Multimedia (pp. 223–232).

  • Chen, X., Sykora, M.D., Jackson, T.W, & Elayan, S. (2018). What about mood swings: identifying depression on twitter with temporal measures of emotions. In Companion of the The Web Conference 2018 on The Web Conference (pp. 1653–1660).

  • Cong, Q., Feng, Z., Li, F., Xiang, Y., Rao, G., & Tao, C. (2018). XA-BiLSTM: a Deep Learning Approach for Depression Detection in Imbalanced Data. In 2018 IEEE International Conference on Bioinformatics and Biomedicine BIBM (pp. 1624–1627).

  • De Choudhury, M., Gamon, M., Counts, S., & Horvitz, E. (2013). Predicting depression via social media, in Seventh international AAAI conference on weblogs and social media.

  • Deng, J., Dong, W., Socher, R., Li, L. -J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248–255).

  • Dos Santos, C., & Gatti, M. (2014). Deep convolutional neural networks for sentiment analysis of short texts. In proceedings of COLING 2014, the 25th International Conference on Computational Linguistics:, Technical Papers (pp. 69–78).

  • Gallagher, S. (2012). Time, emotion, and depression. Emotion Review, 4(2), 127–132.

    Article  Google Scholar 

  • Hann, D., Winter, K., & Jacobsen, P. (1999). Measurement of depressive symptoms in cancer patients: evaluation of the Center for Epidemiological Studies Depression Scale CES-d. Journal of psychosomatic research, 46(5), 437–443.

    Article  Google Scholar 

  • Hassan, A., & Mahmood, A. (2017). Deep learning approach for sentiment analysis of short texts. In 2017 3rd international conference on control, automation and robotics ICCAR, : IEEE, pp. 705–710.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

  • Hu, A., & Flaxman, S. (2018). Multimodal sentiment analysis to explore the structure of emotions. In proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 350–358).

  • Huang, Y.C, Chieh, F.C., & Chen, A.L. (2019). Predicting Depression Tendency based on Image, Text and Behavior Data from Instagram. In International Conference on Data Science Technology and Applications.

  • Kang, K, Yoon, C., & Kim, E.Y. (2016). Identifying depressive users in Twitter using multimodal analysis. In 2016 International Conference on Big Data and Smart Computing BigComp, pp 231–238.

  • Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv:1408.5882.

  • Krizhevsky, A, Sutskever, I., & Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp 1097–1105.

  • Kroenke, K., Spitzer, R.L., & Williams, J.B. (2001). The PHQ-9: validity of a brief depression severity measure. Journal of general internal medicine, 16(9), 606–613.

    Article  Google Scholar 

  • Losada, D.E., Crestani, F., & Parapar, J. (2017). CLEF 2017 eRisk Overview: Early Risk Prediction on the internet: Experimental Foundations, in CLEF Working Notes.

  • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.

  • Nguyen, H. (2017). Nguyen, M.-L.. In International Conference of the Pacific Association for Computational Linguistics (pp. 15–27).

  • Orabi, A.H., Buddhitha, P., Orabi, M.H., & Inkpen, D. (2018). Deep learning for depression detection of twitter users. In proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology:, From Keyboard to Clinic (pp. 88–97).

  • Paszke, A., & et al. (2017). Automatic differentiation in pytorch.

  • Pedregosa, F., & et al. (2011). Scikit-learn: Machine learning in Python. Journal of machine learning research, 12, 2825–2830.

    MathSciNet  MATH  Google Scholar 

  • Reddy, M. (2012). Depression-the global crisis. Indian journal of psychological medicine, 34(3), 201.

    Article  Google Scholar 

  • Reece, A.G., & Danforth, C.M. (2017). Instagram photos reveal predictive markers of depression. EPJ Data Science, 6(1), 15.

    Article  Google Scholar 

  • Sadeque, F., Xu, D., & Bethard, S. (2018). Measuring the latency of depression detection in social media. In proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (pp. 495–503).

  • Shen, G., & et al. (2017). Depression Detection via Harvesting Social media: A Multimodal Dictionary Learning Solution, in IJCAI, pp 3838–3844.

  • Shen, T., & et al. (2018). Cross-Domain Depression Detection via Harvesting Social Media, in IJCAI, pp 1611–1617.

  • Silva, J., Coheur, L., Mendes, A.C., & Wichert, A. (2011). From symbolic to sub-symbolic information in question classification. Artificial Intelligence Review, 35 (2), 137–154.

    Article  Google Scholar 

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.

  • Tian, X., Batterham, P., Song, S., Yao, X., & Yu, G. (2018). Characterizing depression issues on sina weibo. International journal of environmental research and public health, 15(4), 764.

    Article  Google Scholar 

  • Wang, S., & Manning, C.D. (2012). Baselines and bigrams: Simple, good sentiment and topic classification. In proceedings of the 50th annual meeting of the association for computational linguistics: Short papers-volume 2, : Association for Computational Linguistics, pp. 90–94.

  • Wang, Y., & et al. (2017). Understanding and discovering deliberate self-harm content in social media. In proceedings of the 26th International Conference on World Wide Web (pp. 93–102).

  • Wu, M.Y., Shen, C. -Y., Wang, E.T., & Chen, A.L. (2018). A deep architecture for depression detection using posting, behavior, and living environment data, Journal of Intelligent Information Systems.

  • Yazdavar, A.H., & et al. (2017). Semi-supervised approach to monitoring clinical depressive symptoms in social media. In Inproceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (pp. 1191–1198).

  • You, Q., Luo, J., Jin, H., & Yang, J. (2015). Robust image sentiment analysis using progressively trained and domain transferred deep networks. In Twenty-Ninth AAAI Conference on Artificial Intelligence.

  • Zhang, X., & LeCun, Y. (2015). Text understanding from scratch. arXiv:1502.01710.

Download references

Acknowledgments

This study was partially supported by the Research Platform of China Medical University Hospital and Asia University (Grant Number: ASIA-106-CMUH-12) and the Ministry Of Science and Technology, ROC (Grant Number: 106-2221-E-468 -014 -MY2).

Author information

Authors and Affiliations

  1. Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu, Taiwan

    Chun Yueh Chiu

  2. Graduate Institute of Biomedical Sciences, China Medical University School of Medicine, Taichung, Taiwan

    Hsien Yuan Lane

  3. Department of Psychiatry, China Medical University Hospital, Taichung, Taiwan

    Hsien Yuan Lane

  4. Department of Computer Science and Information Engineering, National Taiwan Normal University, Taipei, Taiwan

    Jia Ling Koh

  5. Department of Computer Science and Information Engineering, Asia University, Taichung, Taiwan

    Arbee L. P. Chen

Authors
  1. Chun Yueh Chiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hsien Yuan Lane
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia Ling Koh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arbee L. P. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arbee L. P. Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chiu, C.Y., Lane, H.Y., Koh, J.L. et al. Multimodal depression detection on instagram considering time interval of posts. J Intell Inf Syst 56, 25–47 (2021). https://doi.org/10.1007/s10844-020-00599-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-020-00599-5

Keywords

Search

Navigation