Social networks enable people to interact with one another by sharing information, sending messages, making friends, and having discussions, which generates massive amounts of data every day, popularly called as the user-generated content. This data is present in various forms such as images, text, videos, links, and others and reflects user behaviours including their mental states. It is challenging yet promising to automatically detect mental health problems from such data which is short, sparse and sometimes poorly phrased. However, there are efforts to automatically learn patterns using computational models on such user-generated content. While many previous works have largely studied the problem on a small-scale by assuming uni-modality of data which may not give us faithful results, we propose a novel scalable hybrid model that combines Bidirectional Gated Recurrent Units (BiGRUs) and Convolutional Neural Networks to detect depressed users on social media such as Twitter-based on multi-modal features. Specifically, we encode words in user posts using pre-trained word embeddings and BiGRUs to capture latent behavioural patterns, long-term dependencies, and correlation across the modalities, including semantic sequence features from the user timelines (posts). The CNN model then helps learn useful features. Our experiments show that our model outperforms several popular and strong baseline methods, demonstrating the effectiveness of combining deep learning with multi-modal features. We also show that our model helps improve predictive performance when detecting depression in users who are posting messages publicly on social media.

中文翻译：

---

在社交媒体上使用混合深度学习模型进行多模态抑郁检测

社交网络使人们能够通过共享信息、发送消息、结交朋友和进行讨论来相互交流，每天都会产生大量数据，通常称为用户生成内容。这些数据以各种形式存在，例如图像、文本、视频、链接等，并反映用户行为，包括他们的心理状态。从这些简短、稀疏且有时措辞不当的数据中自动检测心理健康问题具有挑战性但很有希望。然而，人们正在努力在此类用户生成的内容上使用计算模型来自动学习模式。虽然以前的许多工作通过假设数据的单模态在小规模上大量研究了这个问题，但可能无法给出可靠的结果，我们提出了一种新颖的可扩展混合模型，该模型结合了双向门控循环单元 (BiGRU) 和卷积神经网络，以基于多模态特征检测社交媒体（例如 Twitter）上的抑郁用户。具体来说，我们使用预先训练的词嵌入和 BiGRU 对用户帖子中的词进行编码，以捕获潜在的行为模式、长期依赖关系和跨模态的相关性，包括来自用户时间线（帖子）的语义序列特征。然后，CNN 模型有助于学习有用的特征。我们的实验表明，我们的模型优于几种流行且强大的基线方法，证明了将深度学习与多模态特征相结合的有效性。