当前位置: X-MOL 学术Pattern Recogn. Lett. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Multilevel fusion of multimodal deep features for porn streamer recognition in live video
Pattern Recognition Letters ( IF 5.1 ) Pub Date : 2020-09-25 , DOI: 10.1016/j.patrec.2020.09.027
Liyuan Wang , Jing Zhang , Meng Wang , Jimiao Tian , Li Zhuo

Live video hosted by streamers is being sought after by an increasing number of Internet users. Some streamers mix pornographic content with live video for profit and popularity, but this greatly harms the network environment. To effectively identify porn streamers, a multilevel fusion method of multimodal deep features for porn streamer recognition in live video is proposed in this paper. (1) Visual and audio features including spatial, audio, motion, and temporal context in live video are extracted by a multimodal deep network. (2) Audio-visual attention features are obtained by fusing visual and audio features at the feature level based on a multimodal attention mechanism. (3) Text features are extracted by using the bullet screen text network based on the BERT (bidirectional encoder representations from transformers) model after collecting text information from the viewers’ bullet screen comments. (4) The prediction results of the audio-visual deep network and the bullet screen text network are fused at the decision level to improve the porn streamer recognition accuracy. We build a real-world dataset of porn streamers and conduct experiments and demonstrate that our method can improve the porn streamer recognition accuracy.



中文翻译:

多模式深度特征的多级融合,用于实时视频中的色情流光识别

由流媒体主持的实时视频正受到越来越多的Internet用户的追捧。一些流媒体将色情内容与实时视频混合以获取利润和受欢迎程度,但这极大地损害了网络环境。为了有效地识别色情流光,提出了一种多模式深度特征的多级融合方法,用于实时视频中的色情流光识别。(1)通过多模式深度网络提取实时视频中的视觉和音频功能,包括空间,音频,运动和时间上下文。(2)视听注意力特征是通过基于多模式注意力机制在特征级别融合视觉和音频特征而获得的。(3)在从观众的项目符号屏幕注释中收集文本信息之后,使用基于BERT(来自变压器的双向编码器表示)模型的项目符号屏幕文本网络来提取文本特征。(4)在决策层融合视听深度网络和子弹屏文本网络的预测结果,以提高色情流光识别的准确性。我们建立了一个现实世界的色情流光数据集并进行了实验,证明了我们的方法可以提高色情流光的识别准确性。

更新日期:2020-10-13
down
wechat
bug