当前位置: X-MOL 学术IEEE Trans. Vis. Comput. Graph. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
LiveObj: Object Semantics-based Viewport Prediction for Live Mobile Virtual Reality Streaming
IEEE Transactions on Visualization and Computer Graphics ( IF 5.2 ) Pub Date : 2021-04-01 , DOI: 10.1109/tvcg.2021.3067686
Xianglong Feng , Zeyang Bao , Sheng Wei

Virtual reality (VR) video streaming (a.k.a., 360-degree video streaming) has been gaining popularity recently as a new form of multimedia providing the users with immersive viewing experience. However, the high volume of data for the 360-degree video frames creates significant bandwidth challenges. Research efforts have been made to reduce the bandwidth consumption by predicting and selectively streaming the user's viewports. However, the existing approaches require historical user or video data and cannot be applied to live streaming, the most attractive VR streaming scenario. We develop a live viewport prediction mechanism, namely LiveObj , by detecting the objects in the video based on their semantics. The detected objects are then tracked to infer the user's viewport in real time by employing a reinforcement learning algorithm. Our evaluations based on 48 users watching 10 VR videos demonstrate high prediction accuracy and significant bandwidth savings obtained by LiveObj . Also, LiveObj achieves real-time performance with low processing delays, meeting the requirement of live VR streaming.

中文翻译:

LiveObj:用于实时移动虚拟现实流媒体的基于对象语义的视口预测

虚拟现实 (VR) 视频流(又名 360 度视频流)作为一种新的多媒体形式,为用户提供身临其境的观看体验,最近越来越受欢迎。然而,360 度视频帧的大量数据带来了巨大的带宽挑战。已经通过预测和选择性地流式传输用户的视口来降低带宽消耗的研究工作。然而,现有的方法需要历史用户或视频数据,不能应用于最有吸引力的 VR 流媒体直播场景。我们开发了一个实时视口预测机制,即实时对象 ,通过基于语义检测视频中的对象。然后通过使用强化学习算法跟踪检测到的对象以实时推断用户的视口。我们基于 48 个用户观看 10 个 VR 视频的评估表明,通过实时对象 . 还,实时对象 实现实时性低处理延迟,满足VR直播的需求。
更新日期:2021-04-16
down
wechat
bug