Most human synthesis schemes use high-performance servers, so the user interaction experience of mobile devices is not satisfied. Viewing human synthesis results on smartphones directly increases user interaction and enhances user experience. This paper proposes a smart frame selection network (SFSN) on mobile devices to reduce the traffic between smartphones and cloud. We leverage the attention and relationship model to focus on the relationship between a single frame and the entire video, which can better select important frames, thus reducing the traffic and computing effectively. In addition, we build a multi-task human synthesis system based on SFSN to process the generation tasks such as background changing, pose transfer and virtual try-on in a unified framework. Evaluation results indicate proposed approach reduces the number of frames to be processed by more than 42.2%.

人工合成方案大多采用高性能服务器，对移动设备的用户交互体验不满意。在智能手机上查看人工合成结果直接增加了用户交互并增强了用户体验。本文提出了一种移动设备上的智能帧选择网络（SFSN），以减少智能手机和云之间的流量。我们利用注意力和关系模型来关注单个帧与整个视频之间的关系，可以更好地选择重要帧，从而有效地减少流量和计算。此外，我们构建了一个基于SFSN的多任务人体合成系统，在一个统一的框架中处理背景变化、姿态迁移和虚拟试穿等生成任务。