Cross-modal communication is playing an increasingly important role in improving receivers’ immersive experience. The main challenge lies in ensuring the heterogeneous requirements of the cross-modal stream. Especially, the discontinuity of the received haptic signals caused by the delay should be eliminated. Unfortunately, existing schemes study the cross-modal stream separately, which leads to that the haptic signal is distorted, or the audio-visual quality is reduced. To solve this problem fundamentally, we propose a joint transmission framework combining prediction and device-to-device (D2D) communication by taking advantage of the correlation of the haptic signals and the proximity feature of the receivers. Specifically, on the theoretical end, to completely eliminate the discontinuity, we propose a prediction mechanism by predicting and sending the future signals in advance. To compensate for the reliability loss brought by prediction, D2D links are efficiently established on the receivers’ side. On the technical end, we first design a minimum resource (e.g., power) consumption search algorithm based on the binary search to obtain the optimal prediction horizon. Moreover, we develop a simple but efficient transmission mode selection algorithm based on the Hungarian algorithm. Experimental results demonstrate the advantages of our proposed scheme in saving the power consumption.

中文翻译：

---

跨模式传输的异构流调度

跨模式通信在改善接收者的沉浸式体验方面发挥着越来越重要的作用。主要挑战在于确保跨模式流的异构需求。特别是，应消除延迟引起的接收触觉信号的不连续性。遗憾的是，现有方案单独研究跨模态流，导致触觉信号失真，或降低视听质量。为了从根本上解决这个问题，我们提出了一种结合预测和设备到设备（D2D）通信的联合传输框架，利用触觉信号的相关性和接收器的邻近特征。具体来说，在理论上，为了完全消除不连续性，我们提出了一种通过提前预测和发送未来信号的预测机制。为了补偿预测带来的可靠性损失，在接收端有效地建立了 D2D 链路。在技​​术端，我们首先设计了一种基于二分搜索的最小资源（例如，功率）消耗搜索算法，以获得最优预测范围。此外，我们基于匈牙利算法开发了一种简单但高效的传输模式选择算法。实验结果证明了我们提出的方案在节省功耗方面的优势。power) 消耗搜索算法基于二分搜索来获得最优预测范围。此外，我们基于匈牙利算法开发了一种简单但高效的传输模式选择算法。实验结果证明了我们提出的方案在节省功耗方面的优势。power) 消耗搜索算法基于二分搜索来获得最优预测范围。此外，我们基于匈牙利算法开发了一种简单但高效的传输模式选择算法。实验结果证明了我们提出的方案在节省功耗方面的优势。