It is well known that concurrent multipath transfer (CMT) can improve the transmission rate. However, due to multiple heterogeneous paths from users to the access network, a large number of out-of-order packets significantly degrade the overall transmission reliability. Cybertwin provides a potential solution to alleviate the packet out-of-order problem by accurately detecting and perceiving the path state. In this article, we investigate the data scheduling problem and propose a learning-based cybertwin-driven CMT algorithm to obtain the optimal data scheduling policy. In particular, we first formulate the data scheduling problem as an integer linear programming by taking the QoS metrics into account. To cope with the packet out-of-order problem in CMT, we propose a reliable cybertwin-CMT with deep reinforcement learning (CMT-DRL) algorithm to determine the data scheduling decisions. The proposed algorithm takes multipath throughput, end-to-end delay, and packet loss rate into account. Besides, CMT-DRL adopts an asynchronous learning framework to efficiently execute data collection, packet scheduling, and neural network training in sequence by decoupling model training and execution. We conduct extensive experiments in a P4-based programmable network platform. Experimental results indicate that the CMT-DRL outperforms the existing benchmarks in terms of the number of out-of-order packets, round-trip time, and throughput.

中文翻译：

---

具有深度强化学习的可靠的 Cyber​​twin 驱动的并发多路径传输

众所周知，并发多径传输（CMT）可以提高传输速率。但是，由于用户到接入网的路径存在多条异构路径，大量的乱序报文大大降低了整体传输的可靠性。Cyber​​twin 提供了一种潜在的解决方案，通过准确检测和感知路径状态来缓解数据包乱序问题。在本文中，我们研究了数据调度问题，并提出了一种基于学习的网络孪生驱动的 CMT 算法，以获得最优的数据调度策略。特别是，我们首先通过考虑 QoS 指标将数据调度问题表述为整数线性规划。为了应对CMT中的数据包乱序问题，我们提出了一种可靠的具有深度强化学习（CMT-DRL）算法的cybertwin-CMT来确定数据调度决策。所提出的算法考虑了多径吞吐量、端到端延迟和丢包率。此外，CMT-DRL采用异步学习框架，通过解耦模型训练和执行，依次高效执行数据采集、数据包调度和神经网络训练。我们在基于 P4 的可编程网络平台上进行了大量实验。实验结果表明，CMT-DRL 在乱序数据包数量、往返时间和吞吐量方面优于现有基准。CMT-DRL采用异步学习框架，通过模型训练和执行解耦，依次高效执行数据采集、数据包调度、神经网络训练。我们在基于 P4 的可编程网络平台上进行了大量实验。实验结果表明，CMT-DRL 在乱序数据包数量、往返时间和吞吐量方面优于现有基准。CMT-DRL采用异步学习框架，通过模型训练和执行解耦，依次高效执行数据采集、数据包调度、神经网络训练。我们在基于 P4 的可编程网络平台上进行了大量实验。实验结果表明，CMT-DRL 在乱序数据包数量、往返时间和吞吐量方面优于现有基准。