Machine Learning (ML) algorithms have shown to complement standard digital signal processing (DSP) tools in mitigating fiber nonlinearity and improving long-haul transmission performance. However, dynamic transmission impairments such as polarization effects and carrier phase noise corrupt the training data and conventional cost functions for neural network (NN) training become unsuitable. Simple cascade of ML and standard adaptive DSP blocks will also result in suboptimal transmission performance or require impractical training methodologies in presence of such dynamic transmission impairments. We show how the adaptive DSP blocks can be treated as extra stateful NN layers and be combined with the main NN so that standard backpropagation-like training algorithms in ML can be applied. In this case, the adaptive filters are viewed as NN states which are updated in the forward pass of the backpropagation. We study the combined training of linear and nonlinear parameters in the digital backpropagation (DBP) algorithm for fiber nonlinearity compensation (named generalized DBP (GDBP) hereafter), residual impairments, polarization effects, frequency offsets and carrier phase noise compensation filters as a single NN in a 7 × 288 Gb/s polarization multiplexed (PM)-16QAM transmission experiment over 1125 km. We derived the complete set of backpropagation-like gradients and state update equations for the static and dynamic parameters of the combined NN. We further proposed and open-sourced a JAX-based coding framework for their easy and practical implementation. GDBP is more generalized than other DBP variants proposed in literature and for a given total number of steps, GDBP is the first experimental demonstration of optimal single-channel DBP based-fiber nonlinearity compensation algorithm. In addition, for complexity constrained situations with shortened filter taps, GDBP enables a 1 dB performance improvement over other DBP variants.

中文翻译：

---

用于长距离光通信的组合神经网络和自适应 DSP 训练

机器学习 (ML) 算法已证明可以补充标准数字信号处理 (DSP) 工具，以减轻光纤非线性并提高长距离传输性能。然而，极化效应和载波相位噪声等动态传输损伤会破坏训练数据，神经网络 (NN) 训练的传统成本函数变得不合适。ML 和标准自适应 DSP 模块的简单级联也会导致传输性能欠佳，或者在存在此类动态传输损伤时需要不切实际的训练方法。我们展示了如何将自适应 DSP 块视为额外的有状态 NN 层并与主 NN 结合，以便可以应用 ML 中的标准反向传播类训练算法。在这种情况下，自适应滤波器被视为在反向传播的前向传递中更新的 NN 状态。我们研究了数字反向传播 (DBP) 算法中线性和非线性参数的组合训练，用于光纤非线性补偿（以下称为广义 DBP (GDBP)）、残余损伤、极化效应、频率偏移和载波相位噪声补偿滤波器作为单个 NN在超过 1125 公里的 7 × 288 Gb/s 偏振复用 (PM)-16QAM 传输实验中。我们为组合 NN 的静态和动态参数导出了一套完整的类似反向传播的梯度和状态更新方程。我们进一步提出并开源了一个基于 JAX 的编码框架，以实现其简单实用的实现。GDBP 比文献中提出的其他 DBP 变体更通用，对于给定的总步数，GDBP 是基于光纤非线性补偿算法的最佳单通道 DBP 的第一个实验演示。此外，对于具有缩短滤波器抽头的复杂性受限情况，GDBP 可实现比其他 DBP 变体提高 1 dB 的性能。