Maximal correlation measures the statistical dependence between two random variables and has been broadly used by statisticians. In this paper, we present a novel regression framework based on a special case of the maximal correlation using the correlation ratio to train a Multiple-input Multiple-output (MIMO) model called the Bank of Wiener Models (BWM). Nonlinear time series modeling under maximal correlation unifies the cost and the mapping function under the same mathematical optimization framework. This provides direct optimality (maximal statistical dependence) between BWM model outputs and the desired response, without the restriction of constructing an error signal as conventionally done in regression. Based on the idea of correlation ratio, we propose the Maximal Correlation Algorithm (MCA), which approximates directly the correlation ratio between BWM outputs and the desired response. As an important consequence, MCA modularizes the training of models with hidden layers and avoids the end-to-end training of backpropagation (BP), while improving the equivalent mapping capability. We further propose several possible multi layer arrangements to create deep networks trained with MCA. We demonstrate experimentally that MCA performs at the same level of output error or better when compared to a single and multiple-hidden-layer MLP trained with BP and the Mean Squared Error (MSE). We also show that the system identification capabilities of MCA are superior to MLPs trained with BP. Hence, MCA has great promise in both nonlinear system identification and machine learning.

中文翻译：

---

通过显式学习投影空间进行 MIMO 建模：最大相关比成本函数


最大相关性衡量两个随机变量之间的统计依赖性，已被统计学家广泛使用。在本文中，我们提出了一种基于最大相关性特殊情况的新颖回归框架，使用相关性比率来训练称为维纳模型银行（BWM）的多输入多输出（MIMO）模型。最大相关性下的非线性时间序列建模将成本和映射函数统一在相同的数学优化框架下。这提供了 BWM 模型输出和所需响应之间的直接最优性（最大统计依赖性），而不受回归中传统做法构建误差信号的限制。基于相关比的思想，我们提出了最大相关算法（MCA），它直接近似BWM输出和期望响应之间的相关比。一个重要的结果是，MCA将具有隐藏层的模型的训练模块化，避免了反向传播（BP）的端到端训练，同时提高了等效映射能力。我们进一步提出了几种可能的多层安排来创建用 MCA 训练的深度网络。我们通过实验证明，与使用 BP 和均方误差 (MSE) 训练的单个和多个隐藏层 MLP 相比，MCA 的输出误差水平相同或更好。我们还表明，MCA 的系统识别能力优于用 BP 训练的 MLP。因此，MCA 在非线性系统识别和机器学习方面都具有广阔的前景。