Online multiple-output regression is an important machine learning technique for modeling, predicting, and compressing multi-dimensional correlated data streams. In this paper, we propose a novel online multiple-output regression method, called MORES, for streaming data. MORES can dynamically learn the structure of the regression coefficients to facilitate the model's continuous refinement. Considering that limited expressive ability of regression models often leading to residual errors being dependent, MORES intends to dynamically learn and leverage the structure of the residual errors to improve the prediction accuracy. Moreover, we introduce three modified covariance matrices to extract necessary information from all the seen data for training, and set different weights on samples so as to track the data streams' evolving characteristics. Furthermore, an efficient algorithm is designed to optimize the proposed objective function, and an efficient online eigenvalue decomposition algorithm is developed for the modified covariance matrix. Finally, we analyze the convergence of MORES in certain ideal condition. Experiments on two synthetic datasets and three real-world datasets validate the effectiveness and efficiency of MORES. In addition, MORES can process at least 2,000 instances per second (including training and testing) on the three real-world datasets, more than 12 times faster than the state-of-the-art online learning algorithm.

中文翻译：

---

流数据的动态结构嵌入式在线多输出回归


在线多输出回归是一种重要的机器学习技术，用于建模、预测和压缩多维相关数据流。在本文中，我们提出了一种新颖的用于流数据的在线多输出回归方法，称为 MORES。 MORES可以动态学习回归系数的结构，以利于模型的不断细化。考虑到回归模型的表达能力有限，往往导致残差具有依赖性，MORES打算动态学习并利用残差的结构来提高预测精度。此外，我们引入了三个修改的协方差矩阵，从所有可见的数据中提取必要的信息进行训练，并对样本设置不同的权重，以跟踪数据流的演变特征。此外，设计了一种有效的算法来优化所提出的目标函数，并针对修改后的协方差矩阵开发了一种有效的在线特征值分解算法。最后，我们分析了MORES在一定理想条件下的收敛性。对两个合成数据集和三个真实数据集的实验验证了 MORES 的有效性和效率。此外，MORES 可以在三个真实数据集上每秒处理至少 2,000 个实例（包括训练和测试），比最先进的在线学习算法快 12 倍以上。