Generalization of time series prediction remains an important open issue in machine learning; earlier methods have either large generalization errors or local minima. Here, we develop an analytically solvable, unsupervised learning scheme that extracts the most informative components for predicting future inputs, which we call predictive principal component analysis (PredPCA). Our scheme can effectively remove unpredictable noise and minimize test prediction error through convex optimization. Mathematical analyses demonstrate that, provided with sufficient training samples and sufficiently high-dimensional observations, PredPCA can asymptotically identify hidden states, system parameters and dimensionalities of canonical nonlinear generative processes, with a global convergence guarantee. We demonstrate the performance of PredPCA using sequential visual inputs comprising handwritten digits, rotating three-dimensional objects and natural scenes. It reliably estimates distinct hidden states and predicts future outcomes of previously unseen test input data, based exclusively on noisy observations. The simple architecture and low computational cost of PredPCA are highly desirable for neuromorphic hardware.

时间序列预测的泛化仍然是机器学习中一个重要的开放问题；较早的方法要么具有较大的泛化误差，要么具有局部最小值。在这里，我们开发了一种可分析解决的无监督学习方案，该方案提取了用于预测未来输入的信息量最大的成分，我们称之为预测主成分分析 (PredPCA)。我们的方案可以通过凸优化有效地去除不可预测的噪声并最小化测试预测误差。数学分析表明，在提供足够的训练样本和足够高维的观测值的情况下，PredPCA 可以渐近地识别典型非线性生成过程的隐藏状态、系统参数和维数，并具有全局收敛性保证。我们使用包括手写数字、旋转 3D 对象和自然场景的顺序视觉输入来展示 PredPCA 的性能。它仅基于噪声观察可靠地估计不同的隐藏状态并预测以前看不见的测试输入数据的未来结果。PredPCA 的简单架构和低计算成本非常适合神经形态硬件。