Image time series, such as Satellite Image Time Series (SITS) or MRI functional sequences in the medical domain, carry both spatial and temporal information. In many pattern recognition applications such as image classification, taking into account such rich information may be crucial and discriminative during the decision making stage. However, the extraction of spatio-temporal features from image time series is difficult to handle due to the complex representation of the data cube. In this paper, we present a strategy based on Random Walk to build a novel segment-based representation of the data, passing from a 2D+t dimension to a 2D one, more easily manipulable and without losing too much spatial information. Such new representation is then used to feed a classical Convolutional Neural Network (CNN) in order to learn spatio-temporal features with only 2D convolutions and to classify image time series data for a particular classification problem. The influence of the way the 2D+t data are represented, as well as the impact of the network architectures on the results, are carefully studied. The interest of this approach is highlighted on a remote sensing application for the classification of complex agricultural crops.

中文翻译：

---

数据表示和深度架构对图像时间序列分类的影响

图像时间序列，如卫星图像时间序列 (SITS) 或医学领域的 MRI 功能序列，同时携带空间和时间信息。在许多模式识别应用（例如图像分类）中，考虑到如此丰富的信息在决策阶段可能是至关重要的和具有辨别力的。然而，由于数据立方体的复杂表示，从图像时间序列中提取时空特征很难处理。在本文中，我们提出了一种基于随机游走的策略来构建一种新颖的基于分段的数据表示，从 2D+吨维度到二维，更容易操作并且不会丢失太多空间信息。然后将这种新表示用于输入经典卷积神经网络 (CNN)，以便仅使用 2D 卷积来学习时空特征，并对特定分类问题的图像时间序列数据进行分类。2D方式的影响+吨数据被表示，以及网络架构对结果的影响，都经过仔细研究。这种方法的兴趣在于用于复杂农作物分类的遥感应用。