This paper proposes a convolution neural network (CNN) based prediction method for concrete diaphragm wall (CDW) deflections. CNN algorithm is modified for processing the CDW deformation data collected from in-situ measurement in both time and space dimensions, and capable of making dynamic prediction based on the extracted spatiotemporal features of wall deflection. The proposed method is validated through investigating a project of deep excavation in Suzhou, China. The predicted results show excellent agreement with field measurement and yield mean absolute errors of 0.86 mm and 1.55 mm for nowcasting and forecasting tasks, respectively. Three prevailing algorithms in time series prediction, namely, back propagation neural network, long short-term memory and autoregressive integrated moving average, are conducted for comparison. The results illustrate that the CNN outperforms the other algorithms in terms of accuracy and execution time. Therefore, the proposed CNN model is the most suitable for CDW deflection prediction, and can provide reasonable references for construction safety management on site.