Fully automatic spatiotemporal segmentation of 3D LiDAR time series for the extraction of natural surface changes

Abstract

Geographic observation benefits from the increasing availability of time series of 3D geospatial data, which allow analysis of change processes at high temporal detail and over extensive periods. In this context, the demand for advanced methods to detect and extract topographic surface changes from these 4D geospatial data emerges. Changes in natural scenes occur with varying magnitude, duration, spatial extent, and change rate, and the timing of their occurrence is not known. Standard pairwise change detection requires the selection of fixed analysis periods and the specification of magnitude thresholds to determine accumulation or erosion forms. In settings with continuous surface morphology and dynamic changes to the surface due to material transport, such change forms are typically temporary and may be missed or aggregated if they occur with spatial and/or temporal overlap. This is overcome with the extraction of 4D objects-by-change (4D-OBCs). These objects are obtained by firstly detecting surface changes in the temporal domain at locations in the scene. Subsequently, they are spatially delineated by considering the full history of surface change during region growing from the seed location of a detected change. To perform this spatiotemporal segmentation systematically for entire 3D time series, we develop a fully automatic approach of seed detection and selection, combined with locally adaptive thresholding for region growing of individual objects with varying change properties. We apply our workflow to a five-months hourly time series of around 3,000 terrestrial laser scanning point clouds acquired for coastal monitoring at a sandy beach in The Netherlands. This provides 2,021 4D-OBCs as extracted accumulation or erosion forms. Results are validated through majority agreement of six expert analysts, who evaluate the segmentation performance at sample locations throughout the scene. Accordingly, our method extracts surface changes with an error of omission of 4.7% and an error of commission of 16.6%. We examine the results and provide considerations how postprocessing of segments can further improve the change analysis workflow. The developed approach thereby provides a powerful tool for automatic change analysis in 4D geospatial data, namely to detect and delineate natural surface changes across space and time.

Introduction

Time series of topographic 3D data pose great possibilities to geographic analyses, but also challenges to the detection and delineation of surface change information from these 4D geospatial data. Surface change analysis using topographic point clouds has long since gained considerable importance in the observation of Earth surface processes and in advancing geoscientific research in general (Eitel et al., 2016, Qin et al., 2016). Ongoing repetitions of topographic surveys have led to the cumulation of data in the temporal dimension that enable change detection for many use cases of Earth surface observation, among them studies of changes to landslides (Oppikofer et al., 2009, Pfeiffer et al., 2018), rockfalls (Abellán et al., 2010, Rosser et al., 2007), rock glaciers (Bodin et al., 2018, Zahs et al., 2019), snow cover (Grünewald et al., 2010, Fey et al., 2019), and the coast (Fabbri et al., 2017, Miles et al., 2019).

Most recently, the acquisition strategy of permanent terrestrial laser scanning (TLS) generates time series of 3D point clouds at (sub-)hourly intervals over periods of weeks to months (e.g., Kromer et al., 2017, O'Dea et al., 2019, Stumvoll et al., 2020, Vos et al., 2017, Williams et al., 2018). Such high-frequency time series of point clouds can alternatively be acquired by photogrammetric techniques (e.g., Eltner et al., 2017, Kromer et al., 2019), albeit obtained datasets have different properties from laser scanning point clouds and may require other specific processing strategies. The unprecedented temporal density of these 4D datasets provides many more epochs for change analysis to compare evermore combinations of pairwise states of the topography, which are ideally adapted to the rates of target changes, respectively. Even more importantly, the information these data potentially contain on temporal properties of change processes hold opportunity for new insights on spatiotemporal characteristics of topographic activity and, consequently, to extend our fundamental knowledge about the investigated geographic phenomena (Eitel et al., 2016, Eltner et al., 2017).

To leverage the temporal dimension of 3D time series for change detection and delineation, a method of spatiotemporal segmentation was developed that makes use of the full history of surface change to extract periods and spatial extents of surface changes (Anders et al., 2020b). The time series-based approach is designed to advance established approaches of pairwise change detection. Pairwise change analysis typically serves to identify areas of accumulation or erosion over the selected analysis period and to quantify change rates. Based on the bitemporal change information, patterns and underlying drivers of change are interpreted (e.g., Anders et al., 2020a, Eltner et al., 2017, Fey et al., 2019, Zahs et al., 2019). Standard methods for pairwise change detection are the differencing of Digital Elevation Models (DEMs, James et al., 2012) or point cloud distance computation (Girardeau-Montaut et al., 2005, Lague et al., 2013). Alternatively, change can be assessed in object-based approaches, where observed objects are first identified based on morphometric features or even previously derived bitemporal surface change, and subsequently changes in object properties are analysed, such as their location and size (e.g., Mayr et al., 2018).

In the following, we reveal drawbacks of pairwise change detection methods for the analysis of 3D time series. These drawbacks arise from the general circumstance of observing natural, Earth shaping processes that it is not a priori known when and where changes occur within a scene, and what the spatial and temporal properties of these changes are. The required selection of epochs in pairwise change analysis entails that the periods for detecting change are pre-defined. Temporary surface changes, which only persist for a limited amount of time within the observed scene, may hence be missed in the analysis (Anders et al., 2019) as their timing and/or existence are not known to the analyst and their disappearance is not expressed in later topographic information (Fig. 1A). Performing pairwise change analysis for all combinations of epochs to solve this drawback would be somewhat impractical, and has not been done so far to our knowledge. Pairwise change analysis, therefore, is not adequate to analyse 3D time series for changes that occur with highly varying temporal characteristics, that is timing, change rate, duration of change processes, and persistence of change forms. Surface changes further occur at varying spatial scales regarding their extent, shape and magnitude and can therefore not be extracted generically with one-for-all settings. Where morphologic boundaries of objects or forms moreover are not distinct, it is difficult to spatially delineate them in individual scenes. Binary surface change information (change/no change) has been used to identify and delineate so-called change objects (Liu et al., 2010). However, these spatially contiguous areas of surface change do not necessarily stem from the same change-inducing process (Fig. 1B). Separating them into individual objects without a priori knowledge or information on external influences is improved when integrating the history of surface change that is contained in the 3D time series in the analysis (Anders et al., 2020b).

Time series clustering (Kuschnerus et al., 2020) has been proposed for extracting change information from large 4D geospatial data. The method is useful to extract areas that are homogenous in their change dynamics and thereby finding dominant change patterns in the observed scene. It is not possible, though, to identify individual, spatially and temporally limited change occurrences as the full time series of the observation period are used as input.

Object extraction by integrating the history of surface change can be performed with the concept of 4D objects-by-change (4D-OBCs; Anders et al., 2020b). This method identifies areas in the scene where the surface changes similarly over time within sub-periods in the time series at neighbouring locations. Sub-periods are automatically detected in the temporal domain of a location and are subsequently used as seeds for spatial region growing with time series similarity as homogeneity criterion. The seed locations at which to perform the temporal change detection have been selected manually so far. However, changes occurring are spatially variable within a scene and their timing and location is in general not known to the analyst. Automatic extraction of changes from entire datasets will therefore require fully automatic seed selection from all seed candidates. These can be obtained as sub-periods via temporal change detection at all locations in the scene. Many of these detected sub-periods will be both spatially neighbouring and temporally overlapping, and thereby likely belong to the same change form. One could perform the segmentation for all detected sub-periods at all locations and subsequently aggregate segments into unique 4D-OBCs in a post-processing step. This option is hardly viable, given the large data volumes of 3D time series and considering the extreme redundancy of computations if each location within a change form is used as seed to obtain the same, single object.

The drawbacks outlined above become particularly apparent in settings with continuous surface morphology and dynamic changes of the surface due to material transport induced by varying external drivers. Therefore, the use case of this paper is TLS-based monitoring of a sandy beach, using an hourly time series spanning five months. Sandy beaches are highly active in their morphodynamics through multiple processes acting on the surface, as these coastal landscapes are subject to continual sediment transport by wind, waves, as well as anthropogenic modifications. Their surface is hence shaped by a variety of (temporary) forms of accumulation, erosion, and transported material (Walker et al., 2017). Therefore, the target changes to be extracted from our data are temporary accumulation and erosion forms which typically exist over periods of days to few weeks.

The objective of this paper is to develop a fully automatic workflow to extract surface changes as temporary accumulation or erosion forms in their varying spatial and temporal extents from a long and dense 3D time series dataset. To achieve this, we implement methods of automatic seed selection and locally adaptive thresholding for spatiotemporal segmentation. We consider the following methodological aspects:

• Integrating the history of surface change in temporal change detection will avoid missing temporary surface changes in the analysis which may not persist throughout epochs that are selected for fixed-period analyses.

• Sorting and selecting seeds for region growing by an appropriate metric and considering previous segments throughout continued segmentation allows avoiding redundant calculations but also prevents skipping relevant change occurrences.

• A locally adaptive approach of threshold selection is more suitable than pre-defined thresholds, albeit strict or loose, in order to avoid general over- or underestimation of spatial extents due to dependence on magnitude, duration, and change rate of the respective change form.

The developed automatic spatiotemporal segmentation approach is designed to improve both the spatial separation of change forms, e.g. two co-occurring 4D-OBCs that would otherwise be extracted as one accumulation or erosion object, and the temporal separation of change forms, e.g., two consecutive 4D-OBCs that could be aggregated or missed with alternative methods (cf. Fig. 1). Our approach thereby provides an important contribution to the toolset for change analysis in large 4D geospatial data.