1. In biodiversity monitoring, large datasets are becoming more and more widely available and are increasingly used globally to estimate species trends and conservation status. These large-scale datasets challenge existing statistical analysis methods, many of which are not adapted to their size, incompleteness and heterogeneity. The development of scalable methods to impute missing data in incomplete large-scale monitoring datasets is crucial to balance sampling in time or space and thus better inform conservation policies.
2. We developed a new method based on penalized Poisson models to impute and analyse incomplete monitoring data in a large-scale framework. The method allows parameterization of (a) space and time factors, (b) the main effects of predictor covariates, as well as (c) space–time interactions. It also benefits from robust statistical and computational capability in large-scale settings.
3. The method was tested extensively on both simulated and real-life waterbird data, with the findings revealing that it outperforms six existing methods in terms of missing data imputation errors. Applying the method to 16 waterbird species, we estimated their long-term trends for the first time at the entire North African scale, a region where monitoring data suffer from many gaps in space and time series.
4. This new approach opens promising perspectives to increase the accuracy of species-abundance trend estimations. We made it freely available in the r package ‘lori’ (https://CRAN.R-project.org/package=lori) and recommend its use for large-scale count data, particularly in citizen science monitoring programmes.

中文翻译：

---

通过惩罚估计对不完整的大规模监测计数数据进行插补

1. 在生物多样性监测中，大型数据集变得越来越广泛可用，并且越来越多地在全球范围内用于估计物种趋势和保护状况。这些大规模数据集挑战了现有的统计分析方法，其中许多方法不适应其规模、不完整性和异质性。开发可扩展的方法来估算不完整的大规模监测数据集中的缺失数据对于平衡时间或空间的采样至关重要，从而更好地为保护政策提供信息。
2. 我们开发了一种基于惩罚泊松模型的新方法，可以在大规模框架中估算和分析不完整的监测数据。该方法允许参数化（a）空间和时间因素，（b）预测协变量的主要影响，以及（c）时空相互作用。它还受益于大规模设置中强大的统计和计算能力。
3. 该方法在模拟和现实生活中的水鸟数据上进行了广泛测试，结果表明它在缺失数据插补错误方面优于六种现有方法。将该方法应用于 16 种水鸟物种，我们首次在整个北非范围内估计了它们的长期趋势，该地区的监测数据在空间和时间序列上存在许多差距。
4. 这种新方法为提高物种丰富度趋势估计的准确性开辟了前景。我们在r包“ lori ” (https://CRAN.R-project.org/package=lori) 中免费提供它，并建议将其用于大规模计数数据，特别是在公民科学监测计划中。