The era of big data has enabled sophisticated models to predict air pollution concentrations over space and time. Historically these models have been evaluated using overall metrics that measure how close predictions are to monitoring data. However, overall methods are not designed to distinguish error at timescales most relevant for epidemiologic studies, such as day-to-day errors that impact studies of short-term health associations. We introduce frequency band model performance, which quantifies health estimation capacity of air quality prediction models for time series studies of air pollution and health. Frequency band model performance uses a discrete Fourier transform to evaluate prediction models at timescales of interest. We simulated fine particulate matter (PM2.5), with errors at timescales varying from acute to seasonal, and health time series data. To compare evaluation approaches, we use correlations and root mean squared error (RMSE). Additionally, we assess health estimation capacity through bias and RMSE in estimated health associations. We apply frequency band model performance to PM2.5 predictions at 17 monitors in 8 US cities. In simulations, frequency band model performance rates predictions better (lower RMSE, higher correlation) when there is no error at a particular timescale (e.g., acute) and worse when error is added to that timescale, compared to overall approaches. Further, frequency band model performance is more strongly associated (R2 = 0.95) with health association bias compared to overall approaches (R2 = 0.57). For PM2.5 predictions in Salt Lake City, UT, frequency band model performance better identifies acute error that may impact estimated short-term health associations. For epidemiologic studies, frequency band model performance provides an improvement over existing approaches because it evaluates models at the timescale of interest and is more strongly associated with bias in estimated health associations. Evaluating prediction models at timescales relevant for health studies is critical to determining whether model error will impact estimated health associations.

中文翻译：

---

评估空气污染暴露预测模型的健康估计能力

大数据时代使复杂的模型能够预测空间和时间上的空气污染浓度。从历史上看，这些模型是使用衡量预测与监控数据的接近程度的总体指标进行评估的。然而，总体方法并非旨在区分与流行病学研究最相关的时间尺度上的错误，例如影响短期健康关联研究的日常错误。我们引入了频带模型性能，它量化了空气质量预测模型的健康估计能力，用于空气污染和健康的时间序列研究。频带模型性能使用离散傅里叶变换在感兴趣的时间尺度上评估预测模型。我们模拟了细颗粒物 (PM2.5)，其时间尺度上的误差从急性到季节性变化，和健康时间序列数据。为了比较评估方法，我们使用相关性和均方根误差 (RMSE)。此外，我们通过估计的健康关联中的偏差和 RMSE 来评估健康估计能力。我们将频带模型性能应用于美国 8 个城市的 17 个监测器的 PM2.5 预测。在模拟中，与整体方法相比，当在特定时间尺度（例如，急性）没有错误时，频带模型性能对预测的预测更好（更低的 RMSE，更高的相关性），而当错误被添加到该时间尺度时，则更差。此外，与整体方法（R2 = 0.57）相比，频带模型性能与健康关联偏差的相关性更强（R2 = 0.95）。对于犹他州盐湖城的 PM2.5 预测，频带模型性能更好地识别可能影响估计的短期健康关联的急性错误。对于流行病学研究，频带模型性能提供了对现有方法的改进，因为它在感兴趣的时间尺度上评估模型，并且与估计的健康关联中的偏差更密切相关。在与健康研究相关的时间尺度上评估预测模型对于确定模型错误是否会影响估计的健康关联至关重要。