Wearable sensors have been explored as an alternative for real-time monitoring of cattle feeding behavior in grazing systems. To evaluate the performance of predictive models such as machine learning (ML) techniques, data cross-validation (CV) approaches are often employed. However, due to data dependencies and confounding effects, poorly performed validation strategies may significantly inflate the prediction quality. In this context, our objective was to evaluate the effect of different CV strategies on the prediction of grazing activities in cattle using wearable sensor (accelerometer) data and ML algorithms. Six Nellore bulls (average live weight of 345 ± 21 kg) had their behavior visually classified as grazing or not-grazing for a period of 15 d. Elastic Net Generalized Linear Model (GLM), Random Forest (RF), and Artificial Neural Network (ANN) were employed to predict grazing activity (grazing or not-grazing) using 3-axis accelerometer data. For each analytical method, three CV strategies were evaluated: holdout, leave-one-animal-out (LOAO), and leave-one-day-out (LODO). Algorithms were trained using similar dataset sizes (holdout: n = 57,862; LOAO: n = 56,786; LODO: n = 56,672). Overall, GLM delivered the worst prediction accuracy (53%) compared with the ML techniques (65% for both RF and ANN), and ANN performed slightly better than RF for LOAO (73%) and LODO (64%) across CV strategies. The holdout yielded the highest nominal accuracy values for all three ML approaches (GLM: 59%, RF: 76%, and ANN: 74%), followed by LODO (GLM: 49%, RF: 61%, and ANN: 63%) and LOAO (GLM: 52%, RF: 57%, and ANN: 57%). With a larger dataset (i.e., more animals and grazing management scenarios), it is expected that accuracy could be increased. Most importantly, the greater prediction accuracy observed for holdout CV may simply indicate a lack of data independence and the presence of carry-over effects from animals and grazing management. Our results suggest that generalizing predictive models to unknown (not used for training) animals or grazing management may incur poor prediction quality. The results highlight the need for using management knowledge to define the validation strategy that is closer to the real-life situation, i.e., the intended application of the predictive model.

中文翻译：

---

使用交叉验证策略解开数据依赖性，使用机器学习算法和可穿戴传感器数据评估放牧活动的预测质量

可穿戴传感器已被探索为实时监测放牧系统中牛饲养行为的替代方案。为了评估机器学习 (ML) 技术等预测模型的性能，通常采用数据交叉验证 (CV) 方法。然而，由于数据依赖性和混杂效应，执行不佳的验证策略可能会显着提高预测质量。在这种情况下，我们的目标是使用可穿戴传感器（加速度计）数据和 ML 算法评估不同 CV 策略对预测牛放牧活动的影响。六头 Nellore 公牛（平均活重 345 ± 21 kg）在 15 天的时间内将它们的行为视觉分类为放牧或不放牧。弹性网络广义线性模型 (GLM)、随机森林 (RF)、和人工神经网络 (ANN) 用于使用 3 轴加速度计数据预测放牧活动（放牧或不放牧）。对于每种分析方法，评估了三种 CV 策略：坚持、留一动物离开 (LOAO) 和留一天 (LODO)。使用相似的数据集大小训练算法（保留：n = 57,862；LOAO：n = 56,786；LODO：n = 56,672）。总体而言，与 ML 技术（RF 和 ANN 均为 65%）相比，GLM 提供了最差的预测准确度（53%），而 ANN 在 CV 策略中的 LOAO（73%）和 LODO（64%）表现略好于 RF。对于所有三种 ML 方法（GLM：59%、RF：76% 和 ANN：74%），holdout 产生了最高的标称准确度值，其次是 LODO（GLM：49%、RF：61% 和 ANN：63% ) 和 LOAO (GLM: 52%, RF: 57%, 和 ANN: 57%)。使用更大的数据集（即，更多的动物和放牧管理场景），预计可以提高准确性。最重要的是，对于保留 CV 观察到的更高预测准确性可能只是表明缺乏数据独立性以及存在来自动物和放牧管理的遗留效应。我们的结果表明，将预测模型推广到未知（未用于训练）动物或放牧管理可能会导致预测质量不佳。结果强调需要使用管理知识来定义更接近现实情况的验证策略，即预测模型的预期应用。对保留 CV 观察到的更高预测准确性可能只是表明缺乏数据独立性以及存在来自动物和放牧管理的遗留效应。我们的结果表明，将预测模型推广到未知（未用于训练）动物或放牧管理可能会导致预测质量不佳。结果强调需要使用管理知识来定义更接近现实情况的验证策略，即预测模型的预期应用。对保留 CV 观察到的更高预测准确性可能只是表明缺乏数据独立性以及存在来自动物和放牧管理的遗留效应。我们的结果表明，将预测模型推广到未知（未用于训练）动物或放牧管理可能会导致预测质量不佳。结果强调需要使用管理知识来定义更接近现实情况的验证策略，即预测模型的预期应用。