Computational color constancy algorithms are commonly evaluated only through angular error analysis on annotated datasets of static images. The widespread use of videos in consumer devices motivated us to define a richer methodology for color constancy evaluation. To this extent, temporal and spatial stability are defined here to determine the degree of sensitivity of color constancy algorithms to variations in the scene that do not depend on the illuminant source, such as moving subjects or a moving camera. Our evaluation methodology is applied to compare several color constancy algorithms on stable sequences belonging to the Gray Ball and Burst Color Constancy video datasets. The stable sequences, identified using a general-purpose procedure, are made available for public download to encourage future research. Our investigation proves the importance of evaluating color constancy algorithms according to multiple metrics, instead of angular error alone. For example, the popular fully convolutional color constancy with confidence-weighted pooling algorithm is consistently the best performing solution for error evaluation, but it is often surpassed in terms of stability by the traditional gray edge algorithm, and by the more recent sensor-independent illumination estimation algorithm.

中文翻译：

---

关于颜色恒常性算法时空稳定性的评价

计算颜色恒常性算法通常仅通过对静态图像的带注释数据集进行角度误差分析来评估。视频在消费设备中的广泛使用促使我们定义了一种更丰富的色彩恒常性评估方法。就此而言，此处定义了时间和空间稳定性，以确定颜色恒常性算法对不依赖于光源（例如移动对象或移动相机）的场景变化的敏感度。我们的评估方法用于比较属于 Gray Ball 和 Burst Color Constancy 视频数据集的稳定序列的几种颜色恒常算法。使用通用程序确定的稳定序列可供公众下载以鼓励未来的研究。我们的调查证明了根据多个指标评估颜色恒常性算法的重要性，而不仅仅是角度误差。例如，流行的带有置信加权池化算法的全卷积颜色恒常性一直是错误评估的最佳性能解决方案，但在稳定性方面经常被传统的灰度边缘算法和最近的传感器独立照明超越估计算法。