Sanitary risk inspection, an observation protocol for identifying contamination hazards around water sources, is promoted for managing rural water supply safety. However, it is unclear how far different observers consistently identify contamination hazards and consistently classify water source types using standard typologies. This study aimed to quantify inter-observer agreement in hazard identification and classification of rural water sources. Six observers separately visited 146 domestic water sources in Siaya County, Kenya, in wet and dry seasons. Each observer independently classified the source type and conducted a sanitary risk inspection using a standard protocol. Water source types assigned by an experienced observer were cross-tabulated against those of his colleagues, as were contamination hazards identified, and inter-observer agreement measures calculated. Agreement between hazards observed by the most experienced observer versus his colleagues was significant but low (intra-class correlation = 0.49), with inexperienced observers detecting fewer hazards. Inter-observer agreement in classifying water sources was strong (Cohen’s kappa = 0.84). However, some source types were frequently misclassified, such as sources adapted to cope with water insecurity (e.g. tanks drawing on both piped and rainwater). Observers with limited training and experience thus struggle to consistently identify hazards using existing protocols, suggesting observation protocols require revision and their implementation should be supported by comprehensive training. Findings also indicate that field survey teams struggle to differentiate some water source types based on a standard water source classification, particularly sources adapted to cope with water insecurity. These findings demonstrate uncertainties underpinning international monitoring and analyses of safe water access via household surveys.

卫生风险检查是一种用于识别水源周围污染危害的观察规程，被推广用于管理农村供水安全。但是，目前尚不清楚不同的观察者能在多大程度上一致地确定污染危害并使用标准类型对水资源类型进行一致的分类。这项研究的目的是量化农村水资源的危害识别和分类中观察者之间的共识。在干季和湿季，六名观察员分别访问了肯尼亚Siaya县的146种生活用水。每个观察员都对来源类型进行了独立分类，并使用标准协议进行了卫生风险检查。由经验丰富的观察员分配的水源类型与他的同事的交叉列表，以及确定的污染危害，以及观察者间协议测度。最有经验的观察者与他的同事观察到的危害之间的一致性是显着的，但较低（类内相关度= 0.49），而经验不足的观察者发现的危害较少。观察者之间对水源分类的共识很强（Cohen的kappa = 0.84）。但是，某些水源类型经常被错误分类，例如适应水不安全状况的水源（例如，在管道和雨水上都抽水的水箱）。因此，受过培训和经验有限的观察员很难使用现有协议一致地识别危险，这表明观察协议需要修订，其实施应得到全面培训的支持。调查结果还表明，实地调查小组难以根据标准水源分类来区分某些水源类型，特别是适应水不安全状况的水源。这些发现表明，不确定性是通过家庭调查进行国际监测和安全饮水分析的基础。