Skip to main content
Log in

A guide for evaluating and reporting map data quality: Affirming Shao et al. “Overselling overall map accuracy misinforms about research reliability”

  • Perspective
  • Published:
Landscape Ecology Aims and scope Submit manuscript

Abstract

Context

Landscape ecologists often use thematic map data in their research. Greater familiarity with thematic map accuracy assessment protocols will enhance appropriate use and interpretation of map quality data.

Objectives

Provide an overview of thematic map accuracy assessment protocols and simple, non-quantitative guidelines to assess the quality of the thematic map data that landscape ecologists use in their research.

Methods

Synthesis and interpretation of salient literature on map accuracy assessment.

Conclusions

Landscape ecologists can adopt three simple rules to improve their use and interpretation of map data: (1) use the map quality data only if the accuracy assessment protocols adhere to rigorous, well-established standards for the sampling design, response design, and analysis; (2) focus on class-specific accuracy via user’s and producer’s accuracies (or the complementary measures commission and omission error rates); and (3) use the criterion that an accuracy assessment that reports class-specific accuracies accompanied by standard errors is a strong indicator of a rigorous assessment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anderson JR, Hardy EE, Roach JT, Witmer RE (1976) A land use and land cover system for use with remote sensor data. U.S. Geological Survey, Professional Paper 964. https://doi.org/10.3133/pp964; https://pubs.er.usgs.gov/publication/pp964

  • Hess GR, Bay JM (1997) Generating confidence intervals for composition-based landscape indexes. Landsc Ecol 12:309–320

    Article  Google Scholar 

  • Khorram S, Biging GS, Chrisman NR, Colby DR, Congalton RG, Dobson JF, Ferguson RL, Goodchild MF, Jensen JR, Mace TH (1999) Accuracy assessment of remote sensing-derived change detection. ASPRS Monograph Series. American Society of Photogrammetry and Remote Sensing (ASPRS), Bethesda

    Google Scholar 

  • Olofsson P, Foody GM, Herold M, Stehman SV, Woodcock CE, Wulder MA (2014) Good practices for estimating area and assessing accuracy of land change. Remote Sens Environ 148:42–57

    Article  Google Scholar 

  • Pontius RG Jr, Millones M (2011) Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment. Int J Remote Sens 32:4407–4429

    Article  Google Scholar 

  • Shao G, Wu J (2008) On the accuracy of landscape pattern analysis using remote sensing data. Landsc Ecol 23:505–511

    Article  Google Scholar 

  • Shao G, Tang L, Liao J (2019) Overselling overall map accuracy misinforms about research reliability. Landsc Ecol 34:2487–2492

    Article  Google Scholar 

  • Smith JH, Wickham JD, Stehman SV, Yang L (2002) Impacts of patch size and land cover heterogeneity on thematic image classification accuracy. Photogram Eng and Remote Sens 68:65–70

    Google Scholar 

  • Smith JH, Stehman SV, Wickham JD, Yang L (2003) Effects of landscape characteristics on land-cover class accuracy. Remote Sens Environ 84:342–349

    Article  Google Scholar 

  • Stehman SV (2001) Statistical rigor and practical utility in thematic map assessment. Photogramm Eng Remote Sens 67:727–734

    Google Scholar 

  • Stehman SV, Czaplewski RL (1998) Design and analysis of thematic map accuracy assessment: fundamental principles. Remote Sens Environ 64:331–344

    Article  Google Scholar 

  • Stehman SV, Foody GM (2019) Key issues in rigorous assessment of land cover products. Remote Sens Environ 231:111199

    Article  Google Scholar 

  • Stehman SV, Wickham J (2011) Pixels, blocks of pixels, and polygons: choosing a spatial unit for thematic accuracy assessment. Remote Sens Environ 115:3044–3055

    Article  Google Scholar 

  • Stehman SV, Wickham J, Wade TG, Smith JH (2008) Designing a multi-objective, multi-support accuracy assessment of the 2001 National Land Cover Data (NLCD 2001) of the conterminous United States. Photogramm Eng Remote Sens 74:1561–1571

    Article  Google Scholar 

  • Story M, Congalton RG (1986) Accuracy assessment: a user’s perspective. Photogramm Eng Remote Sens 57:397–399

    Google Scholar 

  • Wickham J, Stehman SV, Gass L, Dewitz J, Fry JA, Wade TG (2013) Accuracy assessment of NLCD 2006 land cover and impervious surface. Remote Sens Environ 130:294–304

    Article  Google Scholar 

  • Wickham J, Stehman SV, Gass L, Dewitz JA, Sorenson DG, Granneman BJ, Poss RV, Baer LA (2017) Thematic accuracy assessment of the 2011 National Land Cover Database (NLCD). Remote Sens Environ 191:328–341

    Article  Google Scholar 

  • Wickham JD, O’Neill RV, Riitters KH, Wade TG, Jones KB (1997) Sensitivity of selected landscape pattern metrics to land-cover misclassification and differences in land cover composition. Photogramm Eng Remote Sens 63:397–402

    Google Scholar 

Download references

Acknowledgements

The commentary described in this paper has been funded by the U.S. Environmental Protection Agency. We thank the anonymous reviewers and Maliha Nash (US EPA) for their valuable comments on earlier versions of the paper. The paper has been subjected to Agency review and has been approved for publication. The views expressed in this journal article are those of the authors and do not necessarily reflect the views or policies of the U.S. Environmental Protection Agency. Mention of trade names or commercial products does not constitute endorsement or recommendation for use. S. Stehman’s participation was underwritten by contract G12AC20221 between SUNY-ESF and the U.S. Dept. of Interior, Geological Survey.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to J. Wickham.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stehman, S.V., Wickham, J. A guide for evaluating and reporting map data quality: Affirming Shao et al. “Overselling overall map accuracy misinforms about research reliability”. Landscape Ecol 35, 1263–1267 (2020). https://doi.org/10.1007/s10980-020-01029-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10980-020-01029-1

Navigation