Abstract
With data comes uncertainty, which is a widespread and frequent phenomenon in data science and analysis. The amount of information available to us is growing exponentially, owing to never-ending technological advancements. Data visualization is one of the ways to convey that information effectively. Since the error is intrinsic to data, users cannot ignore it in visualization. Failing to observe it in visualization can lead to flawed decision-making by data analysts. Data scientists know that missing out on uncertainty in data visualization can lead to misleading conclusions about data accuracy. In most cases, visualization approaches assume that the information represented is free from any error or unreliability; however, this is rarely true. The goal of uncertainty visualization is to minimize the errors in judgment and represent the information as accurately as possible. This survey discusses state-of-the-art approaches to uncertainty visualization, along with the concept of uncertainty and its sources. From the study of uncertainty visualization literature, we identified popular techniques accompanied by their merits and shortcomings. We also briefly discuss several uncertainty visualization evaluation strategies. Finally, we present possible future research directions in uncertainty visualization, along with the conclusion.
Graphic Abstract
Similar content being viewed by others
References
Aerts JC, Clarke KC, Keuper AD (2003) Testing popular visualization techniques for representing model uncertainty. Cartogr Geogr Inform Sci 30(3):249–261
Aggarwal CC, Philip SY (2009) A survey of uncertain data algorithms and applications. IEEE Transactions on Knowledge and Data Engineering 21(5):609–623
Ai T, Xin R, Yan X, Yang M, Ai B (2019) Shape decision-making in map-like visualization design using the simulated annealing algorithm. IEEE Access 7:131577–131592
Aigner W, Miksch S, Thurnher B, Biffl S (2005) Planninglines: novel glyphs for representing temporal uncertainties and their evaluation. In: Proceedings ninth international conference on information visualisation. IEEE, pp 457–463
Allendes Osorio R, Brodlie KW (2008) Contouring with uncertainty. In: Proceedings of theory and practice of computer graphics. Eurographics Association, pp 59–66
AArras P, Frank P, Leike R, Westermann R, Enßlin T (2019) Unified radio interferometric calibration and imaging with joint uncertainty quantification. arXiv preprint arXiv:190311169
Auber D, Huet C, Lambert A, Renoust B, Sallaberry A, Saulnier A (2013) Gospermap: using a gosper curve for laying out hierarchical data. IEEE Trans Vis Comput Graph 19(11):1820–1832
Balabanian JP, Viola I, Gröller E (2010) Interactive illustrative visualization of hierarchical volume data. In: Proceedings of graphics interface 2010. Canadian Information Processing Society, pp 137–144
Balzer M, Deussen O, Lewerentz C (2005) Voronoi treemaps for the visualization of software metrics. In: Proceedings of the 2005 ACM symposium on Software visualization. ACM, pp 165–172
Barthelmé S, Mamassian P (2009) Evaluation of objective uncertainty in the visual system. PLoS Comput Biol 5(9):e1000504
Bechara A, Damasio H, Tranel D, Anderson SW (1998) Dissociation of working memory from decision making within the human prefrontal cortex. J Neurosci 18(1):428–437
Benjamini Y (1988) Opening the box of a boxplot. Am Stat 42(4):257–262
Bensema K, Gosink L, Obermaier H, Joy K (2016) Modality-driven classification and visualization of ensemble variance. IEEE Trans Vis Comput Graph 22(10):2289–2299
Bertin J (1973) Sémiologie graphique: Les diagrammes-les réseaux-les cartes. Technical report. Gauthier-VillarsMouton & Cie
Bertin J (1983) Semiology of graphics: diagrams, networks, maps (wj berg, trans.). The University of Wisconsin Press, Ltd, Madison, WI
Bertin J (1999) Graphics and graphic information processing. In: Readings in information visualization. Morgan Kaufmann Publishers Inc., pp 62–65
Bisantz AM, Kesevadas T, Scott P, Lee D, Basapur S, Bhide P, Bhide P, Bhide P (2002) Holistic battlespace visualization: advanced concepts in information visualization and cognitive studies. U Buffalo
Bonneau GP, Hege HC, Johnson CR, Oliveira MM, Potter K, Rheingans P, Schultz T (2014) Overview and state-of-the-art of uncertainty visualization. In: Scientific Visualization. Springer, pp 3–27
Bordoloi UD, Kao DL, Shen HW (2004) Visualization techniques for spatial probability density function data. Data Sci J 3:153–162
Borland D, Ii RMT (2007) Rainbow color map (still) considered harmful. IEEE Comput Graph Appl 27(2)
Botchen RP, Weiskopf D, Ertl T (2005) Texture-based visualization of uncertainty in flow fields. In: VIS 05. IEEE visualization 2005. IEEE, pp 647–654
Boukhelifa N, Duke DJ (2009) Uncertainty visualization: why might it fail? In: CHI’09 extended abstracts on human factors in computing systems. ACM, pp 4051–4056
Brodlie K, Osorio RA, Lopes A (2012) A review of uncertainty in data visualization. In: Expanding the frontiers of visual analytics and visualization. Springer, pp 81–109
Brodlie KW, Carpenter L, Earnshaw R, Gallop JR, Hubbold RJ, Mumford A, Osland C, Quarendon P (1992) Scientific visualization: techniques and applications
Brown R (2004) Animated visual vibrations as an uncertainty visualisation technique. In: Proceedings of the 2nd international conference on computer graphics and interactive techniques in Australasia and South East Asia. ACM, pp 84–89
Bruckner S, Moller T (2010) Result-driven exploration of simulation parameter spaces for visual effects design. IEEE Trans Vis Comput Graph 16(6):1468–1476
Bruls M, Huizing K, Van Wijk JJ (2000) Squarified treemaps. In: Data visualization 2000. Springer, pp 33–42
Buttenfield B, Beard MK (1994) Graphical and geographical components of data quality. Vis Geograph Inform Syst 150–157
Buttenfield BP, Ganter JH (1990) Visualization and gis: what should we see? what might we miss. In: Proceedings of the 4th international symposium on spatial data handling, vol 1. pp 307–316
Cai W, Sakas G (1999) Data intermixing and multi-volume rendering. Comput Graph Forum 18:359–368
Cedilnik A, Rheingans P (2000) Procedural annotation of uncertain information. In: Proceedings of the conference on Visualization’00. IEEE Computer Society Press, pp 77–83
Chambers JM (2018) Graphical methods for data analysis. CRC Press
Chlan EB, Rheingans P, (2005) Multivariate glyphs for multi-object clusters. In: IEEE symposium on information visualization, INFOVIS 2005. IEEE, pp 141–148
Cho S, Lee G, Choi J (2020) Interpretation of deep temporal representations by selective visualization of internally activated units. arXiv preprint arXiv:200412538
Choonpradub C, McNeil D (2005) Can the box plot be improved. Songklanakarin J Sci Technol 27(3):649–657
Cohen DJ, Cohen J (2006) The sectioned density plot. Am Stat 60(2):167–174
Coninx A, Bonneau GP, Droulez J, Thibault G (2011) Visualization of uncertain scalar data fields using color scales and perceptually adapted noise. In: Proceedings of the ACM SIGGRAPH symposium on applied perception in graphics and visualization, pp 59–66
Correa CD, Chan YH, Ma KL, (2009) A framework for uncertainty-aware visual analytics. In: IEEE symposium on visual analytics science and technology, VAST 2009. IEEE, pp 51–58
Couclelis H (2003) The certainty of uncertainty: Gis and the limits of geographic knowledge. Trans GIS 7(2):165–175
Crosetto M, Ruiz JAM, Crippa B (2001) Uncertainty propagation in models driven by remotely sensed data. Remote Sens Environ 76(3):373–385
Deitrick S, Edsall R (2006) The influence of uncertainty visualization on decision making: an empirical evaluation. In: Progress in spatial data handling. Springer, pp 719–738
Deitrick SA (2007) Uncertainty visualization and decision making: does visualizing uncertain information change decisions. In: Proceedings of the XXIII international cartographic conference, pp 4–10
Demir I, Dick C, Westermann R (2014) Multi-charts for comparative 3d ensemble visualization. IEEE Trans Vis Comput Graph 20(12):2694–2703
Demir I, Kehrer J, Westermann R (2016) Screen-space silhouettes for visualizing ensembles of 3d isosurfaces. In: 2016 IEEE pacific visualization symposium (PacificVis). IEEE, pp 204–208
Dempster AP (1968) A generalization of bayesian inference. J Roy Stat Soc Ser B 30(2):205–232
Diepenbrock S, Praßni JS, Lindemann F, Bothe HW, Ropinski T (2011) Interactive visualization techniques for neurosurgery planning. In: Eurographics, 2011, the 32th annual conference of the European association for computer graphics, 11–15 April 2011. Llandudno, Wales, UK
Diggle P, Diggle PJ, Heagerty P, Liang KY, Heagerty PJ, Zeger S et al (2002) Analysis of longitudinal data. Oxford University Press
Djurcilov S, Kim K, Lermusiaux PF, Pang A (2001) Volume rendering data with uncertainty information. In: Data visualization 2001. Springer, pp 243–252
Duckham M, Mason K, Stell J, Worboys M (2001) A formal approach to imperfection in geographic information. Comput Environ Urban Syst 25(1):89–103
Dungan JL, Kao D, Pang A (2002) The uncertainty visualization problem in remote sensing analysis. In: 2002 IEEE international geoscience and remote sensing symposium, 2002. IGARSS’02, vol 2. IEEE, pp 729–731
Dungan JL, Kao DL, Pang A (2003) Modeling and visualizing uncertainty in continuous variables predicted using remotely sensed data. In: 2003 IEEE international geoscience and remote sensing symposium, 2003. IGARSS’03. Proceedings, vol 5. IEEE, pp 3017–3019
Elliott R, Rees G, Dolan RJ (1999) Ventromedial prefrontal cortex mediates guessing. Neuropsychologia 37(4):403–411
Evenden J, Robbins T (1983) Dissociable effects of d-amphetamine, chlordiazepoxide and \(\alpha \)-flupenthixol on choice and rate measures of reinforcement in the rat. Psychopharmacology 79(2–3):180–186
Fekete JD (2013) Visual analytics infrastructures: from data management to exploration. Computer 46(7):22–29
Fernandes M, Walls L, Munson S, Hullman J, Kay M, et al. (2018) Uncertainty displays using quantile dotplots or cdfs improve transit decision-making
Ferstl F, Bürger K, Westermann R (2015) Streamline variability plots for characterizing the uncertainty in vector field ensembles. IEEE Trans Vis Comput Graph 22(1):767–776
Ferstl F, Kanzler M, Rautenhaus M, Westermann R (2016a) Time-hierarchical clustering and visualization of weather forecast ensembles. IEEE Trans Vis Comput Graph 23(1):831–840
Ferstl F, Kanzler M, Rautenhaus M, Westermann R (2016b) Visual analysis of spatial variability and global correlations in ensembles of iso-contours. Comput Graph Forum 35:221–230
Fofonov A, Molchanov V, Linsen L (2015) Visual analysis of multi-run spatio-temporal simulations using isocontour similarity for projected views. IEEE Trans Vis Comput Graph 22(8):2037–2050
Foody GM, Atkinson PM (2003) Uncertainty in remote sensing and GIS. John Wiley & Sons
Frigge M, Hoaglin DC, Iglewicz B (1989) Some implementations of the boxplot. Am Stat 43(1):50–54
Fua YH, Ward MO, Rundensteiner EA (1999) Hierarchical parallel coordinates for exploration of large datasets. IEEE
Fuchs R, Hauser H (2009) Visualization of multi-variate scientific data. Comput Graph Forum 28:1670–1690
Gahegan M, Ehlers M (2000) A framework for the modelling of uncertainty between remote sensing and geographic information systems. ISPRS J Photogrammet Remote Sens 55(3):176–188
Gebhardt N (2003) Einige brdf modelle. ttp://www irrlicht3d org/papers/BrdfModelle pdf
Gershon N (1998) Visualization of an imperfect world. IEEE Comput Graph Appl 18(4):43–45
Gershon ND (1992) Visualization of fuzzy data using generalized animation. In: IEEE conference on visualization 1992. Visualization’92, Proceedings. IEEE, pp 268–273
Gleicher M, Albers D, Walker R, Jusufi I, Hansen CD, Roberts JC (2011) Visual comparison for information visualization. Inform Vis 10(4):289–309
Görtler J, Schulz C, Weiskopf D, Deussen O (2018) Bubble treemaps for uncertainty visualization. IEEE Trans Vis Comput Graph 24(1):719–728
Goubergrits L, Hellmeier F, Bruening J, Spuler A, Hege HC, Voss S, Janiga G, Saalfeld S, Beuing O, Berg P (2019) Multiple aneurysms anatomy challenge 2018 (match): uncertainty quantification of geometric rupture risk parameters. Biomed Eng 18(1):35
Griethe H, Schumann H et al (2006) The visualization of uncertain data: Methods and problems. In: SimVis, pp 143–156
Grigoryan G, Rheingans P (2002) Probabilistic surfaces: Point based primitives to show surface uncertainty. In: Proceedings of the conference on visualization’02. IEEE Computer Society, pp 147–154
Grigoryan G, Rheingans P (2004) Point-based probabilistic surfaces to show surface uncertainty. IEEE Trans Vis Comput Graph 10(5):564–573
Haber RB, McNabb DA (1990) Visualization idioms: a conceptual model for scientific visualization systems. Vis Sci Comput 74:93
Haemer KW (1948) Range-bar charts. Am Stat 2(2):23
Hagh-Shenas H, Kim S, Interrante V, Healey C (2007) Weaving versus blending: a quantitative assessment of the information carrying capacities of two alternative methods for conveying multivariate data with color. IEEE Trans Vis Comput Graph 13(6):1270–1277
Hao L, Healey CG, Bass SA (2015) Effective visualization of temporal ensembles. IEEE Trans Vis Comput Graph 22(1):787–796
Haroz S, Ma KL, Heitmann K (2008) Multiple uncertainties in time-variant cosmological particle data. In: 2008 IEEE pacific visualization symposium. IEEE, pp 207–214
Hazarika S, Biswas A, Shen HW (2017) Uncertainty visualization using copula-based analysis in mixed distribution models. IEEE Trans Vis Comput Graph 24(1):934–943
Hearnshaw HM, Unwin DJ (1994) Visualization in geographical information systems
Hibbard B, Böttinger M, Schultz M, Biercamp J (2002) Visualization in earth system science. Acm Siggraph Comput Graph 36(4):5–9
Hintze JL, Nelson RD (1998) Violin plots: a box plot-density trace synergism. Am Stat 52(2):181–184
Hlawatsch M, Leube P, Nowak W, Weiskopf D (2011) Flow radar glyphs-static visualization of unsteady flow with uncertainty. IEEE Trans Vis Comput Graph 17(12):1949–1958
Hodgett RE, Siraj S (2019) Sure: a method for decision-making under uncertainty. Exp Syst Appl 115:684–694
Holliman NS, Coltekin A, Fernstad SJ, Simpson MD, Wilson KJ, Woods AJ (2019) Visual entropy and the visualization of uncertainty. arXiv preprint arXiv:190712879
Hollister BE, Pang A (2015) Bivariate quantile interpolation for ensemble derived probability density estimates. Int J Uncertain Quantif 5(2):123–137. https://doi.org/10.1615/Int.J.UncertaintyQuantification.2015011789
Höllt T, Magdy A, Zhan P, Chen G, Gopalakrishnan G, Hoteit I, Hansen CD, Hadwiger M (2014) Ovis: a framework for visual analysisof ocean forecast ensembles. IEEE Trans Vis Comput Graph 20(8):1114–1126
Hullman J (2019) Why authors don’t visualize uncertainty. IEEE Trans Vis Comput Graph 26(1):130–139
Hullman J, Qiao X, Correll M, Kale A, Kay M (2018) In pursuit of error: a survey of uncertainty visualization evaluation. IEEE Trans Vis Comput Graph 25(1):903–913
Hunter GJ, Goodchild M (1993) Managing uncertainty in spatial databases: putting theory into practice. In: Papers from the annual conference-urban and regional information systems association. Urisa Urban and Regional Information Systems, p 15
Interrante V (2000) Harnessing natural textures for multivariate visualization. IEEE Comput Graph Appl 20(6):6–11
Jää-Aro K (2006) Lecture notes: visualisation of uncertainty. KTH Stockholm
Jarema M, Kehrer J, Westermann R (2016) Comparative visual analysis of transport variability in flow ensembles
Johnson B, Shneiderman B (1991) Tree-maps: a space-filling approach to the visualization of hierarchical information structures. In: Proceedings of the 2nd conference on Visualization’91. IEEE Computer Society Press, pp 284–291
Johnson C (2004) Top scientific visualization research problems. IEEE Comput Graph Appl 24(4):13–17
Johnson CR, Sanderson AR (2003) A next step: visualizing errors and uncertainty. IEEE Comput Graph Appl 23(5):6–10
Jones DK (2003) Determining and visualizing uncertainty in estimates of fiber orientation from diffusion tensor mri. Magnet Reson Med 49(1):7–12
Joseph AJ, Lodha S, Renteria J, Pang A (1998) Uisurf: visualizing uncertainty in isosurfaces. Master’s thesis, University of California, Santa Cruz
Kao D, Dungan JL, Pang A (2001) Visualizing 2d probability distributions from eos satellite image-derived data sets: a case study. In: Visualization, 2001, VIS’01. Proceedings. IEEE, pp 457–589
Kao D, Luo A, Dungan JL, Pang A (2002) Visualizing spatially varying distribution data. In: Sixth international conference on information visualisation, 2002. Proceedings. IEEE, pp 219–225
Kao DL, Kramer MG, Love AL, Dungan JL, Pang AT (2005) Visualizing distributions from multi-return lidar data to understand forest structure. Cartograph J 42(1):35–47
Karami A (2015) A framework for uncertainty-aware visual analytics in big data. In: AIC, pp 146–155
Khan M, Xu L, Nandi A, Hellerstein JM (2017) Data tweening: incremental visualization of data transforms. Proc VLDB Endowm 10(6):661–672
Kim YS, Walls LA, Krafft P, Hullman J (2019) A bayesian cognition approach to improve data visualization. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1–14
Kniss JM, Van Uitert R, Stephens A, Li GS, Tasdizen T, Hansen C (2005) Statistically quantitative volume visualization. In: VIS 05. IEEE Visualization, 2005. IEEE, pp 287–294
Kolmogorov-Smirnov A, Kolmogorov A, Kolmogorov M (1933) Sulla determinazione empírica di uma legge di distribuzione
Li H, Fu CW, Li Y, Hanson A (2007) Visualizing large-scale uncertainty in astrophysical data. IEEE Trans Vis Comput Graph 13(6):1640–1647
Li W, Lang J, Zhang H, Yang F, Zhang L, Pan J (2019) Parallel coordinates based visualization for high-dimensional data. In: Proceedings of the 2019 3rd international conference on big data research, pp 161–165
Liu M, Shi J, Li Z, Li C, Zhu J, Liu S (2017a) Towards better analysis of deep convolutional neural networks. IEEE Trans Vis Comput Graph 23(1):91–100
Liu S, Levine JA, Bremer PT, Pascucci V (2012) Gaussian mixture model based volume visualization. In: IEEE symposium on large data analysis and visualization (LDAV). IEEE, pp 73–77
Liu S, Wang X, Liu M, Zhu J (2017b) Towards better analysis of machine learning models: a visual analytics perspective. Vis Inform 1(1):48–56
Lodha SK, Pang A, Sheehan RE, Wittenbrink CM (1996a) Uflow: visualizing uncertainty in fluid flow. In: Proceedings of the 7th conference on Visualization’96. IEEE Computer Society Press, pp 249–ff
Lodha SK, Sheehan B, Pang AT, Wittenbrink CM et al (1996b) Visualizing geometric uncertainty of surface interpolants. Graph Interfa 96:238–245
Luo A, Kao D, Pang A (2003) Visualizing spatial distribution data sets. VisSym 3:29–38
MacEachren AM (1992) Visualizing uncertain information. Cartograph Perspect 13:10–19
MacEachren AM, Robinson A, Hopper S, Gardner S, Murray R, Gahegan M, Hetzler E (2005) Visualizing geospatial information uncertainty: what we know and what we need to know. Cartograph Geograph Inform Sci 32(3):139–160
MacEachren AM, Roth RE, O’Brien J, Li B, Swingley D, Gahegan M (2012) Visual semiotics & uncertainty visualization: an empirical study. IEEE Trans Vis Comput Graph 18(12):2496–2505
Malik MM, Heinzl C, Groeller ME (2010) Comparative visualization for parameter studies of dataset series. IEEE Trans Vis Comput Graph 16(5):829–840
Masalonis A, Mulgund S, Song L, Wanke C, Zobell S (2004) Using probabilistic demand predictions for traffic flow management decision support. In: AIAA guidance, navigation, and control conference and exhibit, p 5231
Matkovic K, Gracanin D, Klarin B, Hauser H (2009) Interactive visual analysis of complex scientific data as families of data surfaces. IEEE Trans Vis Comput Graph 15(6):1351–1358
Mirzargar M, Whitaker RT, Kirby RM (2014) Curve boxplot: generalization of boxplot for ensembles of curves. IEEE Trans Vis Comput Graph 20(12):2654–2663
Mittal A, Belyaev V, Fernandez SSJ, Rubin G, Mascarenhas A, Lotia S, Raj A, Fuller J, Chowdhury S (2019) System and method for data visualization using machine learning and automatic insight of outliers associated with a set of data. US Patent App. 16/148,680
Mühlbacher T, Piringer H, Gratzl S, Sedlmair M, Streit M (2014) Opening the black box: strategies for increased user involvement in existing algorithm implementations. IEEE Trans Vis Comput Graph 20(12):1643–1652
Newman TS, Lee W (2004) On visualizing uncertainty in volumetric data: techniques and their evaluation. J Vis Lang Comput 15(6):463–491
Nguyen Q, Eades P, Hong SH (2012) On the faithfulness of graph visualizations. In: International symposium on graph drawing, Springer, pp 566–568
Nguyen QH, Eades P (2017) Towards faithful graph visualizations. arXiv preprint arXiv:170100921
Obermaier H, Bensema K, Joy KI (2015) Visual trends analysis in time-varying ensembles. IEEE Trans Vis Comput Graph 22(10):2331–2342
Olston C, Mackinlay JD, (2002) Visualizing data with bounded uncertainty. In: IEEE symposium on information visualization, 2002, INFOVIS 2002. IEEE, pp 37–40
Otto M, Theisel H (2012) Vortex analysis in uncertain vector fields. Comput Graph Forum 31:1035–1044
Pagendarm HG, Post FH (1995) Comparative visualization: approaches and examples. Delft University of Technology, Faculty of Technical Mathematics and Informatics
Pang A, Furman JJ, Nuss W (1994) Data quality issues in visualization. In: IS&T/SPIE 1994 international symposium on electronic imaging: science and technology. International Society for Optics and Photonics, pp 12–23
Pang AT, Wittenbrink CM, Lodha SK (1997) Approaches to uncertainty visualization. Vis Comput 13(8):370–390
Parsons S, Hunter A (1998) A review of uncertainty handling formalisms. In: Applications of uncertainty formalisms. Springer, pp 8–37
Pauly M, Mitra NJ, Guibas LJ (2004) Uncertainty and variability in point cloud surface data. SPBG 4:77–84
Pfaffelmoser T, Westermann R (2012) Visualization of global correlation structures in uncertain 2d scalar fields. Comput Graph Forum 31:1025–1034
Pfaffelmoser T, Westermann R (2013) Visualizing contour distributions in 2d ensemble data. In: EuroVis (short Papers)
Piringer H, Pajer S, Berger W, Teichmann H (2012) Comparative visual analysis of 2d function ensembles. Comput Graph Forum 31:1195–1204
Pöthkow K, Hege HC (2013) Nonparametric models for uncertainty visualization. Comput Graph Forum 32:131–140
Pöthkow K, Weber B, Hege HC (2011) Probabilistic marching cubes. Comput Graph Forum 30:931–940
Potter K (2010) The visualization of uncertainty. The University of Utah
Potter K, Hagen H, Kerren A, Dannenmann P (2006) Methods for presenting statistical information: the box plot. Vis Large Unstruct Data Sets 4:97–106
Potter K, Wilson A, Bremer PT, Williams D, Doutriaux C, Pascucci V, Johnson CR (2009) Ensemble-vis: a framework for the statistical visualization of ensemble data. In: IEEE international conference on data mining workshops, 2009. ICDMW’09. IEEE, pp 233–240
Potter K, Kniss J, Riesenfeld R, Johnson CR (2010) Visualizing summary statistics and uncertainty. Comput Graph Forum 29:823–832
Potter K, Rosen P, Johnson C (2012) From quantification to visualization: a taxonomy of uncertainty visualization approaches. Uncertain Quantif Sci Comput 226–249
Prassni JS, Ropinski T, Hinrichs K (2010) Uncertainty-aware guided volume segmentation. IEEE Trans Vis Comput Graph 16(6):1358–1365
Raglin A, Dennison M, Metu S, Trout T, James D (2020) Decision making with uncertainty in immersive systems. In: Virtual, augmented, and mixed reality (XR) technology for multi-domain operations, vol 11426. International Society for Optics and Photonics, p 114260L
Ren K, Qu D, Xu S, Jiao X, Tai L, Zhang H (2020) Uncertainty visualization of transport variance in a time-varying ensemble vector field. ISPRS Int J Geo-Inform 9(1):19
Rheingans P, Brown W, Morrow A, Stull D, Winner K, et al. (2014) Visualizing uncertainty in predictive models. In: Scientific visualization. Springer, pp 61–69
Rhodes PJ, Laramee RS, Bergeron RD, Sparr TM et al (2003) Uncertainty visualization methods in isosurface rendering. Eurographics 2003:83–88
Riveiro M (2007) Evaluation of uncertainty visualization techniques for information fusion. In: 10th international conference on information fusion, 2007. IEEE, pp 1–8
Sacha D, Senaratne H, Kwon BC, Ellis G, Keim DA (2016) The role of uncertainty, awareness, and trust in visual analytics. IEEE Trans Vis Comput Graph 22(1):240–249
Sanderson AR, Johnson CR, Kirby RM (2004) Display of vector fields using a reaction-diffusion model. In: Proceedings of the conference on visualization’04. IEEE Computer Society, pp 115–122
Sanyal J, Zhang S, Bhattacharya G, Amburn P, Moorhead R (2009) A user study to compare four uncertainty visualization methods for 1d and 2d datasets. IEEE Trans Vis Comput Graph 15(6):1209–1218
Sanyal J, Zhang S, Dyer J, Mercer A, Amburn P, Moorhead R (2010) Noodles: a tool for visualization of numerical weather model ensemble uncertainty. IEEE Trans Vis Comput Graph 16(6):1421–1430
Schmidt GS, Chen SL, Bryden AN, Livingston MA, Rosenblum LJ, Osborn BR (2004) Multidimensional visual representations for underwater environmental uncertainty. IEEE Comput Graph Appl 24(5):56–65
Schulz C, Nocaj A, Goertler J, Deussen O, Brandes U, Weiskopf D (2017) Probabilistic graph layout for uncertain network visualization. IEEE Trans Vis Comput Graph 23(1):531–540
Schulz HJ (2011) Treevis. net: a tree visualization reference. IEEE Comput Graph Appl 31(6):11–15
Schulz HJ, Hadlak S, Schumann H (2011) The design space of implicit hierarchy visualization: a survey. IEEE Trans Vis Comput Graph 17(4):393–411
Sedlmair M, Meyer M, Munzner T (2012) Design study methodology: reflections from the trenches and the stacks. IEEE Trans Vis Comput Graph 18(12):2431–2440
Shafer G (1976) A mathematical theory of evidence, vol 42. Princeton University Press
Shafer G (1992) Dempster-shafer theory. Encyclop Artif Intell 1:330–331
Shen L, Hsee CK, Talloen JH (2019) The fun and function of uncertainty: uncertain incentives reinforce repetition decisions. J Consum Res 46(1):69–81
Shu Q, Guo H, Liang J, Che L, Liu J, Yuan X (2016) Ensemblegraph: interactive visual analysis of spatiotemporal behaviors in ensemble simulation data. In: 2016 IEEE pacific visualization symposium (PacificVis). IEEE, pp 56–63
Sinton D (1978) The inherent structure of information as a constraint to analysis: mapped thematic data as a case study. Harvard Papers Geograph Inform Syst 6:1–17
Skeels M, Lee B, Smith G, Robertson GG (2010) Revealing uncertainty for information visualization. Inform Vis 9(1):70–81
Slingsby A, Dykes J, Wood J (2011) Exploring uncertainty in geodemographics with interactive graphics. IEEE Trans Vis Comput Graph 17(12):2545–2554
Smithson M (1989) Ignorance and uncertainty: emerging paradigms
Spear ME (1952) Charting statistics
Stokking R, Zubal IG, Viergever MA (2003) Display of fused images: methods, interpretation, and diagnostic improvements. Semin Nucl Med 33:219–227
Strothotte T, Masuch M, Isenberg T (1999) Visualizing knowledge about virtual reconstructions of ancient architecture. In: cgi. IEEE, p 36
Thompson D, Levine JA, Bennett JC, Bremer PT, Gyulassy A, Pascucci V, Pébay PP (2011) Analysis of large-scale scalar data using hixels. In: 2011 IEEE symposium on large data analysis and visualization. IEEE, pp 23–30
Thomson J, Hetzler E, MacEachren A, Gahegan M, Pavel M (2005) A typology for visualizing uncertainty pp 146–157
Tufte ER (1985) The visual display of quantitative information. J Healthc Qual 7(3):15
Tufte ER (2006) Envisioning information, 1990. Visual explanations: images and quan
Tufte ER, McKay SR, Christian W, Matey JR (1998) Visual explanations: images and quantities, evidence and narrative
Tukey JW (1977) Exploratory data analysis
Urness T, Interrante V, Marusic I, Longmire E, Ganapathisubramani B (2003) Effectively visualizing multi-valued flow data using color and texture. In: Proceedings of the 14th IEEE visualization 2003 (VIS’03). IEEE Computer Society, p 16
Wang J, Liu X, Shen HW, Lin G (2016a) Multi-resolution climate ensemble parameter analysis with nested parallel coordinates plots. IEEE Trans Vis Comput Graph 23(1):81–90
Wang J, Hazarika S, Li C, Shen HW (2018) Visualization and visual analysis of ensemble data: a survey. IEEE Trans Vis Comput Graph 25(9):2853–2872
Wang X, Liu S, Liu J, Chen J, Zhu J, Guo B (2016b) Topicpanorama: a full picture of relevant topics. IEEE Trans Vis Comput Graph 22(12):2508–2521
Weatherston J, Perin C, Hore D, Wallace B, Storey MA (2020) An unquantified uncertainty visualization design space during the opioid crisis. In: Extended abstracts of the 2020 CHI conference on human factors in computing systems. Extended Abstracts, pp 1–8
Whitaker RT, Mirzargar M, Kirby RM (2013) Contour boxplots: a method for characterizing uncertainty in feature sets from simulation ensembles. IEEE Trans Vis Comput Graph 19(12):2713–2722
Wilkinson L (1999) Dot plots. Am Stat 53(3):276–281
Wilkinson L (2012) The grammar of graphics. In: Handbook of computational statistics. Springer, pp 375–414
Windhager F, Salisu S, Mayr E (2019) Exhibiting uncertainty: Visualizing data quality indicators for cultural collections. In: Informatics, multidisciplinary digital publishing institute, vol 6, p 29
Wittenbrink CM, Pang AT, Lodha SK (1996) Glyphs for visualizing uncertainty in vector fields. IEEE Trans Vis Comput Graph 2(3):266–279
Wu Y, Wei F, Liu S, Au N, Cui W, Zhou H, Qu H (2010) Opinionseer: interactive visualization of hotel customer feedback. IEEE Trans Vis Comput Graph 16(6):1109–1118
Wu Y, Yuan GX, Ma KL (2012) Visualizing flow of uncertainty through analytical processes. IEEE Trans Vis Comput Graph 18(12):2526–2535
Xie Z, Huang S, Ward MO, Rundensteiner EA (2006) Exploratory visualization of multivariate data with variable quality. In: IEEE symposium on visual analytics science and technology. IEEE, pp 183–190
Yager RR (2019) Decision-making with measure modeled uncertain payoffs and multiple goals. Granul Comput 1–6
Yan L, Wang Y, Munch E, Gasparovic E, Wang B (2019) A structural average of labeled merge trees for uncertainty visualization. IEEE Trans Vis Comput Graph 26(1):832–842
Yang L, Hyde D, Grujic O, Scheidt C, Caers J (2019) Assessing and visualizing uncertainty of 3d geological surfaces using level sets with stochastic motion. Comput Geosci 122:54–67
Zadeh LA (1996) Fuzzy sets. In: Zadeh LA (ed) Fuzzy sets, fuzzy logic. And fuzzy systems, selected papers by. World Scientific, pp 394–432
Zadeh LA (1999) Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst 100:9–34
Zehner B, Watanabe N, Kolditz O (2010) Visualization of gridded scalar data with uncertainty in geosciences. Comput Geosci 36(10):1268–1275
Zhang H, Qu D, Liu Q, Shang Q, Hou Y, Shen HW (2018) Uncertainty visualization for variable associations analysis. Vis Comput 34(4):531–549
Zhao H, Lu L (2015) (2015) Variational circular treemaps for interactive visualization of hierarchical data. In: Visualization symposium (PacificVis). IEEE Pacific. IEEE, pp 81–85
Ziemkiewicz C, Kosara R (2008) The shaping of information by visual metaphors. IEEE Trans Vis Comput Graph 14(6):1269–1276
Zuk T, Carpendale S (2006) Theoretical analysis of uncertainty visualizations. In: Electronic imaging 2006. International Society for Optics and Photonics, p 606007
Zuk T, Carpendale S (2007) Visualization of uncertainty and reasoning. In: Smart graphics. Springer, pp 164–177
Zuk T, Carpendale S, Glanzman WD (2005) Visualizing temporal uncertainty in 3d virtual reconstructions. In: VAST, vol 2005, p 6
Zuk T, Downton J, Gray D, Carpendale S, Liang J (2008) Exploration of uncertainty in bidirectional vector fields. In: Visualization and data analysis 2008, vol 6809. International Society for Optics and Photonics, p 68090B
Zukab T, Downtonb J, Grayb D, Carpendalea S, Liangb J (2008) Exploration of uncertainty in bidirectional vector fields. In: Proceedings of SPIE
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This research is supported by the Air Force Research Laboratory, under grant number FA8650-16-C-6768, “EpEx:Episodic Memory Reconstruction for UAS Behavior Explanation”
Rights and permissions
About this article
Cite this article
Kamal, A., Dhakal, P., Javaid, A.Y. et al. Recent advances and challenges in uncertainty visualization: a survey. J Vis 24, 861–890 (2021). https://doi.org/10.1007/s12650-021-00755-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12650-021-00755-1