Skip to main content
SpringerLink
Log in

Photo4Action: phone camera-based interaction for graph visualizations on large wall displays

  • Regular Paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

We present Photo4Action, a system that integrates mobile devices and large wall display devices for efficient network visualization. Our work allows users to locate a region of a large graph visualization on a wall display by photo taking. An efficient match algorithm is proposed to match the subgraph users select with the full graph. Sets of interaction techniques have been designed to support various graph exploration and visualization tasks. The system also supports interactions for multi-users’ collaborative analysis for graphs. We also report the interaction patterns and collaboration behaviors of users during the visual exploration as observed during our study

Graphic abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Ardito C, Buono P, Costabile MF, Desolda G (2015) Interaction with large displays: A survey. ACM Compu Surv, 47(3):46:38

  • Badam S.K, Amini F, Elmqvist N, Irani P. (2016) Supporting visual exploration for multiple users in large display environments. In Proceedings of IEEE Conference on Visual Analytics Science and Technology, pages 1–10

  • Ball R,North C. Bowman D.A (2007) Move to improve: promoting physical navigation to increase user performance with large displays. In Proceedings of ACM Conference on Human Factors in Computing Systems, pages 191–200, 2007

  • Ballagas R, Rohs M, Sheridan J G (2005) Sweep and point and shoot: phonecam-based interactions for large public displays. In Proceedings of CHI’05 extended abstracts on Human factors in computing systems, pages 1200–1203, 2005

  • Bezerianos A, Isenberg P (2012) Perception of visual variables on tiled wall-sized displays for information visualization applications. IEEE Transact Visualizat Comp Graphics 18(12):2516–2525

    Article  Google Scholar 

  • Boring S, Baur D, Butz A, Gustafson S, Baudisch P (2010) Touch projector: mobile interaction through video. In Proceedings of the 28th International Conference on Human Factors in Computing Systems, pages 2287–2296

  • Bostock M, Ogievetsky V, Heer J (2011) D\({^3}\) data-driven documents. IEEE Transact Visualizat Comp Graphics 17(12):2301–2309

    Article  Google Scholar 

  • Chang T.H, Li Y (2011) Deep shot: a framework for migrating tasks across devices using mobile phone cameras. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pages 2163–2172

  • Chegini, M, Andrews K, Schreck T, Sourin A. (2019) Multiple linked-view exploration on large displays facilitated by a secondary handheld device. In Proceedings of International Workshop on Advanced Image Technology (IWAIT) 2019, volume 11049, page 110490H

  • Cheng K, Li , Müller-Tomfelde C. (2012) Supporting interaction and collaboration on large displays using tablet devices. In Proceedings of the International Working Conference on Advanced Visual Interfaces, pages 774–775

  • Dachselt R, Buchholz R (2009) Natural throw and tilt interaction between mobile phones and distant displays. In Proceedings of CHI’09 extended abstracts on Human factors in computing systems, pages 3253–3258

  • Gustafsson MG (2000) Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J microscop 198(2):82–87

    Article  Google Scholar 

  • Horak, T, Badam S.K., Elmqvist N, Dachselt R. (2018) When david meets goliath: Combining smartwatches with a large vertical display for visual data exploration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pages 19:1–19:13

  • Ioannou D, Huda W, Laine AF (1999) Circle recognition through a 2d hough transform and radius histogramming. Image Vis Comput 17(1):15–26

    Article  Google Scholar 

  • Jansen Y, Dragicevic P, Fekete J.-D.(2012) Tangible remote controllers for wall-size displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2865–2874

  • Kister U, Reipschläger P, Matulic F, Dachselt R (2015) Bodylenses: Embodied magic lenses and personal territories for wall displays. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces, pages 117–126

  • Kister U, Klamka K, Tominski C, Dachselt R (2017) Grasp: combining spatially-aware mobile devices and a display wall for graph visualization and interaction. Comput Graphic Forum 36(3):503–514

    Article  Google Scholar 

  • Koutsabasis P, Vogiatzidakis P (2019) Empirical research in mid-air interaction: a systematic review. Int J Human-Comput Interact 35(18):1747–1768

    Article  Google Scholar 

  • Langner R, Dachselt R (2018) Towards visual data exploration at wall-sized displays by combining physical navigation with spatially-aware devices. IEEE VIS 2018 Poster Program

  • Langner R, von Zadow U, Horak T, Mitschick A, Dachselt R (2016) Content sharing between spatially-aware mobile phones and large vertical displays supporting collaborative work. Collab Meets Interact Spaces. https://doi.org/10.1007/978-3-319-45853-3_5

    Article  Google Scholar 

  • Lee B, Plaisant C, Parr C.S, Fekete J.-D, Henry N (2006) Task taxonomy for graph visualization. In Proceedings of the 2006 AVI workshop on BEyond time and errors: novel evaluation methods for information visualization, pages 1–5

  • Lehmann A, Schumann H, Staadt O, Tominski C (2011) Physical navigation to support graph exploration on a large high-resolution display. In Proceedings of International Symposium on Visual Computing, pages 496–507

  • Marner M.R, Smith R.T, Thomas B.H, Klein K. Eades P, HongS.-H (2014) Gion: interactively untangling large graphs on wall-sized displays. In Proceedings of International Symposium on Graph Drawing, pages 113–124

  • Moscovich T, Chevalier F, Henry N, Pietriga E, Fekete J.-D (2009) Topology-aware navigation in large networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 2319–2328

  • Nancel M, Chapuis O, Pietriga E, Yang X.D, Irani P.P., Lafon M.B (2013) High-precision pointing on large wall displays using small handheld devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 831–840

  • Nancel M, Wagner J, Pietriga E, Chapuis O, Mackay W.E. (2011) Mid-air pan-and-zoom on wall-sized displays. In Proceedings of the International Conference on Human Factors in Computing Systems, pages 177–186

  • Olwal A, Feiner S, Heyman S (2008) Rubbing and tapping for precise and rapid selection on touch-screen displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 295–304

  • Olwal A, Frykholm O, Groth K, Moll J (2011) Design and evaluation of interaction technology for medical team meetings. In Proceedings of the 13 International Conference on Human Computer Interaction, pages 505–522

  • Otsu N (2007) A threshold selection method from gray-level histograms. IEEE Transact Syst Man Cybernetics 9(1):62–66

    Article  MathSciNet  Google Scholar 

  • Prouzeau A, Bezerianos A, Chapuis O (2017) Evaluating multi-user selection for exploring graph topology on wall-displays. IEEE Trans Visual Comput Graphics 23(8):1936–1951

    Article  Google Scholar 

  • Reda K, Johnson A.E, Papka M.E, Leigh J (2015) Effects of display size and resolution on user behavior and insight acquisition in visual exploration. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 2759–2768

  • Song P, Goh W.B, Fu C.W, Meng Q, Heng P (2011) WYSIWYF: exploring and annotating volume data with a tangible handheld device. In Proceedings of the International Conference on Human Factors in Computing Systems, pages 1333–1342

  • Spindler M, Tominski C, Schumann H, Dachselt R (2010) Tangible views for information visualization. In Proceedings of ACM International Conference on Interactive Tabletops and Surfaces, pages 157–166. ACM

  • Tsandilas T, Bezerianos A, Jacob T (2015) Sketchsliders: Sketching widgets for visual exploration on wall displays. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 3255–3264. ACM

  • van Wijk J.J (2005) The value of visualization. In Proceedings of IEEE Visualization Conference, pages 79–86

  • Vogel D, Balakrishnan R (2005) Distant freehand pointing and clicking on very large, high resolution displays. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pages 33–42

  • Yi J.S, ahKang Y, Stasko J.T, Jacko J.A (2007) Toward a deeper understanding of the role of interaction in information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6):1224–1231

  • Yu L, Svetachov P, Isenberg P, Everts MH, Isenberg T (2010) FI3D: direct-touch interaction for the exploration of 3d scientific visualization spaces. IEEE Trans Visual Comput Graphics 16(6):1613–1622

    Article  Google Scholar 

  • Zitová B, Flusser J (2003) Image registration methods: a survey. Image Vision Comput 21(11):977–1000

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the anonymous reviewers for their valuable comments. This work is supported by National Numerical Windtunnel Project NNW2018-ZT6B12.

Author information

Authors and Affiliations

  1. Key Laboratory of Machine Perception (Ministry of Education), and School of EECS, Peking University, Beijing, China

    Shuai Chen, Hao Wu, Zhixian Lin, Cong Guo, Lijing Lin, Fan Hong & Xiaoru Yuan

  2. National Engineering Laboratory for Big Data Analysis and Application, Peking University, Beijing, China

    Xiaoru Yuan

Authors
  1. Shuai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhixian Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lijing Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fan Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaoru Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoru Yuan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Wu, H., Lin, Z. et al. Photo4Action: phone camera-based interaction for graph visualizations on large wall displays. J Vis 24, 1083–1095 (2021). https://doi.org/10.1007/s12650-021-00751-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12650-021-00751-5

Keywords

Search

Navigation