Skip to main content
SpringerLink
Log in

A new storytelling genre: combining handicraft elements and storytelling via mixed reality technology

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Handicraft is a traditional cultural form which can improve children’s hands-on ability and spatial imagination ability. In addition, storytelling is another beneficial activity that can improve children’s language skill and creative power in their growing process. By combining handicraft elements and storytelling via mixed reality technology, this paper presents a new storytelling genre, which supports children and parents learning how to make handicraft and then use them as tangible interaction tools to design, create, tell and share stories, together with gesture interaction. Taking origami as an example, we design and implement a system and conduct a user study. Findings illustrate that our system has good usability and can motivate users’ interest in handicraft and storytelling and even promote parent–child interaction effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Alofs, T., Theune, M., Swartjes, I.: A tabletop interactive storytelling system: designing for social interaction. IJART 8(3), 188–211 (2015)

    Article  Google Scholar 

  2. Bae, J.H.: The effects of origami on the improvement of hand dexterity. J. Int. Acad. Phys. Therapy Res. 4(2), 588–594 (2013)

    Article  Google Scholar 

  3. Bayon, V., Wilson, J.R., Stanton, D., Boltman, A.: Mixed reality storytelling environments. Virtual Reality 7(1), 54–63 (2003)

    Article  Google Scholar 

  4. Boakes, N.: Origami-mathematics lessons: Paper folding as a teaching tool. Mathidues 1(1), 1–9 (2008)

    Google Scholar 

  5. Boakes, N.J.: Origami instruction in the middle school mathematics classroom: Its impact on spatial visualization and geometry knowledge of students. RMLE Online 32(7), 1–12 (2009)

    Article  Google Scholar 

  6. Cao, X., Lindley, S.E., Helmes, J., Sellen, A.: Telling the whole story: anticipation, inspiration and reputation in a field deployment of telltable. In: Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, pp. 251–260 (2010)

  7. Chang, W.: Origami in education enhanced by computer technology: A case study of teaching hexaflexagon in math class. In: International Conference on Technology Enhanced Learning, pp. 170–175. Springer (2010)

  8. Chng, E.: New ways of accessing information spaces using 3d multitouch tables. In: 2012 International Conference on Cyberworlds, pp. 144–150. IEEE (2012)

  9. Davis, J., Leslie, R., Billington, S., Slater, P.R.: Origami: a versatile modeling system for visualising chemical structure and exploring molecular function. Chem. Edu. Res. Practice 11(1), 43–47 (2010)

    Article  Google Scholar 

  10. De Lima, E.S., Feijó, B., Barbosa, S.D., Furtado, A.L., Ciarlini, A.E., Pozzer, C.T.: Draw your own story: Paper and pencil interactive storytelling. Entertainment Comput. 5(1), 33–41 (2014)

    Article  Google Scholar 

  11. Harris, C.G., Stephens, M., et al.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, pp. 10–5244. Citeseer (1988)

  12. Isbell, R., Sobol, J., Lindauer, L., Lowrance, A.: The effects of storytelling and story reading on the oral language complexity and story comprehension of young children. Early Childhood Edu. J. 32(3), 157–163 (2004)

    Article  Google Scholar 

  13. Iyobe, M., Ishida, T., Miyakawa, A., Sugita, K., Uchida, N., Shibata, Y.: Proposal of a virtual traditional japanese crafting presentation system mobile edition. In: 2016 10th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS), pp. 120–125. IEEE (2016)

  14. Ji, Y., Tan, P.: Exploring personalized learning pattern for studying chinese traditional handicraft. In: Proceedings of the Sixth International Symposium of Chinese CHI, pp. 140–143 (2018)

  15. Kara, N., Aydin, C.C., Cagiltay, K.: Design and development of a smart storytelling toy. Interact. Learn. Environ. 22(3), 288–297 (2014)

    Article  Google Scholar 

  16. Mallan, K.M.: Children as storytellers. Primary English Teaching Association (1991)

  17. Mi, H., Krzywinski, A., Sugimoto, M., Chen, W.: Robostory: A tabletop mixed reality framework for children’s role play storytelling. In: Proceedings of the 1st International Workshop on Interactive Storytelling for Children (ACM IDC’10). Association for Computing Machinery (2010)

  18. Morgan, J., Rinvolucri, M., Ur, P.: Once Upon a Time: Using Stories in the Language Classroom. Cambridge University Press, Cambridge (1983)

    Google Scholar 

  19. Posada, J.E.G., Hornung, H., Martins, M.C., Baranauskas, M.C.C.: A tui-based storytelling for promoting inclusion in the preschool classroom. In: International Conference on Universal Access in Human-Computer Interaction, pp. 89–100. Springer (2015)

  20. Raffle, H., Ballagas, R., Revelle, G., Horii, H., Follmer, S., Go, J., Reardon, E., Mori, K., Kaye, J., Spasojevic, M.: Family story play: reading with young children (and elmo) over a distance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1583–1592 (2010)

  21. Raffle, H., Vaucelle, C., Wang, R., Ishii, H.: Jabberstamp: embedding sound and voice in traditional drawings. In: Proceedings of the 6th International Conference on Interaction Design and Children, pp. 137–144 (2007)

  22. Rizkyandy, R., et al.: Bbvphone: Video digital learning for handicraft. In: International Conference on Interactive Collaborative Learning, pp. 551–558. Springer (2017)

  23. Sauro, J., Lewis, J.R.: Quantifying the user experience: Practical statistics for user research. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

  24. Sin, N.S.M., Ali, Z.: Paper toy as a modern storytelling aid among children (education). Jurnal Teknologi 75(3), 81–85 (2015)

    Google Scholar 

  25. Stanton, D., O’Malley, C., Huing, K., Fraser, M., Benford, S.: Situating historical events through mixed reality. In: Designing for Change in Networked Learning Environments, pp. 293–302. Springer (2003)

  26. Sud, A.: Augmenting tangible interactions for storytelling

  27. Sugimoto, M.: A mobile mixed-reality environment for children’s storytelling using a handheld projector and a robot. IEEE Trans. Learn. Technol. 4(3), 249–260 (2011)

    Article  Google Scholar 

  28. Taylor, H.A., Hutton, A.: Think3d!: Training spatial thinking fundamental to stem education. Cognit. Instruct. 31(4), 434–455 (2013)

    Article  Google Scholar 

  29. Teepe, R., Molenaar, I., Oostdam, R., Verhoeven, L.: Stimulating parent-child interaction through storytelling activities of a family literacy program (2017)

  30. Teepe, R., Molenaar, I., Verhoeven, L.: Technology-enhanced storytelling stimulating parent-child interaction and preschool children’s vocabulary knowledge. J. Comput. Assisted Learn. 33(2), 123–136 (2017)

    Article  Google Scholar 

  31. Tenbrink, T., Taylor, H.A.: Conceptual transformation and cognitive processes in origami paper folding. J. Problem Solving 8(1), 1 (2015)

    Article  Google Scholar 

  32. Watanabe, T., Kinoshita, Y.: Folding support for beginners based on state estimation of origami. In: TENCON 2012 IEEE Region 10 Conference, pp. 1–6. IEEE (2012)

  33. Wiwatwattana, N., Laphom, C., Aggaitchaya, S., Chattanon, S.: Origami guru: An augmented reality application to assist paper folding. In: Information Technology: New Generations, pp. 1101–1111. Springer (2016)

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (2018YFC0831003) and the National Natural Science Foundation of China (61972233, 61802232).

Author information

Authors and Affiliations

  1. School of Software, Shandong University, Jinan, China

    Yingjie Song, Chenglei Yang, Wei Gai & Juan Liu

  2. Engineering Research Center of Digital Media Technology, MOE, Shandong University, Jinan, China

    Yingjie Song, Chenglei Yang, Wei Gai & Yulong Bian

  3. School of Mechanical, Electrical and Information Engineering, Shandong University, Weihai, China

    Yulong Bian & Juan Liu

Authors
  1. Yingjie Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenglei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Gai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yulong Bian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Juan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chenglei Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, Y., Yang, C., Gai, W. et al. A new storytelling genre: combining handicraft elements and storytelling via mixed reality technology. Vis Comput 36, 2079–2090 (2020). https://doi.org/10.1007/s00371-020-01924-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01924-3

Keywords

Search

Navigation