Skip to main content

Advertisement

SpringerLink
Log in

Connected design rationale: a model for measuring design learning using epistemic network analysis

  • Original Research
  • Published:
Instructional Science Aims and scope Submit manuscript

Abstract

Virtual learning environments have the potential to support students’ development of design skills in engineering education. However, few approaches exist for modeling and measuring design learning as it emerges in authentic practices, which often includes collaboration. This study merges learning sciences research with engineering design education to develop an approach for modeling and measuring design thinking. I propose a connected design rationale model which identifies relationships among design moves and rationale. Results from a qualitative examination of how professional engineers make connections among moves and rationales were used as the foundation to examine students in virtual internships. Using digital collaborative chat data and Epistemic Network Analysis (ENA), the discourse networks of students who had high and low scores in the virtual internship were compared to the discourse patterns of professional engineers to determine if measuring connected design rationale reveals meaningful differences between expert and novice design thinking. The results show a significant difference between high and low-performing students in terms of their patterns of connections and that high-performing students in the virtual internship made connections that were more like experts than low-performing students. Results suggest that a connected design rationale model distinguishes between experts and novices in meaningful ways and can be a robust approach for research in learning sciences and engineering education.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Adams, R. S., Daly, S. R., Mann, L. M., & Dall’Alba, G. (2011). Being a professional: Three lenses into design thinking, acting, and being. Design Studies, 32(6), 588–607.

    Article  Google Scholar 

  • Arastoopour, G., Chesler, N. C., & Shaffer, D. W. (2014). Epistemic persistence: A simulation-based approach to increasing participation of women in engineering. Journal of Women and Minorities in Science and Engineering, 20(3), 211–234.

    Article  Google Scholar 

  • Arastoopour, G., & Shaffer, D. W. (2015). Epistemography and Professional CSCL Environment Design. In Lindwall, O., Häkkinen, P., Koschman, T. Tchounikine, P. & Ludvigsen, S. (Eds.) Exploring the Material Conditions of Learning: The Computer Supported Collaborative Learning (CSCL) Conference 2015, Volume 1 (pp. 204–211) Gothenburg, Sweden.

  • Arastoopour, G., Shaffer, D. W., Swiecki, Z., Ruis, A. R., & Chesler, N. C. (2016). Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. International Journal of Engineering Education, 32(2), 1492–501.

    Google Scholar 

  • Arastoopour Irgens, G. (2017). Connected design rationale: Modeling and measuring engineering design learning [Ph.D., The University of Wisconsin—Madison]. In ProQuest Dissertations and Theses (1901451547). ProQuest Dissertations & Theses Global. https://www.proquest.com/docview/1901451547/

  • Arastoopour Irgens, G. (2019). Facilitating a sense of belonging for women of color in engineering. In R. D. Roscoe, E. K. Chiou, & A. R. Wooldrige (Eds.), Advancing diversity, inclusion, and social justice through human systems engineering (pp. 221–239). CRC Press. https://doi.org/10.1201/9780429425905

  • Atman, C. J., Adams, R. S., & Turns, J. (2000). Using multiple methods to evaluate a freshmen design course. In 30th Annual Frontiers in Education Conference. Building on a Century of Progress in Engineering Education. Conference Proceedings (IEEE Cat. No. 00CH37135) 2000, pp. S1A/6-S1A13 vol.2, https://doi.org/10.1109/FIE.2000.896605.

  • Atman, C. J., Chimka, J. R., Bursic, K. M., & Nachtmann, H. L. (1999). A comparison of freshman and senior engineering design processes. Design Studies, 20(2), 131–152.

    Article  Google Scholar 

  • Atman, C. J., Eris, O., McDonnell, J., Cardella, M. E., & Borgford-Parnell, J. L. (2014). Engineering design education. In A. Johri, & B. M. E. Olds (Eds.), Cambridge handbook of engineering education research (pp. 201–226). Cambridge University Press. https://doi.org/10.1017/CBO9781139013451.015

  • Atman, C. J., Kilgore, D., & McKenna, A. (2008). Characterizing design learning: A mixed-methods study of engineering designers’ use of language. Journal of Engineering Education, 97(3), 309–326.

    Article  Google Scholar 

  • Bagley, E., & Shaffer, D. W. (2009). When people get in the way: Promoting civic thinking through epistemic gameplay. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 1(1), 36–52.

    Article  Google Scholar 

  • Bagley, E., & Shaffer, D. W. (2010). The epistemography of urban and regional planning 912: Appropriation in the face of resistance. In K. Gomez, L. Lyons, & J. Radinksy (Eds.), Learning in the disciplines: Proceedings of the 9th International Conference of the Learning Sciences (ICLS 2010)—volume 1 full papers (ICLS 2010): Vol. volume 1 (pp. 81–88).

  • Bailey, R., & Szabo, Z. (2006). Assessing engineering design process knowledge. International Journal of Engineering Education, 22(3), 508–518.

    Google Scholar 

  • Bartholomew, S. R. (2017). Assessing open-ended design problems. Technology and Engineering Teacher, 76(6), 13.

    Google Scholar 

  • Bartholomew, S. R., Strimel, G. J., & Jackson, A. (2018). A comparison of traditional and adaptive comparative judgment assessment techniques for freshmen engineering design projects. International Journal of Engineering Education, 34(1), 20–33.

    Google Scholar 

  • Bergsman, K. (2018). The “Structured” engineering design notebook: A new tool for design thinking within a studio design course. In 2018 ASEE Annual Conference & Exposition Proceedings, 31076. https://doi.org/10.18260/1-2--31076

  • Buchal, R. O. (2018). Collaborative digital design notebooks. In Proceedings of the Canadian Engineering Education Association (CEEA). https://doi.org/10.24908/pceea.v0i0.10362

  • Cai, Z., Siebert-Evenstone, A., Eagan, B., Shaffer, D. W., Hu, X., & Graesser, A. C. (2019). nCoder+: A semantic tool for improving recall of nCoder coding. In B. Eagan, M. Misfeldt, & A. Siebert-Evenstone (Eds.), Advances in quantitative ethnography (pp. 41–54). Springer International Publishing. https://doi.org/10.1007/978-3-030-33232-7_4

  • Chesler, N. C., Arastoopour, G., D’Angelo, C. M., Bagley, E. A., & Shaffer, D. W. (2013). Design of a professional practice simulator for educating and motivating first-year engineering students. Advances in Engineering Education, 3(3), n3.

    Google Scholar 

  • Chesler, N. C., Ruis, A. R., Collier, W., Swiecki, Z., Arastoopour, G., & Williamson Shaffer, D. (2015). A novel paradigm for engineering education: Virtual internships with individualized mentoring and assessment of engineering thinking. Journal of Biomechanical Engineering. https://doi.org/10.1115/1.4029235

    Article  Google Scholar 

  • Dalkey, N. C. (1969). The Delphi Method: An experimental study of group opinion. RAND Corporation. https://www.rand.org/pubs/research_memoranda/RM5888.html

  • DiSessa, A. A. (1993). Toward an epistemology of physics. Cognition and Instruction, 10(2–3), 105–225.

    Article  Google Scholar 

  • Dym, C. L., Agogino, A. M., Eris, O., Frey, D. D., & Leifer, L. J. (2005). Engineering design thinking, teaching, and learning. Journal of Engineering Education, 94(1), 103–120.

    Article  Google Scholar 

  • Eagan, B., Misfeldt, M., & Siebert-Evenstone, A. (Eds.). (2019). Advances in quantitative ethnography: First international conference, ICQE 2019, Madison, WI, October 20–22, 2019, Proceedings (1st ed. 2019 edition). Springer.

  • Eagan, B. R., Rogers, B., Serlin, R., Ruis, A. R., Arastoopour Irgens, G., Shaffer, D. W. (2017). Can We Rely on IRR? Testing the assumptions of inter-rater reliability. In Smith, B., Borge, M., Mercier, E., & Lim, K.Y. (Eds.), Making a Difference: Prioritizing Equity and Access in CSCL: Proceedings of the 12th International Conference on Computer Supported Collaborative Learning Volume 1 (pp. 529–532). Philadelphia, PA.

  • Febrian, A., Lawanto, O., & Cromwell, M. (2015). Advancing research on engineering design using e-Journal. In 2015 IEEE Frontiers in Education Conference (FIE) (pp. 1–5). https://doi.org/10.1109/FIE.2015.7344191

  • Gee, J. P. (2011). An introduction to discourse analysis: Theory and method (3rd ed.). New York: Routledge.

  • Gero, J. S., & Kannengiesser, U. (2008). An ontological account of Donald Schön’s reflection in designing. International Journal of Design Sciences and Technologies, 15(2), 77–90.

    Google Scholar 

  • Glaser, B. G., & Strauss, A. L. (1967). Discovery of grounded theory: Strategies for qualitative research. Chicago: Adline Publishing Co.

  • Hatfield, D. (2015). The right kind of telling: An analysis of feedback and learning in a journalism epistemic game. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 7(2), 1–23.

    Article  Google Scholar 

  • Hatfield, D., & Shaffer, D. W. (2010). The epistemography of journalism 335: Complexity in developing journalistic expertise. In K. Gomez, L. Lyons, & J. Radinksy (Eds.), Learning in the disciplines: Proceedings of the 9th International Conference of the Learning Sciences (ICLS 2010)—volume 1 full papers (ICLS 2010): Vol. volume 1 (pp. 628–635).

  • Herder, T., Swiecki, Z., Fougt, S. S., Tamborg, A. L., Allsopp, B. B., Shaffer, D. W., & Misfeldt, M. (2018). Supporting teachers’ intervention in students’ virtual collaboration using a network based model. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 21–25). https://doi.org/10.1145/3170358.3170394

  • Hod, Y., & Sagy, O. (2019). Conceptualizing the designs of authentic computer-supported collaborative learning environments in schools. International Journal of Computer-Supported Collaborative Learning, 14(2), 143–164. https://doi.org/10.1007/s11412-019-09300-7

    Article  Google Scholar 

  • Iacobucci, D., Posavac, S. S., Kardes, F. R., Schneider, M. J., & Popovich, D. L. (2015). Toward a more nuanced understanding of the statistical properties of a median split. Journal of Consumer Psychology, 25(4), 652–665. https://doi.org/10.1016/j.jcps.2014.12.002

    Article  Google Scholar 

  • Lee, J. (1997). Design rationale systems: Understanding the issues. IEEE Expert, 12(3), 78–85.

    Article  Google Scholar 

  • Lee, J., & Lai, K.-Y. (1991). What’s in design rationale? Human-Computer Interaction, 6(3–4), 251–280.

    Article  Google Scholar 

  • Linn, M. C., Eylon, B.-S., & Davis, E. A. (2004). The knowledge integration perspective on learning. In M. C. Linn, E. A. Davis & P. Bell (Eds.), Internet environments for science education (pp. 29–46). Mahwah, NJ: Lawrence Erlbaum.

  • Lloyd, P. (2019). You make it and you try it out: Seeds of design discipline futures. Design Studies, 65, 167–181. https://doi.org/10.1016/j.destud.2019.10.008

    Article  Google Scholar 

  • McCall, R., & Burge, J. (2016). Untangling wicked problems. Artificial Intelligence for Engineering Design, Analysis and Manufacturing: AI EDAM, 30(2), 200–210. https://doi.org/10.1017/S089006041600007X

    Article  Google Scholar 

  • Moyne, M. M., Herman, M., Gajos, K. Z., Walsh, C. J., & Holland, D. P. (2018b). The development and evaluation of DEFT, a web-based tool for engineering design education. IEEE Transactions on Learning Technologies, 11(4), 545–550. https://doi.org/10.1109/TLT.2018.2810197

    Article  Google Scholar 

  • Moyne, M., Herman, M., Walsh, C., & Holland, D. (2018). An evaluation of an engineering design class using mixed methods techniques. In 2018 ASEE Annual Conference & Exposition Proceedings, 29782. https://doi.org/10.18260/1-2--29782

  • Nash, P., & Shaffer, D. W. (2011). Mentor modeling: The internalization of modeled professional thinking in an epistemic game. Journal of Computer Assisted Learning, 27(2), 173–189. https://doi.org/10.1111/j.1365-2729.2010.00385.x

    Article  Google Scholar 

  • Pawley, A. L. (2017). Shifting the “default”: The case for making diversity the expected condition for engineering education and making whiteness and maleness visible. Journal of Engineering Education, 106(4), 531–533.

    Article  Google Scholar 

  • Rittel, H. (1988). The reasoning of designers. Stuttgart: IGP.

  • Rockwell, J. A., Grosse, I. R., Krishnamurty, S., & Wileden, J. C. (2010). A semantic information model for capturing and communicating design decisions. Journal of Computing and Information Science in Engineering. https://doi.org/10.1115/1.3462926

    Article  Google Scholar 

  • Schön, D. A. (1983). The reflective practitioner: How professionals think in action. Routledge.

  • Schön, D. A. (1984). Problems, frames and perspectives on designing. Design Studies, 5(3), 132–136.

    Article  Google Scholar 

  • Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. JosseyBass.

  • Schön, D. A. (1988). Designing: Rules, types and worlds. Design Studies, 9(3), 181–190.

    Article  Google Scholar 

  • Shaffer, D. W. (2006). How computer games help children learn. Macmillan.

    Book  Google Scholar 

  • Shaffer, D. W. (2017). Quantitative ethnography. Madison: Cathcart Press.

  • Shaffer, D. W. & Ruis, A. R. (2017). Epistemic Network Analysis: A Worked Example of Theory-Based Learning Analytics. In C. Lang, G. Siemens, A. Wise, & D. Gasevic, Eds., Handbook of Learning Analytics (First Edition, pp. 175–187). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.015

  • Siebert-Evenstone, A. L., Arastoopour Irgens, G., Collier, W., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2017). In search of conversational grain size: Modeling semantic structure using moving stanza windows. Journal of Learning Analytics, 4(3), 123–139.

    Article  Google Scholar 

  • Simon, H. A. (1996). The sciences of the artificial (3rd ed.). Cambridge: MIT Press.

  • Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research. Journal of Information Technology Education: Research, 6, 001–021. https://doi.org/10.28945/199

    Article  Google Scholar 

  • Strauss, A., & Corbin, J. (1994). Grounded theory methodology. Handbook of Qualitative Research, 17(1), 273–285.

    Google Scholar 

  • Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks: Sage Publications

  • Svarovsky, G. N. (2011). Exploring complex engineering learning over time with epistemic network analysis. Journal of Pre-College Engineering Education Research (J-PEER), 1(2), 4.

    Google Scholar 

  • Tener, R. K., Winstead, M. T., & Smaglik, E. J. (2001). Experiential learning from internships in construction engineering. ASEE Annual Conference Proceedings. 4889–4920

  • Yoder, B. L. (2017). Engineering by the numbers. American Society for Engineering Education.

Download references

Acknowledgements

This work was funded in part by the National Science Foundation (DRL-1661036, DRL-1713110), the Wisconsin Alumni Research Foundation, and the Office of the Vice Chancellor for Research and Graduate Education at the University of Wisconsin-Madison. The opinions, findings, and conclusions do not reflect the views of the funding agencies, cooperating institutions, or other individuals.

Author information

Authors and Affiliations

  1. Clemson University, 207 Old Main, Clemson, SC, 29631, USA

    Golnaz Arastoopour Irgens

Authors
  1. Golnaz Arastoopour Irgens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Golnaz Arastoopour Irgens.

Ethics declarations

Conflict of interest

The author declares no conflict of interest.

Ethical Approval

The data collection and analysis procedures employed in this study were granted approval by the Institutional Review Board at the University of Wisconsin-Madison.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Fig. 7.

Fig. 7
figure 7

Prompt that was given to Nephrotex students in the design activity for writing in their engineering notebooks

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arastoopour Irgens, G. Connected design rationale: a model for measuring design learning using epistemic network analysis. Instr Sci 49, 561–587 (2021). https://doi.org/10.1007/s11251-021-09551-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11251-021-09551-8

Keywords

Search

Navigation