Abstract
In a previous paper, we introduced the process of developing a new prescriptive method for conceptual engineering design called ICE (idea-configuration-evaluation), and demonstrated its application in detail. The new design model is based on the well-established theoretical foundation of C–K theory and the empirically derived parameter analysis method. In this paper, we describe the testing and evaluation of the model’s formulation through a design experiment that examines the performance of external participants when applying it. Two categories of evaluation metrics are used: applicability and effectiveness of the method. Nine quantitative and qualitative metrics describe aspects such as the ease of teaching and using the design procedure, and its support of creativity and innovation. The experiment took place in an academic environment and the results were obtained by analyzing detailed written design reports, assessing characteristics of the final solutions, and examining replies to reflective questionnaires. The conclusions from the study are that the ICE model presents a clear and concise step-by-step procedure, is conducive to teaching and practicing design, captures the dynamics and rationale of the conceptual design process, and can therefore lead to viable and innovative conceptual designs. Another, general contribution of the paper is in delineating the application of a plurality of measures in a non-comparative assessment of a design method.
Similar content being viewed by others
Notes
As pointed out by Seepersad et al. (2006), validation and verification are sometimes used with the opposite meaning to this definition.
A design studio, sometimes called “clinic”, is a setting whereby a small number of student teams work independently but under the supervision of an instructor. The instructor demonstrates the proper design practices and guides the students in the correct direction, but without actually solving problems for them.
References
Agogué M, Kazakçi A, Hatchuel A, Le Masson P, Weil B (2014) The impact of type of examples on originality: explaining fixation and stimulation effects. J Creat Behav 48:1–12
Blessing LTM, Chakrabarti A (2009) DRM, a design research methodology. Springer, London
Boehm BW (1981) Software engineering economics. Prentice-Hall, Upper Saddle River
Brun J, Le Masson P, Weil B (2018) Getting inspiration or creating inspiration? The role of knowledge structures in idea generation. In: Proceedings of 15th international design conference (DESIGN 2018), Dubrovnik
Chulvi V, Mulet E, Chakrabarti A, López-Mesa B, González-Cruz C (2012) Comparison of the degree of creativity in the design outcomes using different design methods. J Eng Des 23(4):241–269
Dorst K, Cross N (2001) Creativity in the design process: co-evolution of problem–solution. Des Stud 22(5):425–437
Flick U (2009) An introduction to qualitative research, 4th edn. Sage, London
Forsberg K, Mooz H (1999) System engineering for faster, cheaper, better. INCOSE Int Symp 9:924–932. https://doi.org/10.1002/j.2334-5837.1999.tb00258.x
Frey DD, Dym CL (2006) Validation of design methods: lessons from medicine. Res Eng Des 17:45–57
Goldschmidt G, Sever AL (2011) Inspiring design ideas with texts. Des Stud 32(2):139–155
Green G, Kennedy P, McGown A (2002) Management of multi-method engineering design research: a case study. J Eng Technol Manag 19(2):131–140
Hatchuel A, Weil B (2002) C–K theory: notions and applications of a unified design theory. In: Proceedings of Herbert Simon international conference on design sciences, Lyon
Hatchuel A, Weil B (2003) A new approach of innovative design: an introduction to C–K theory. In: Proceedings of 14th international conference on engineering design (ICED’03), Stockholm
Hatchuel A, Weil B (2009) C-K design theory: an advanced formulation. Res Eng Des 19:181–192
Hernandez N-V, Shah JJ, Smith SM (2010) Understanding design ideation mechanisms through multilevel aligned empirical studies. Des Stud 31(4):382–410
Keller A, Binz H (2009) Requirements on engineering design methodologies. In: Proceedings of 17th international conference on engineering design (ICED’09), Stanford University
Kroll E (2013) Design theory and conceptual design: contrasting functional decomposition and morphology with parameter analysis. Res Eng Des 24:165–183
Kroll E, Shihmanter A (2011) Capturing the conceptual design process with concept-configuration-evaluation triplets. In: Proceedings 18th international conference on engineering design (ICED’11), Copenhagen
Kroll E, Condoor SS, Jansson DG (2001) Innovative conceptual design: theory and application of parameter analysis. Cambridge University Press, Cambridge
Kroll E, Le Masson P, Weil B (2014) Steepest-first exploration with learning-based path evaluation: uncovering the design strategy of parameter analysis with C-K theory. Res Eng Des 25:351–373
Le Masson P, Weil B, Hatchuel A (2010) Strategic management of innovation and design. Cambridge University Press, Cambridge
Lewis W, Field B, Weir J (2011) Assessing quality of ideas in conceptual mechanical design. In: Proceedings 18th international conference on engineering design (ICED’11), Copenhagen
Moss J (1966) Measuring creative abilities in junior high school industrial arts. American Council on Industrial Arts Teacher Education, Washington, DC
Motte D, Eriksson M (2016) Assessment framework for a methodology under development—application to the PDA methodology. In: Proceedings 11th international symposium on tools and methods of competitive engineering (TMCE 2016), Delft
Reich Y (2010) My method is better. Res Eng Des 21:137–142
Sarkar P, Chakrabarti A (2011) Assessing design creativity. Des Stud 32(4):348–383
Seepersad CC, Pedersen K, Emblemsvåg J, Bailey R, Allen JK, Mistree F (2006) The validation square: how does one verify and validate a design method? In: Lewis KE, Chen W, Schmidt LC (eds) Decision making in engineering design. ASME, New York, pp 303–314
Shah JJ, Smith SM, Noe V-H (2003) Metrics for measuring ideation effectiveness. Des Stud 24(2):111–134
Strickfaden M, Heylighen A (2010) Scrutinizing design educators’ perceptions of the design process. Artif Intell Eng Des Anal Manuf 24(3):357–366
Teegavarapu S, Summers JD, Mocko GM (2008) Case study method for design research: a justafication. In: Proceedings ASME 2008 international design engineering technical conferences and computers and information in engineering conference, Brooklyn, New York, August 3–6
Vermaas PE (2014) Design theories, models and their testing: on the scientific status of design research. In: Chakrabarti A, Blessing LTM (eds) An anthology of theories and models of design. Springer, London, pp 47–66
Weisbrod G, Kroll E (2018) Idea-configuration-evaluation (ICE): development and demonstration of a new prescriptive model of the conceptual engineering design process based on parameter analysis and C–K theory. Res Eng Des 29:203–225
Yin RK (1994) Case study research: design and methods, 2nd edn. Sage Publications, Thousand Oaks
Acknowledgements
This work was supported by the Israel Science Foundation under grant number 546/12.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kroll, E., Weisbrod, G. Testing and evaluating the applicability and effectiveness of the new idea-configuration-evaluation (ICE) method of conceptual design. Res Eng Design 31, 103–122 (2020). https://doi.org/10.1007/s00163-019-00324-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00163-019-00324-6