Students’ evaluation of teaching in the project-based learning programme: An instrument and a development process
Introduction
Students’ evaluation of teaching (SET) is a commonly adopted practice to measure teaching effectiveness at universities (Dodeen, 2013; Mart, 2017). It may allow teachers to refine their teaching and serves as an important consideration when making administrative decisions on tenure, promotion, merit pay, salary increase, teaching award and contract renewal.
Typically, SET instrument contains a Likert-type rating scale and some open-ended questions that allow students to write their comments or suggestions (Alok, 2011). It covers characteristics of effective teaching (Henderson et al., 2014; Sadrina et al., 2018) that include the extent of knowledge covered in class, preparation and organization of lesson, interaction with students, clarity of teaching, effectiveness of communication, effective use of technology, enthusiasm of teacher, frequency and quality of feedback, fairness of grading and availability of teacher outside class.
Increasingly, universities are embracing an alternative teaching pedagogy in project-based learning (PBL) that inculcates learning philosophy of taking what was learned in one situation and applying it to new situations (Lee et al., 2014). Universities' teaching pedagogy is gradually shifting from the traditional teaching method of mainly delivering content, to engaging students in pedagogy that centers on applying and reflecting knowledge (Pan et al., 2020). PBL results in a shift of teacher's role from transmitter of information to facilitator of learning (Pan et al., 2019; Seow et al., 2019; Prince & Felder, 2007). This shift in the role of a teacher may alter students' expectation of how a class is conducted and influence students' evaluation of teaching (Sadrina et al., 2018).
While SET instrument has gained widespread use in most universities, nevertheless, little is known about whether a single SET instrument is used to measure the effectiveness of alternative teaching method in PBL pedagogy. The ambiguity in the SET literature is a relevant concern because the performance dimensions and examples of the SET instrument used in one teaching method may now need to be re-designed to capture the idiosyncrasy of another teaching method. For instance, questions in the SET instrument such as ‘teacher's facilitation and mentoring skills' and ‘experience working with real companies’, are directly applicable to the PBL course but less relevant for a course taught using the traditional method. Therefore, without having appropriate performance dimensions and examples in the SET instrument, it is difficult to surface relevant student feedback to enhance improvement in teaching and conduct effective assessment of PBL pedagogy.
Accordingly, this paper focuses on the SET instrument developed at a university in Singapore, UNIS (a pseudonym), for PBL-type courses called UNIS-X programme. The instrument uses nine performance examples along four performance dimensions (i.e., effectiveness of instructor, experience in the course, effort put into the course and experience working with real companies) on a seven-point Likert-type rating scale. The instrument also includes some open-ended questions that allow students to provide their comments or suggestions in SET.
This paper will begin with a review of SET and PBL literature, followed by a discussion on the context in which the SET instrument was developed. Next, the paper will describe in detail the instrument's development processes. The paper will also show the performance dimensions and examples of the SET instrument. Finally, the paper concludes by highlighting the implications for research and education.
Section snippets
SET
SET is the primary method used to evaluate teaching effectiveness (Hobson & Talbot, 2001) in most universities (Richardson, 2005). Comm and Mathaisel (1998) indicate that almost all Association to Advance Collegiate Schools of Business (AACSB) accredited business schools responding to a survey use SET instruments as an example in determining teaching effectiveness. SET offers ideas to faculty for enhancing their teaching performance (Chen & Hoshower, 2003) and for university administrators to
Research methodology
Our strategy was to undertake a qualitative study of UNIS's development process of SET instrument for the PBL courses during 2016–17 period. The qualitative approach is particularly appropriate for our exploratory study since it allows us to better capture the organizational dynamics of the phenomenon and its ability to explain the phenomenon based on interpretation of data (McCray et al., 2021; Taylor & Bogdan, 1998).
For data collection, the study used secondary documentations from a variety
Case study
In 2009, UNIS recognized the importance of developing a rigorous and relevant instrument FACETS (“For Assessment of Continuing Excellence in Teaching”) to measure teaching and learning. It therefore formed a Task Force with a faculty representative from each of the six schools within UNIS to develop, test and launch an instrument that would be consistent with the evidence on best practices for assessing university teaching and learning, and would be able to capture the performance dimensions
Development of SET instrument for the PBL programme at UNIS (FACETS-X)
Recognizing the need to prepare its students with twenty-first century competencies so as to tackle increasingly complex real-world problems, UNIS launched undergraduate courses that adopt PBL pedagogy called UNIS-X programme. The UNIS-X Initiative is a paradigm shift which focuses on experiential learning as opposed to teaching as well as a mind-set shift to get the university to collaborate both internally and with our external stakeholders more. The PBL pedagogy at UNIS-X comprises four
Conclusion and implications
This paper presents the development process of a SET instrument used to measure the effectiveness of project-based learning pedagogy. By drawing upon a case study of UNIS, we demonstrate the development of a SET instrument and highlight the considerations surfaced during the development process. For researchers, this paper contributes to the SET and PBL literature by highlighting the need to have a dedicated SET instrument to measure the alternative teaching pedagogy in project-based learning.
Author contribution
Gary Pan: Methodology, writing, visualisation. Venky Shankararaman: Conceptualization, supervision, project administration. Kevin Koh: Resources, review & editing. Sandy Gan: Validation, formal analysis, data curation.
References (43)
- et al.
Wicked problems and project-based learning: Value-in-Use approach
International Journal of Management in Education
(2020) - et al.
Experiencing temporal patterns of action learning and the implications for leadership development
International Journal of Management in Education
(2021) - et al.
Examining an experiential learning approach to prepare students for the volatile, uncertain, complex and ambiguous (VUCA) work environment
International Journal of Management in Education
(2019) Project-based learning: Implementation & challenges
International Journal of Education, Learning and Development
(2018)Student evaluation of teaching: An instrument and a development process
International Journal of Teaching and Learning in Higher Education
(2011)- et al.
What teachers and administrators “need to know” about project-based learning implementation
- et al.
Assessing faculty work: Enhancing individual and institutional performance
(1994) - et al.
Student evaluation of teaching effectiveness: An assessment of student perception and motivation
Assessment & Evaluation in Higher Education
(2003) The effective use of secondary data
Learning and Motivation
(2001)- et al.
Personality and the student evaluation of teaching
Journal of Marketing Education
(2006)
Evaluating teaching effectiveness in America's business schools: Implications for service markets
Journal of Professional Services Marketing
The use of secondary data in business ethics research
Journal of Business Ethics
Creating a comprehensive system for evaluating and supporting effective teaching
Stock market reactions to announced corporate illegalities
Journal of Business Ethics
College students' evaluation of effective teaching: Developing an instrument and assessing its psychometric properties
Research in Higher Education Journal
Student evaluation instruments: The interactive impact of course requirement, student level, department and anticipated grade
American Journal of Business Education
A comparison of two response scale formats used in teaching evaluation questionnaires
Assessment & Evaluation in Higher Education
A study of the efficacy of project-based learning integrated with computer-based simulation - stella
Educational Technology & Society
Project-based learning: Moving students through the GED with meaningful learning
The effect of project based learning on learning outcomes in the 5th grade social studies course in primary education
Educational Sciences: Theory and Practice
Assessment of teaching effectiveness: Lack of alignment between instructors, institutions, and research recommendations
Physics Education Research
Cited by (18)
A multiple criteria framework to assess learning methodologies
2023, Thinking Skills and CreativityThe pivotal role of students’ absorptive capacity in management learning
2022, International Journal of Management EducationCitation Excerpt :Traditional learning methods, predominantly theoretical lectures (i.e., classes where the teacher is speaking most of the time, usually with minimal student participation or feedback), are extensively applied in management education (Ungaretti et al., 2015). However, innovative learning methodologies, such as flipped teaching (Boevé et al., 2017; Fernández-Mesa et al., 2016; Martínez-Jiménez & Ruiz-Jiménez, 2020; Tsai & Chiang, 2018; Wang et al., 2019), problem-based learning (Ahlfeldt et al., 2005; Ungaretti et al., 2015), action learning (Yeadon-Lee & Worsdale, 2012), experiential learning (Seow et al., 2019), project-based learning (Pan et al., 2021), or student-centred learning (Coleman & Money, 2020; Severiens et al., 2015), among other approaches, have been suggested to teach management skills (Daspit & D'Souza, 2012; Okolie et al., 2021). These innovative learning methodologies often improve educational outcomes (Beachboard et al., 2011; Daspit & D'Souza, 2012), particularly blended teaching, which combines traditional teaching with more innovative approaches (Daspit & D'Souza, 2012).
It's not you, It's me: Can others learn from the challenges of teaching operations and supply chain management?
2022, International Journal of Management EducationCitation Excerpt :Likewise, individual instructor personality strengths could be evaluated relative to the list. Academic programs may also consider adjusting teaching evaluation instruments to better capture different teaching pedagogies for relatively unique classes like OSCM (Pan et al., 2021). Extant theory further supports several of the factors in Table 9.
The biasing impact of positive instructor reputation on student evaluations of teaching
2022, International Journal of Management EducationCitation Excerpt :In fact, SETs are often the primary, or perhaps exclusive measure of professors' teaching performance (Waldman, 2008)—although some progressive institutions have begun encouraging or using additional methods. The accuracy of SETs are thus paramount since they are used in decisions related to professors' contract renewal, raises, tenure, and promotion (Munz & Munz, 1997; Pan et al., 2021). In business organizations, supervisors' ratings of others’ performance has often been found to be influenced by things not reflective of actual job performance, and thus contain biases and distortions (e.g., Higgins, Judge, and Ferris, 2003).
Curriculum Development in Higher Education in Light of Culture and Religiosity: A Case Study in Aceh of Indonesia
2024, International Journal of Society, Culture and Language