Faculty perceptions of a town hall model for engagement in continuous quality improvement

Abstract

Introduction

To determine faculty perceptions of participating in a town hall model on the American Association of Colleges of Pharmacy Faculty Curriculum Quality Surveys (CQS) focused on continuous quality improvement (CQI) at two institutions.

Methods

To support a culture of assessment focused on CQI, Ferris State University College of Pharmacy (FSUCOP) developed and implemented a town hall model for use with the CQS in fall 2017. It was determined that involving faculty in the CQS analysis process may assist with interpretation and CQI. FSUCOP sought to determine if the CQI model utilized could also be implemented at another institution. Therefore, Cedarville University School of Pharmacy (CUSOP) replicated the FSUCOP town hall model for use in 2018–2019 academic year CQS data interpretation and dissemination. To determine the perceptions of efficacy, faculty at FSUCOP and CUSOP received a pre-survey prior to the fall 2018 town hall and a post-survey following completion of the town hall. The pre- and post-surveys utilized five-point, Likert-type agreement questions (strongly disagree to strongly agree).

Results

A total of 47 and 26 faculty completed the pre- and post-surveys, respectively. Faculty agreement on the survey items increased after participating in the town hall. Nearly 81% of faculty strongly or somewhat agreed the town hall model was a good use of their time.

Conclusion

The town hall model was replicable at another institution. Faculty perceived the town hall model as a positive approach to CQI, and discussions provided assessment personnel with valuable information for data interpretation and usage.

Introduction

Accrediting bodies have required varying types of assessment data to demonstrate compliance with accreditation requirements.¹ The Accreditation Council for Pharmacy Education (ACPE) Standards 2016 emphasis on curricular and programmatic assessment aligns with trends seen in higher education as a whole. Specifically, Standards 24 and 25 require schools/colleges of pharmacy (S/COP) to utilize data obtained from valid and reliable assessments to implement constructive change and enhance overall program quality in the spirit of continuous quality improvement (CQI).² CQI is a process where goals are established for the program or program element, and the S/COP formally and systematically sets out to achieve those goals through cyclically examining assessment data and improving accordingly. According to ACPE, all 141 fully-accredited S/COP are currently compliant with Standards 24 and 25 as of November 2020. Based on the success of S/COP in maintaining their fully accredited status, compliance with assessment practices is occurring. However, it is unknown whether S/COP are complying with assessment practices or if the assessment taking place is translating into developing a culture of assessment for CQI, as opposed to one merely for compliance.

In a culture of assessment, institutions are able to move beyond the gathering of results and move toward facilitating questions, decision making, and data-driven actions that guide CQI.³ Lakos and Phipps⁴ define a culture of assessment as an “organizational environment in which decisions are based on facts, research, and analysis, and where services are planned and delivered in ways that maximize positive outcomes and impacts for customers and stakeholders.” To achieve such a culture requires a clear vision and a shared understanding regarding what assessment is taking place and how the results are going to be utilized.

Stakeholders are critical in the development of a culture of assessment, and as Hill⁵ concludes, “Such a culture cannot be imposed or created overnight. Rather, it is the result of gradual steps that serve to build trust in the process and confidence in the ability to succeed in it.” Anderson et al⁶ summarize, “only through a continuous and systematic process with full collaboration among all vested individuals can a true culture of assessment be attained.”

The critical role faculty play in developing effective assessment programs, and, ultimately, a culture of assessment in higher education is well established.7, 8, 9, 10 However, those striving to develop a culture of assessment often experience challenges attempting to involve faculty due to the availability of resources and the complexity of developing a systematic approach to utilizing data.¹¹ To create a culture of assessment, faculty engagement is key.

Since 2007, S/COP have been administering the American Association of Colleges of Pharmacy (AACP) Graduating Student, Alumni, Preceptor, and Faculty Curriculum Quality Surveys (CQS). The CQS are intended to be utilized for CQI and accreditation.¹² In 2015, the surveys underwent a revision to reflect Standards 2016, specifically the changes in Standards 1 to 4. Currently, 20 of the 25 standards require results from the administration of one of the four CQS as part of the required self-study documentation.¹³ ACPE Standards 2016 Guidance more specifically recommends revisions to curricula based on assessment data from the AACP standardized CQS.¹⁴ While the expectation for CQS utilization is clear within the standards, the literature lacks examples and best practices of how these tools are being utilized by S/COP for CQI.

AACP CQS Principles of Good Use guidelines have been developed to aid S/COP in their use of CQS data. The guide recommends triangulation of data with a variety of additional data sources to gather a more comprehensive understanding of the data provided. The guide also recommends S/COP develop action plans following CQS administration. More specifically, one method outlined for the action plans is to implement a “town hall” model with stakeholders.¹² Currently, there is little evidence to show whether S/COP are utilizing a town hall model to gain additional feedback from survey respondents for the purpose of CQI.

Town halls have a long-standing history of utilization in politics and business. However, there are limited examples to support their use in engaging pharmacy faculty as a precursor to developing a culture of assessment or the perceived value of town halls to the participants. In order to continue to promote a culture of assessment by engaging faculty in the data use process, the objective of this study was to determine faculty perceptions of (1) the use of data for CQI and (2) participation in the feedback process for CQI through a town hall model related to the AACP Faculty CQS at two institutions.