The optimal method of teaching and acquiring procedural skills is a contested area in medical education, where historically the “apprenticeship model” has been used.1,2,3,4,5,6,7 Feedback as a method of engaging learners 8 has long been identified as an important component of learning in medical education.3,9 In healthcare learning, studies have shown that video feedback (VF), benchmarking, and modeling may be superior to traditional methods of verbal feedback alone.10,11,12,13,14

Video feedback occurs when learners watch videos of themselves performing skills, with or without expert audio annotation. Learners may watch these videos independently, or with an expert to provide concurrent guidance. This technique has contributed to the appraisal of procedural skill competence in medicine.10,12,15,16,17,18 The use of VF originated in athletics learning,19 which inspired robust literature supporting VF as an effective means of improving communication skills in healthcare education.11,12,15,20,21,22,23 When applied to medical procedural skills, there are fewer studies that investigate its effectiveness as a procedural teaching tool. The Kirkpatrick model is a well-established educational model used to rank the impact of educational programs.24 In that model, there are four levels of learning evaluation: level 1 (reaction) evaluates how well a program was received, level 2 (learning) assesses skill acquisition, level 3 (transfer) determines whether the acquired skills change field behaviour, and level 4 (results) determines the final outcome (e.g., lower cost, higher efficiency).24 Current evidence robustly supports that VF improves learner satisfaction at level 1 (reaction),11,12,25,26,27,28,29 but there is a need to expand level 2 evidence (skill acquisition) in surgery30,31 and anesthesia learning.26,27

Video modeling (VM) is an alternate learning method that stems from behavioural research where learners watch videos of experts performing skills, providing a “perceptual blueprint”32 activating “mirror neurons”.33 Observational practice provides unique opportunities to refine these skills and modify behaviour.34 Video modeling has been effective in improving athletic performance35 and laparoscopic skills.13

Video modeling and VF both have educational roots in deliberate practice, but in isolation from each other may not work as effectively. Deliberate practice involves focused repetitive practice for skill improvement, with feedback as the crux to augment clinical learning.1,3,36,37 During deliberate practice, VF can help to promote self-assessment as a source of feedback.1 Video modeling is also used in deliberate practice for learners to observe and analyze how experts perform at key decision points, assisted by external expert guidance.37 Some studies have not shown efficacy of VF28,29,35 or VM,38,39 possibly because interventions consisted of single practice sessions29,38,39 or long periods between sessions.28 These two methodologies combined (VM + VF) may address a gap between observational learning, imitation, and self-assessment.19

A combination of VM and VF has been studied in athletics,19 with variable efficacy based upon activity type.35,39,40 This combined technique has shown promising results in a small study of medical residents, showing improved hernia repair technique compared with traditional oral feedback.41 Nevertheless, this is the only medical procedural skills study to utilize this combined methodology.

No study to date has investigated the effects of combined VF + VM administered in a deliberate practice session with repetitive practice within a short time frame in novice learners. In the present study, we hypothesized that repeated VM with VF (group VM+VF) compared with repeated VM alone would reduce the time required for novice medical students to attempt peripheral IV cannulation.

Methods

Recruitment

This study was reviewed and approved by the Research Ethics Board at the University of Saskatchewan (Bio # 15-321, January 20 2016). The full study protocol can be reviewed at https://harvest.usask.ca/handle/10388/12454. This prospective, randomized, single-blind embedded mixed-methods study compared VM+VF review to VM alone review (Fig. 1). A convenience sample of novice medical students was recruited as part of an extracurricular IV skills workshop between September 22 2016 and March 20 2017. Recruitment was accomplished through posters placed at the College of Medicine, University of Saskatchewan and distribution of an e-mail invitation to student class representatives. Learners that expressed interest provided written informed consent. Learners that had more than five prior experiences with IV cannulation were excluded.

Fig. 1
figure 1

Study design for peripheral intravenous (IV) cannulation in novice medical students. Participants received an entry survey and standardized instruction (shown in orange), and then performed IV cannulation on each other while being audio and video recorded. Participants were assigned to either groups that consisted of video modeling (VM) alone (shown in blue), or both VM and VF (video feedback) stated as VM+VF (shown in red) group. The VM alone group reviewed an expert video performing IV cannulation, while the VM+VF group reviewed the expert video and their own recent IV cannulation experience. Both groups (VM alone or VM+VF) then performed another IV cannulation. This was repeated for four IV cannulation encounters and three video reviews. A post-test interview was conducted (shown in orange) to assess participant satisfaction and this was analyzed qualitatively using thematic content analysis.

Full size image

Pre-Test Survey

We administered a pre-test survey gathering data on the participants’ year of training, number of previous IV encounters, and any previous formal teaching they had received on IV cannulation, as well as the presence of any allergies to the local anesthetic Maxilene cream (RGR Pharma Ltd., LaSalle, ON, Canada) or medical conditions that would increase in risk by participation, such as bleeding conditions or active skin infections.

Procedures

Participants brought their own personal mobile device with video recording capabilities. If they did not possess one, a device was provided. We informed participants that they were being both audio and video recorded, and asked both facilitators and participants not to verbalize participant names during the procedure to maintain anonymity for data collection.

Participants received standardized instruction from a senior anesthesia resident facilitator or staff anesthesiologist facilitator on how to perform peripheral IV cannulation. A stepwise instructions list was provided to the facilitator beforehand that corresponded to a prepared expert video model for the study (Electronic Supplementary Material eVideo); facilitators were free to provide the instruction steps in their own words. The quality of the instruction was reinforced by the study investigator on the day of data collection, clarifying the instruction list and answering questions prior to study commencement. Instructional facilitators provided instruction to both groups. Both participants in an instructed pair could belong to either the same study group or a different study group. All facilitators provided instruction to participants in both groups.

Pretreatment with Maxilene cream was applied by the facilitators over what they judged would be feasible veins to perform cannulation; a minimum of 20 min elapsed prior to the intervention, allowing for full efficacy of the local anesthetic. Participants performed IV cannulation on each other as live models (with informed consent). At the time the research was performed, this form of paired practice was standard at this institution, as it allowed practical, realistic learning in a safe environment prior to patient encounters. All IV cannulation encounters were audio and video recorded. Only participants’ hands were captured to preserve anonymity, and reasonable attempts were made to cover any distinctive markings still captured in the video (i.e., tattoos). Facilitators and participants were discouraged from using their names during video recording to avoid identification of the recorded participant.

All participants received concurrent verbal feedback in all groups as deemed appropriate by the facilitator during all encounters as per standard practice. This feedback was tailored to the needs of the participant at the discretion of the facilitator. Questions clarifying concurrent feedback and video modeling examples were permitted throughout the intervention period. A maximum of ten minutes was given to perform each IV encounter, which started at the beginning of skin puncture. Multiple attempts were permitted per encounter. The trial period ended with successful cannulation, as defined by: i) free blood backflow into the cannula; ii) securing of the saline-locked IV tubing; and iii) patency confirmed by saline flush with lack of obvious extravasation. A failed encounter was determined by the inability to attain all three criteria in the same attempt after ten minutes, or if more than three attempts were required in one encounter. A single attempt was defined by skin puncture and removal of needle from skin. All encounters were done in a private, closed-door room.

Upon completion of each encounter, all participants were brought to a video review room. After the initial instruction and baseline IV encounter, each participant was randomly assigned by sealed envelope to one of two groups for the study by a study investigator: VM alone or VM+VF. Throughout the study, the participants remained assigned to these groups. The instructional facilitator was blinded to group assignment. Headphones were provided to the participants to facilitate audio review. Participants assigned to the VM alone group reviewed a video of an expert performing IV cannulation with audio narration explaining the steps (ESM eVideo). Participants assigned to the VM+VF group reviewed the expert video, alongside their own most recent encounter at IV cannulation, which included the instructor feedback. Additional feedback was not given during video review. All participants in both groups were given ten minutes for the total video review period, even though the VM+VF group had more video material to review. They were allowed to fast forward or rewind any portion of the video review if so desired.

Participants then returned to the procedural room and performed IV cannulation again, with their new encounter video recorded. Upon completion of this second encounter, participants returned to the video review room. The VM alone group once again reviewed the expert video, whereas the VM+VF group reviewed their newest recorded encounter alongside the expert video. This process was repeated to a total of four IV cannulation encounters and three audio-video review periods. Participants were free to withdraw or stop the intervention at any point in time. If participants felt lightheaded or unwell during the procedure, they were withdrawn by the investigators, and were welcome to be rescheduled if they so volunteered at a later date. Their respective partners were rescheduled at a later date when feasible.

Post-Test Interview and Qualitative Analysis

Upon completion of four IV cannulation encounters and three audio-video review periods, each participant was individually interviewed by the primary author (J.Y.), and interviews were audio-recorded. Participants were asked the following questions to initiate and guide the interview in an open-ended fashion: 1) What was helpful about this learning experience with the video review? 2) What was not helpful or made the learning experience difficult with the video review? 3) Do you think the video review improved your learning experience: why or why not? and 4) How satisfied are you with the feedback you have received?

Qualitative data were analyzed using thematic content analysis to identify common patterns and themes in the manifest content of responses.42 Verbal interview data were managed reflexively and iteratively43 by: 1) audiotaping the interview and taking notes; 2) reflective journaling immediately post-interview; 3) listening to the audiotape and amending/revising field notes and observations; 4) preliminary content analysis; 5) secondary content analysis; and 6) thematic review. Themes were established by the primary author, then reviewed and refined through iterative discussions with co-authors (J.O.).

Video Analysis

The video recordings were analyzed by study authors blinded to group allocation after data were collected. To help preserve anonymity and avoid bias, no audio was reviewed for the purposes of analysis. The time stamp for initial skin puncture was zero seconds. The time stamp at successful cannulation, or the time of abandonment if the encounter was a failure, was recorded as the end time. The maximum time of abandonment was ten minutes. Successful or failed encounters were recorded. Complications were defined as: > 1 attempts, hematoma formation, bleb formation from saline extravasation, inability to flush the cannula with saline, or removal of needle and/or cannula with no blood flow back into the hub. These were counted for each encounter and recorded.

Outcomes

The primary outcome measure was time required to perform IV cannulation during the final encounter. Secondary outcomes included the difference in time required to perform an IV cannulation encounter between baseline and final encounters within both groups, success percentage, number of IV complications (including > 1 attempts), and what participants perceived to be subjectively helpful or not with their respective video reviews.

Statistical Analysis

The study data were analyzed for normality using the Kolmogorov–Smirnov test. Data presentation included the median [interquartile range (IQR)], unless otherwise described. A convenience sample size of 34 was utilized. The primary endpoint of final encounter times was compared between groups using the Mann–Whitney test. Baseline encounter times were also compared between groups using the Mann–Whitney test. Categorical secondary outcomes (e.g., success) was analyzed with Fisher’s exact test, as two cells had an expected count < 5. Continuous secondary outcomes (final vs baseline encounter time) within each group were compared with Wilcoxon signed rank test. The between group and within group median differences (e.g., encounter time, > 1 insertion attempts, complications) were estimated with Hodges–Lehman median difference, and corresponding 95% confidence intervals (CI) were calculated. A P < 0.05 was considered statistically significant. No correction for multiplicity was performed. Statistical analysis was performed using IBM SPSS v25 (IBM Corp., Armonk, NY, USA).

Results

Thirty-four participants volunteered for this study. Nine participants’ data were not included in our analysis including two participants who were part of an initial pilot trial run with flaws in study conduction, four participants who felt lightheaded during the initial procedural intervention, and three participants who could not be scheduled. Thus, 25 participants were analyzed in our final sample: 12 participants in the VM alone group, and 13 in the VM+VF group. These participants’ baseline experience in IV cannulation is shown in Table 1. There were no meaningful demographic differences between groups. Of interest, two participants stated they had previous IV cannulation experience, but did not meet exclusion criteria of five previous IV attempts or more. One participant elaborated in the pre-test survey:

Table 1 Baseline demographics and education experience from pre-test survey
Full size table

“[During a] skills [training] day in Regina, I had the opportunity to watch someone put an IV in and try one myself.”

Quantitative Data

Primary and secondary outcomes in unpaired analysis are shown in Table 2. There were no significant differences in median time required for IV cannulation at baseline between the two groups (P = 0.70). Comparing the primary outcome, the median [IQR] time required to perform IV cannulation in the final encounter was significantly different between the VM+VF group vs VM alone group (126 [133] sec vs 345 [406] sec, respectively; median difference, 111 sec; 95% CI, 8 to 391; P = 0.02). The percentage of successful IV cannulation for the two groups (VM alone vs VM +VF) was not significantly different (P = 0.65) (Table 2). The difference in time required to perform successful IV cannulation between final and baseline encounters within the VM alone group and the VM+VF group is shown in Fig. 2.

Table 2 Analysis of primary and secondary quantitative outcomes between groups
Full size table
Fig. 2
figure 2

The difference in time required to perform an intravenous (IV) cannulation encounter for two groups (VM alone or VM+VF). The results were calculated by taking the time it took each participant to perform their baseline IV cannulation subtracted from the time it took each participant to complete their paired final cannulation. This is stated as “final − baseline”. Negative numbers represent a reduction in time required for a participant to complete the IV encounter. The medians are statistically significantly different (P = 0.04). The bold line represents the median. The box lower and upper parameters represent the first and third quartiles respectively. The upper and lower whisker represent the minimum and maximum values, respectively. VF = video feedback. VM = video modeling.* P = 0.04

Full size image

Paired-related analysis of participants attempts (i.e., final vs baseline within groups) was also performed. For the VM+VF group, the median [IQR] time to perform IV cannulation in the final encounter was lower after intervention compared with baseline values (126 [93-226] sec vs 465 [339-610] sec, respectively; median difference, 284 sec; 95% CI, 124 to 439; P = 0.002). Conversely, when these values were measured for the VM alone group, the time to perform IV cannulation in the final encounter was not significantly lower compared with baseline values (345 [131-537] sec vs 418 [365-598] sec, respectively; median difference, 63 sec; 95% CI, -67 to 260; P = 0.35).

A comparison of related samples was also performed for complications. For the VM+VF group, there was a significant reduction in the median [IQR] number of complications when comparing final to baseline encounters (0 [0-1], vs 3 [0-4], respectively; median difference, 1.5; 95% CI, 0 to 2.5; P = 0.02). There was no significant difference in the number of complications between final and baseline encounters within the VM alone group (2 [0-3] vs 2.5 [1.5-3.3], respectively; median difference, 0.5; 95% CI, -0.5 to 2; P = 0.17).

A further analysis was performed specifically on > 1 IV attempts within each group. In the VM+VF group, there was no significant reduction in the median [IQR] number of attempts > 1 when comparing their final encounter to their baseline encounter (0 [0-1] vs 1[ 0-2], respectively; median difference 0.5; 95% CI, 0 to 1; P = 0.10). Similarly, there was no difference in the VM alone group when comparing their final encounter to their baseline encounter (1 [0-1.3] vs 1 [0.8-2], respectively; median difference, 0.5; 95% CI, 0 to 1; P = 0.12).

Qualitative Data

Video Modeling: Schema Solidification

Participants perceived an overwhelming value of VM in general, as it assisted with learning schema, as suggested by ten participants. Participants believed that the first and second video review periods, in conjunction with concurrent in-task facilitator instruction, helped to solidify the verbal instruction. It allowed an opportunity to review specific steps that were more challenging than others. As stated by two participants, VM was helpful:

“ …between the first and second (reviews)… because I could re-watch it a bunch of times and really cement it in my head…repetition was good.”

Participants found VM also allowed for a period of reflection on practice, when mentally comparing their previous encounter with the current model.

“you notice different things that you might (want to) try to improve, every time I watch the (model) video.”

Video Modeling: Limited Educational Tool

Nine participants felt that, while useful in this initial review period, repetition of the same model content had yielded reduced utility as the session proceeded. As summarized by one participant:

“I was just repeating the one part again and again…by the time it hit the third review, I was like, ‘nah, I got it’…I needed my own personal experience of it in order to learn further, didn’t think the video could offer me more at that point”

There were concerns regarding the lack of depth and detail of standardized VM, which meant it was unable to address specific learning objectives that some participants struggled with more than others. One participant stated:

“It [video modeling] doesn’t help when you’re stuck and you don’t know what you’re doing wrong… I thought I needed more guidance than just watching a video (model) could provide.”

Where some felt the video was the appropriate duration, others felt it was not slow enough. Overall, however, participants also felt any deficiencies were well-addressed during the instructor facilitation.

Video Feedback: Expert Audio-Guided Reflection on Practice

Seven participants felt strongly that audio review of the facilitator feedback in conjunction to the video review was integral to the utility of this learning method. Multiple learners felt it allowed for another opportunity to better perceive and understand what their instructors had observed, in a safe environment. Participants commented:

“it was helpful to go back… [the instructor’s] feedback was on there [in audio], so I could listen to what I was doing and what needed to be changed, so I could listen to that again, because if it’s only happening once, it washes over you, so it kind of got ingrained a little bit more.”

“I could review what I had done, without the additional stress of hurting someone (in the moment)”

Video Feedback: Inferiority to Concurrent Feedback

Some participants felt the strength of the concurrent feedback in its nature by being direct, timely, and specific, was superior to VF or VM. With each video period taking up to ten minutes, some learners felt it took too long to go through an unhelpful review. One participant felt that rather than an interrupting review, they would have found it more useful to continue practice with facilitator guidance. Being able to ask questions to clarify concepts in a timely fashion was considered valuable. One participant even stated:

“I would have preferred to just … have the one on one coaching. … I just wanted to keep practicing without the interruption [of the video review].”

Video Modeling and Video Feedback: Combined Modalities of Learning

Video modeling + VF activated multiple different modalities of learning, including kinetic, visual, and auditory, which two participants specifically found helpful.

“I got all different types of learning, I got verbally, I got … to actually do it physically, and then I got to see. So the video review was really good because I [was not] just abstract thinking ‘oh what am I doing’?”

“…it was another form of learning that I was exposed to…the visual and auditory stimulus, and then going back and doing the tactile, kinesthetic learning.”

Study Design

Some participant comments specifically addressed issues of study design. One individual found value in paired practice, where their partner was able to give them feedback on how an IV felt, and how their attempt compared with their partner’s. The pain of previous IV attempts was distracting for some. Others felt that the lack of in vivo demonstration or of VM prior to their first encounter impeded their ability to perform well. Additionally, one person felt that deviation between video model instructions and facilitator instruction distracted them from overall learning.

Discussion

This study showed that the VM+VF group required less time to perform IV cannulation than the VM alone group. Qualitative analysis suggested both visual and audio reviews of both expert and self-performance were integral to maximizing procedural learning. Participants attributed success to the ability to reflect on their previous encounter. While VM+VF was perceived as inferior to in vivo preceptorship, VM + VF provides a viable alternative where direct preceptorship is not possible.

There has been conflicting evidence in the literature regarding procedural learning using VF. Video feedback has improved epidural performance, with an anesthesiologist guiding the review.26 Conversely, visual-only video review of laryngoscopy attempts did not improve performance.44 This variability in efficacy might be explained by the lack of audio-guided expert review post-intervention, which may be a necessary component that was not previously appreciated. Review of the same concurrent verbal feedback can be received differently when reviewed after a procedural attempt. Previous studies have shown differences in learning retention when comparing in-task feedback to end-task feedback.45 Differences in stress in-task, which are not well replicated by simulated environments,46 may explain the differences in feedback processing.

Despite showing a difference in time to task completion, we did not show a difference in overall success percentage. This is not surprising and is consistent with existing reports looking at binary outcomes of success. For example there was no difference in binary-scored suturing performance of medical students with expert guided VF compared with unguided VF.31 Because small sample sizes may not be not well powered to observe for overall binary success or failure, time to completion has often been used as a surrogate for proficiency in other VF studies in skill mastery,44,47 and may better capture nuances in early learning by encompassing multiple aspects of task efficiency and performance.

The superior learning effect seen with combined VM+VF in our study may be secondary to repeated audio review, which is consistent with the principles of deliberate practice. Deliberate practice originates from the concept of attaining maximal level of performance through deliberate efforts rather than unfocused extended experience. Ericsson et al. defined the conditions under which deliberate practice occurs—i.e., it transpires when there is a focused goal, with intention to improve, with provided feedback, and with opportunity for refinement.36,48 Pusic et al. reinforce the concept of “sequential, mindful repetitions of a training task, along with immediate feedback, such that expert performance is acquired gradually”.49 The learner adjusts and makes changes relevant for improvement before engaging in the next cycle of practice. In the current study, both groups had a well-defined goal demonstrated in the expert video, as well as specific and immediate concurrent feedback. Nevertheless, where VM+VF may have been superior is by providing an additional source of audio-video feedback during video review, facilitating further self-reflection at end-task prior to the next cycle of practice. The VM group on the other hand required the learner to expend more effort self-reflecting based upon memory of the events and feedback of the prior practice cycle.

Previously, the “apprenticeship model” was the most common teaching model used for clinical procedures in undergraduate medical education,50,51,52 contributing to long work hours. Recent trends limiting resident work hours reduce direct trainee-trainer interactions, and may have significant implications on skill development. Apprenticeship is therefore supplemented with focused clinical teaching environments.51 Trainee-centred models are recommended to focus on specific learner needs from observation of a clinical interaction.51,53 The effectiveness of VM+VF compared with VM alone highlights the benefit of trainee-specific feedback. In the era of competency-based medical education, methods utilizing trainee-specific feedback, such as the one described, may enhance the effectiveness of focused clinical learning.

The strengths of this study are multifold. First, this study shows the feasibility of an easily translatable learning method. Video technology is ubiquitous, and allows a learner to maximize existing learning opportunities. Second, this study used paired analysis of participants on themselves, accounting for learner variability. Vein anatomic variability (e.g., adiposity, skin tone) was partially controlled by having participants practice on the same partner. Anatomic difficulty may even have escalated, with easier veins used first, and more difficult veins reserved for later. Third, this study protocol examined novice learners in the context of intensive, repetitive practice, in a focused and supervised workshop within a short time frame, which can easily be incorporated into a structured curriculum. Fourth, this was an embedded mixed-method study, with primary quantitative data collection and subsequent qualitative data collection that allowed robust analysis of the experience. The combination of these data show both Kirkpatrick level 1 and level 2 evidence, a well-received intervention, and skill improvement.

The limitations of this study include a small sample size that may not represent the true breadth of learners, given that keen volunteers seeking additional experience don’t necessarily represent all learners. Additionally, the effectiveness of VM+VF is less clear in high acuity low occurrence procedures, or of more complex procedures. Lastly, because of study design, the skill was not demonstrated in a high-fidelity model prior to baseline encounter, potentially rendering it unrealistically challenging. This, however, may reflect clinical encounters at the bedside, where high-fidelity model observation is not always possible.

Future research could test VM + VF in the acquisition of other complex procedural skills, or in more advanced learners. Participants suggested that skill refinement was better developed with VM+VF in later repetitions; a similar phenomenon may be observed with experienced learners looking to refine mastery of skill sets. Other future studies may include assessing for learning curves, and teaching efficacy over traditional means. Additionally, while this study has increased level 1 and level 2 evidence, a follow up study to assess for Kirkpatrick level 3 transfer evidence may be warranted.

Conclusions

In this small study, VM+VF shortened time to skill completion and reduced complications in novice learners, but did not improve success. It is a ubiquitous technology that can be immediately applied in current clinical learning environments to maximize learning and allow for later independent or mentored review.