Video self-assessment of basic suturing and knot tying skills by novice trainees.

Self-assessment is important to learning but few studies have utilized video self-assessment of basic surgical skills. We compared a video self-assessment of suturing and knot tying skills by novice trainees to the assessment by a senior attending surgeon. Sixteen senior medical students and 7 beginner surgical interns were video-recorded while performing five suturing and knot tying tasks. All videos were analyzed using an objective structured assessment of technical skills (OSATS) metrics (1-5 scale; 1 = novice, 5 = expert). Video self-assessment was carried out within 4 weeks of an instructional session and subsequently by one senior surgery instructor (blinded to the individual). Both a Global score and total combined OSATS scores were analyzed. Total possible OSATS scores were: interrupted suture-30, subcuticular closure-30, one and two-handed knot tying-25 each, tying in a restricted space 20; maximum combined score-130 points). Confidence levels in performing the tasks pre-test and the value of video self-assessment were rated on a 1-5 Likert scale (1 = low and 5 = high). Data are mean±SD and statistical significance was evaluated using Friedman's test. Self-assessment scoring was significantly higher than the assessment by a senior instructor for three tasks by global score and all five tasks by combined OSATS score (self-assessment 71.8±16.7 vs attending assessment 56.7±11.0, p = 0.007). Mean self-assessment Global scores ranged from 2.5 to 2.8 for all tasks performed compared to 1.8-2.3 for attending surgeon assessment (p<0.05). Confidence levels demonstrated no correlation to performance speed or proficiency. The video self-assessment was rated as a highly valuable (mean 4.3±0.8) component to skills training. Novice trainees over-estimate their basic technical skills performance compared to the assessment by a senior surgeon. Video self-assessment may be a valuable addition to a pre-residency and surgical internship preparatory curriculum in basic suturing and knot tying.

Hu Y ，Tiemann D ，Michael Brunt L 《-》

Developing an Objective Structured Assessment of Technical Skills for Laparoscopic Suturing and Intracorporeal Knot Tying.

To develop a teaching and assessment tool for laparoscopic suturing and intracorporeal knot tying. We designed an Objective Structured Assessment of Technical Skills (OSATS) tool that includes a procedure-specific checklist (PSC) and global rating scale (GRS) to assess laparoscopic suturing and intracorporeal knot-tying performance. Obstetrics and Gynecology residents at our institution were videotaped while performing a laparoscopic suturing and intracorporeal knot-tying task at a surgical simulation workshop. A total of 2 expert reviewers assessed resident performance using the OSATS tool during live performance and 1 month later using the videotaped recordings. OSATS scores were analyzed using the Wilcoxon rank-sum test. Data are presented as median scores (interquartile range [IQR]). Intrarater and interrater reliabilities were assessed using a Spearman correlation and are presented as an r correlation coefficient and p value. An r ≥ 0.8 was considered as a high correlation. After testing, we received feedback from residents and faculty to improve the OSATS tool as part of an iterative design process. In all, 14 of 21 residents (66.7%) completed the study, with 9 junior residents and 5 senior residents. Junior residents had a lower score on the PSC than senior residents did; however, this was not statistically significant (median = 6.0 [IQR: 4.0-10.0] and median = 13.0 [IQR: 10.0-13.0]; p = 0.09). There was excellent intrarater reliability with our OSATS tool (for PSC component, r = 0.88 for Rater 1 and 0.93 for Rater 2, both p < 0.0001; for GRS component, r = 0.85 for Rater 1 and 0.88 for Rater 2, both p ≤ 0.0002). The PSC also has high interrater reliability during live evaluation (r = 0.92; p < 0.0001), and during the videotape scoring with r = 0.77 (p = 0.001). Our OSATS tool may be a useful assessment and teaching tool for laparoscopic suturing and intracorporeal knot-tying skills. Overall, good intrarater reliability was demonstrated, suggesting that this tool may be useful for longitudinal assessment of surgical skills.

Chang OH ，King LP ，Modest AM ，Hur HC ... -  《-》

Year one outcomes assessment of a masters suturing and knot-tying program for surgical interns.

We implemented and evaluated the outcomes of a proficiency-based program to enhance basic suturing and knot-tying skills in surgical interns. A suturing and knot-tying "masters program" was implemented for 9 surgical interns. Assessment consisted of timed and videotaped technical performance (TP) of 6 suturing (simple interrupted, subcuticular, and vertical mattress) and tying tasks (1-handed, 2-handed, and tie on pass) preinstruction (baseline) and at 4 months. Feedback and remediation were done after 4 months, and interns were retested until goals were met. TP was scored on a 1-5 scale (1 = novice, 3 = proficient, and 5 = expert). The results were compared with 6 end of PGY2 year residents and rotation evaluations, case numbers, and American Board of Surgery In-Training Examination (ABSITE) scores. The data are mean ± standard deviation (SD); statistical analysis was by 1-way ANOVA, Kruskal-Wallis test, and Pearson correlation. The total combined task times improved significantly from initial (1289 ± 301 seconds) to final assessment (770 ± 139 seconds, p = 0.0003) and between 4-month assessment (1092 ± 253 seconds) and final assessment (p = 0.0237), but not baseline to 4-month assessment (p = 0.213). Final TP scores were improved significantly (baseline, 1.87 ± 0.81; 4-month assessment, 2.63 ± 0.75; and final assessment 3.51 ± 0.39 [p < 0.001]. All interns reached proficiency on 6/6 tasks but not until 10.1 months (range, 4-12 months) and 2.4 assessment sessions (range, 1-3). No significant differences in final total task times or TP scores were found between interns (770 ± 139 seconds and TP, 3.51 ± 0.39) and end of PGY-2 residents (677 ± 109 seconds and TP, 3.6 ± 0.43). The total task time and rotation technical comments were associated significantly, but TP scores, case numbers, and ABSITE scores were not. Training, feedback, and remediation are necessary to reach proficiency in basic suturing and knot tying. This program provides an objective assessment of interns' skills not discernible by conventional measures.

Schill M ，Tiemann D ，Klingensmith ME ，Brunt LM ... -  《-》

An objective assessment tool for basic surgical knot-tying skills.

To determine if a knot-tying checklist can provide a valid score and if the checklist can be used by novice surgeons in a reliable manner. This study was conducted at the Surgical Skills Center at the University of California, San Francisco. A knot-tying checklist was developed from a kinesthetic knot-tying curriculum. Novice (67 first-year medical students) and experienced surgeons (8 residents postgraduate year 3 and higher and 2 attending physicians) were videotaped performing 4 knot-tying tasks, and the videotapes were rated with a global score and a checklist by interns (n = 3) and experienced (n = 3) surgeons. Both interns and experienced surgeons can use the knot-tying checklist with acceptable reliabilities (>0.8 with 3 raters). The checklist is able to differentiate between novice and experienced surgeons, when used by both interns and experienced raters. The expert knot-tying score correlated with the global score overall (r = 0.88) and for each task (r was 0.82 for task 1, 0.85 for task 2, 0.80 for task 3, and 0.81 for task 4). The knot-tying checklist provides a valid score for basic surgical knot-tying and can be used by novice and experienced raters. Its use supports peer assessment of performance in a surgical skills laboratory setting.

Huang E ，Vaughn CJ ，Chern H ，O'Sullivan P ，Kim E ... -  《-》

Accelerated skills preparation and assessment for senior medical students entering surgical internship.

Brunt LM ，Halpin VJ ，Klingensmith ME ，Tiemann D ，Matthews BD ，Spitler JA ，Pierce RA ... -  《-》