Eye tracking for skills assessment and training: a systematic review.

The development of quantitative objective tools is critical to the assessment of surgeon skill. Eye tracking is a novel tool, which has been proposed may provide suitable metrics for this task. The aim of this study was to review current evidence for the use of eye tracking in training and assessment. A systematic literature review was conducted in line with PRISMA guidelines. A search of EMBASE, OVID MEDLINE, Maternity and Infant Care, PsycINFO, and Transport databases was conducted, till March 2013. Studies describing the use of eye tracking in the execution, training or assessment of a task, or for skill acquisition were included in the review. Initial search results returned 12,051 results. Twenty-four studies were included in the final qualitative synthesis. Sixteen studies were based on eye tracking in assessment and eight studies were on eye tacking in training. These demonstrated feasibility and validity in the use of eye tracking metrics and gaze tracking to differentiate between subjects of varying skill levels. Several training methods using gaze training and pattern recognition were also described. Current literature demonstrates the ability of eye tracking to provide reliable quantitative data as an objective assessment tool, with potential applications to surgical training to improve performance. Eye tracking remains a promising area of research with the possibility of future implementation into surgical skill assessment.

Tien T ，Pucher PH ，Sodergren MH ，Sriskandarajah K ，Yang GZ ，Darzi A ... -  《-》

Eye-tracking technology in medical education: A systematic review.

Ashraf H ，Sodergren MH ，Merali N ，Mylonas G ，Singh H ，Darzi A ... -  《-》

Automated Methods of Technical Skill Assessment in Surgery: A Systematic Review.

The goal of the current study is to systematically review the literature addressing the use of automated methods to evaluate technical skills in surgery. The classic apprenticeship model of surgical training includes subjective assessments of technical skill. However, automated methods to evaluate surgical technical skill have been recently studied. These automated methods are a more objective, versatile, and analytical way to evaluate a surgical trainee's technical skill. A literature search of the Ovid Medline, Web of Science, and EMBASE Classic databases was performed. Articles evaluating automated methods for surgical technical skill assessment were abstracted. The quality of all included studies was assessed using the Medical Education Research Study Quality Instrument. A total of 1715 articles were identified, 76 of which were selected for final analysis. An automated methods pathway was defined that included kinetics and computer vision data extraction methods. Automated methods included tool motion tracking, hand motion tracking, eye motion tracking, and muscle contraction analysis. Finally, machine learning, deep learning, and performance classification were used to analyse these methods. These methods of surgical skill assessment were used in the operating room and simulated environments. The average Medical Education Research Study Quality Instrument score across all studies was 10.86 (maximum score of 18). Automated methods for technical skill assessment is a growing field in surgical education. We found quality studies evaluating these techniques across many environments and surgeries. More research must be done to ensure these techniques are further verified and implemented in surgical curricula.

Levin M ，McKechnie T ，Khalid S ，Grantcharov TP ，Goldenberg M ... -  《-》

Basic Surgical Skill Retention: Can Patriot Motion Tracking System Provide an Objective Measurement for it?

Knot tying is a fundamental skill that surgical trainees have to learn early on in their training. The aim of this study was to establish the predictive and concurrent validity of the Patriot as an assessment tool and determine the skill retention in first-year surgical trainees after 5 months of training. First-year surgical trainees were recruited in their first month of the training program. Experts were invited to set the proficiency level. The subjects performed hand knot tying on a bench model. The skill was assessed at baseline in the first month of training and at 5 months. The assessment tools were the Patriot electromagnetic tracking system and Objective Structured Assessment of Technical Skills (OSATS). The trainees' scores were compared to the proficiency score. The data were analyzed using paired t-test and Pearson correlation analysis. A total of 14 first-year trainees participated in this study. The time taken to complete the task and the path length (PL) were significantly shorter (p = 0.007 and p = 0.0085, respectively) at 5 months. OSATS scoring showed a significant improvement (p = 0.0004). There was a significant correlation between PL and OSATS at baseline (r = -0.873) and at Month 5 (r = -0.774). In all, 50% of trainees reached the proficiency PL at baseline and at Month 5. Among them, 3 trainees improved their PL to reach proficiency and the other 3 trainees failed to reach proficiency. The parameters from the Patriot motion tracker demonstrated a significant correlation with the classical observational assessment tool and were capable of highlighting the skill retention in surgical trainees. Therefore, the automated scoring system has a significant role in the surgical training curriculum as an adjunct to the available assessment tool.

Shaharan S ，Nugent E ，Ryan DM ，Traynor O ，Neary P ，Buckley D ... -  《-》

A systematic review of assessment of skill acquisition and operative competency in vascular surgical training.

The aim of this systematic review is to describe the literature and assessment tools evaluating vascular surgical operative performance that could potentially be used for the assessment of educational outcomes applicable to the Milestone Project and the Next Accreditation System. A systematic review of PubMed/MEDLINE, EMBASE, PsycINFO, and key journals from 1985 to 2013 was performed to identify English-language articles describing assessment of vascular surgical skills and competence. Qualifying studies were abstracted for data concerning study aims, study and assessment setting, skills measured, and metrics used to determine competency. Strengths, weaknesses, and psychometric robustness of the assessment tools were determined. The literature search identified 617 citations. After title and abstract review, 65 articles were retrieved for full-text assessment and 48 articles were included in the final review. Twenty-nine articles assessed open vascular skills; 19, endovascular skills; six, nontechnical skills; and one, teamwork skills. The majority (84%) of studies were performed in a simulated environment, four (8%) were performed in the operating room, and the remaining three were performed in both a simulated environment and an operating room. Strengths and weaknesses of assessment tools were study and assessor dependent, with none applicable to all study scenarios or procedures. The literature describing assessment tools pertinent to vascular surgery is diverse. Existing assessment tools may be relevant to individual technical skill acquisition assessment; however, an operative assessment tool relevant to vascular/endovascular surgery and generalizable to the wide spectrum of technical and nontechnical skills pertinent to vascular surgery needs to be developed, validated, and implemented to allow the practical assessment of resident readiness to operate in an unsupervised setting.

Mitchell EL ，Arora S ，Moneta GL ，Kret MR ，Dargon PT ，Landry GJ ，Eidt JF ，Sevdalis N ... -  《-》