How competent are emergency medicine interns for level 1 milestones: who is responsible?

The Next Accreditation System (NAS) of the Accreditation Council for Graduate Medical Education (ACGME) includes the implementation of developmental milestones for each specialty. The milestones include five progressively advancing skill levels, with Level 1 defining the skill level of a medical student graduate, and Level 5, that of an attending physician. The goal of this study was to query interns on how well they thought their medical school had prepared them to meet the proposed emergency medicine (EM) Level 1 milestones. In July 2012, an electronic survey was distributed to the interns of 13 EM residency programs, asking interns whether they were taught and assessed on the proposed Level 1 milestones. Of possible participants, 113 of 161 interns responded (70% response rate). The interns represented all four regions of the country. The interns responded that the rates of Level 1 milestones they had been taught ranged from 61% for ultrasound to 98% for performance of focused history and physical examination. A substantial number of interns (up to 39%) reported no instruction on milestones such as patient disposition, pain management, and vascular access. Graduating medical students were less commonly assessed than taught the milestones. Skills with technology, including "explain the role of the electronic health record and computerized physician order entry," were assessed for only 39% of interns, and knowledge (USMLE) and history and physical were assessed in nearly all interns. Disposition, ultrasound, multitasking, and wound management were assessed less than half of the time. Many entering EM interns may not have had either teaching or assessment on the knowledge, skills, and behaviors making up the Level 1 milestones expected for graduating medical students. Thus, there is a potential gap in the teaching and assessment of EM interns. Based on these findings, it is unclear who will be responsible (medical schools, EM clerkships, or residency programs) for ensuring that medical students entering residency have achieved Level 1 milestones.

Santen SA ，Rademacher N ，Heron SL ，Khandelwal S ，Hauff S ，Hopson L ... -  《-》

Programmatic assessment of level 1 milestones in incoming interns.

With the Accreditation Council for Graduate Medical Education (ACGME) Next Accreditation System, emergency medicine (EM) residency programs will be required to report residents' progress through the EM milestones. The milestones include five progressively advancing skill levels, with Level 1 defining the skill set of a medical school graduate and Level 5, that of an attending physician. The ACGME stresses that multiple forms of assessment should be used to ensure capture of the multifaceted competencies. The objective of this study was to determine the feasibility and results of programmatic assessment of Level 1 milestones using multisource assessments for incoming EM interns in July. The study population was interns starting in 2012 and 2013. Interns' Level 1 milestone assessment was done with four distinct methods: 1) the postgraduate orientation assessment (POA) by the Graduate Medical Education Office for all incoming interns (this multistation examination covers nine of the EM milestones and includes standardized patient cases, task completion, and computer-based stations); 2) direct observation of patient encounters by core faculty using a milestones-based clinical skills competency checklist; 3) the global monthly assessment at the end of the intern orientation month that was updated to reflect the EM milestones; and 4) faculty assessment during procedural labs. These occurred during the July orientation month that included the POA, clinical shifts, didactic sessions, and procedure labs. In the POA, interns were competent in 48% to 93% of the milestones assessed. Overall, competency was 70% to 80%, with low scores noted in aseptic technique (patient care Milestone 13 [PC13]) and written and verbal hand-off (interpersonal communications skills [ICS]2). In overall communication, 70% of interns demonstrated competency. In excess of 80% demonstrated competency in critical values interpretation (PC3), informed consent (PC9), pain assessment (PC11), and geriatric functional assessment (PC3). On direct observation, almost all Level 1 milestones were achieved (93% to 100%); however, only 78% of interns achieved competency in pharmacotherapy (PC5). On global monthly evaluations, all interns met Level 1 milestones. A multisource assessment of EM milestones is feasible and useful to determine Level 1 milestones achievement for incoming interns. A structured assessment program, used in conjunction with more traditional forms of evaluation such as global monthly evaluations and direct observation, is useful for identifying deficits in new trainees and may be able inform the creation of early intervention programs.

Hauff SR ，Hopson LR ，Losman E ，Perry MA ，Lypson ML ，Fischer J ，Santen SA ... -  《-》

Practicing emergency physicians report performing well on most emergency medicine milestones.

The Accreditation Council for Graduate Medical Education's Next Accreditation System endorsed specialty-specific milestones as the foundation of an outcomes-based resident evaluation process. These milestones represent five competency levels (entry level to expert), and graduating residents will be expected to meet Level 4 on all 23 milestones. Limited validation data on these milestones exist. It is unclear if higher levels represent true competencies of practicing emergency medicine (EM) attendings. Our aim was to examine how practicing EM attendings in academic and community settings self-evaluate on the new EM milestones. An electronic self-evaluation survey outlining 9 of the 23 EM milestones was sent to a sample of practicing EM attendings in academic and community settings. Attendings were asked to identify which level was appropriate for them. Seventy-nine attendings were surveyed, with an 89% response rate. Sixty-one percent were academic. Twenty-three percent (95% confidence interval [CI] 20%-27%) of all responses were Levels 1, 2, or 3; 38% (95% CI 34%-42%) were Level 4; and 39% (95% CI 35%-43%) were Level 5. Seventy-seven percent of attendings found themselves to be Level 4 or 5 in eight of nine milestones. Only 47% found themselves to be Level 4 or 5 in ultrasound skills (p = 0.0001). Although a majority of EM attendings reported meeting Level 4 milestones, many felt they did not meet Level 4 criteria. Attendings report less perceived competence in ultrasound skills than other milestones. It is unclear if self-assessments reflect the true competency of practicing attendings. The study design can be useful to define the accuracy, precision, and validity of milestones for any medical field.

Peck TC ，Dubosh N ，Rosen C ，Tibbles C ，Pope J ，Fisher J ... -  《-》

The development of the emergency medicine milestones.

The Accreditation Council for Graduate Medical Education (ACGME) has outlined its "Next Accreditation System" (NAS) that will focus on resident and residency outcome measurements. Emergency medicine (EM) is one of seven specialties that will implement the NAS beginning July 2013. All other specialties will follow in July 2014. A key component of the NAS is the development of assessable milestones, which are explicit accomplishments or behaviors that occur during the process of residency education. Milestones describe competencies more specifically and identify specialty-specific knowledge, skills, attitudes, and behaviors (KSABs) that can be used as outcome measures within the general competencies. The ACGME and the American Board of Emergency Medicine (ABEM) convened an EM milestone working group to develop the EM milestones. This article describes the development, use within the NAS, and challenges of the EM milestones.

Beeson MS ，Carter WA ，Christopher TA ，Heidt JW ，Jones JH ，Meyer LE ，Promes SB ，Rodgers KG ，Shayne PH ，Swing SR ，Wagner MJ ... -  《-》

The emergency medicine milestones: a validation study.

The Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties sought to define milestones for skill and knowledge acquisition during residency training. Milestones are significant objective observable events. The milestones are listed within a structure that is derived from the ACGME general competencies. Major groups of milestones are called "subcompetencies." The original 24 subcompetencies containing 255 milestones for emergency medicine (EM) were developed through a multiorganizational group representing most EM stakeholder groups. To assure that the milestones reflected EM resident progress throughout training, the EM Milestones Working Group (EM MWG) sought to validate the individual milestones. A computer-based survey was sent to all EM residency programs. The survey period began on April 30, 2012, and concluded on May 15, 2012. Respondents were asked to assign each milestone to a specific level of skill or knowledge acquisition. These levels ranged from a beginning resident to an accomplished clinician. There were two different forms that divided the milestones into two groups of 12 subcompetencies each. Surveys were randomly assigned to programs. There were five respondents (the program director and four key faculty) requested from each of the 159 residences. There were responses from 96 programs (60.4%). Of the 795 survey recipients, 28 were excluded due to prior exposure to the EM milestones. Of the remaining 767 potential respondents, 281 completed the survey (36.6%) within a 16-day period. Based on the survey results, the working group adjusted the milestones in the following ways: one entire subcompetency (teaching) was eliminated, six new milestones were created, 34 milestones were eliminated, 26 milestones were reassigned to a lower level score, and 20 were reassigned to a higher level. Nineteen milestones were edited to provide greater clarity. The final result was 227 discrete milestones among 23 subcompetencies. The EM milestones were validated through a milestone assignment process using a computer-based survey completed by program directors and key faculty. Milestones were revised in accordance with the results to better align assignment within each performance level.

Korte RC ，Beeson MS ，Russ CM ，Carter WA ，Emergency Medicine Milestones Working Group ，Reisdorff EJ ... -  《-》