PD101: Leaning into New Program Directors’ Challenges to Inspire a Multi-institutional PD Learning Community

Academic Pediatrics · 2025-11-01

“This Attending is Terrible!”—Teaching Trainees to Write Evaluation Comments That Make an Impact

Journal of Graduate Medical Education · 2025-06-01

The Accreditation Council for Graduate Medical Education requires that every training program must have a process for trainees to provide written, anonymous, and confidential evaluations annually of faculty members. Barriers to effectively meeting this standard include lack of trainee time and instruction in giving “upward feedback,” and uncertainty about how trainee evaluations are used. As a result, evaluation comments may lack specific, actionable items that promote improvement.1,2 This multi-institutional study sought to empower trainees to provide accurate observations of faculty and proposed a framework to make writing valuable evaluation comments easier.We designed a workshop to teach trainees a framework for composing high-quality narrative comments on professional evaluations. The workshop first reviewed the importance of high-quality narrative feedback in physician professional development, then used a case example to teach the comment framework and a small-group exercise to apply the framework by rewriting comments for simulated teaching scenarios. Comments were extracted from actual, anonymized faculty evaluations and demonstrated shortcomings typical of trainee comments (ie, nonspecific, inflammatory, or lacking solutions); background scenarios were crafted to provide context for each comment. Using the Kirkpatrick Model of learning evaluation, we assessed application of the framework and learner satisfaction following the workshop.A pilot workshop was delivered in 2020 at Phoenix Children’s Hospital (PCH). Reworded comments from the small-group exercise were collected and analyzed by 2 separate reviewers and scored from 0 to 5, based on how many elements of the framework were incorporated. Results demonstrated that this approach was feasible and successful in teaching this skill. In spring/summer 2023, the workshop was held again at PCH and separately at University of Wisconsin (UW) for medical students, pediatric residents, and fellows. Workshop participants completed a survey that was analyzed using Likert scores frequencies. Open-ended comments were grossly reviewed for themes. The study was deemed exempt by the institutional review boards of both institutions, and a data-usage agreement enabled data to be shared.During the 2020 pilot small-group exercise, participants (N=29) readily applied the 5-element framework on comment rewrites (Box). The most commonly incorporated element was 3 (26 of 29; 90%), followed by 2 (24 of 29; 83%) and 5 (21 of 29; 72%). Elements 1 (19 of 29; 66%) and 4 (16 of 29; 55%) were less frequently incorporated and led to modifications in workshop content. Approximately 65 trainees at PCH and 40 trainees at UW attended the 2023 workshops. Sixty-six of 105 total participants (63%) completed the post-workshop survey from both sites: 18 of 66 (27%) medical students, 14 of 66 (21%) residents, and 8 of 66 (12%) fellows. Just over half (34 of 66) had received prior training on how to write comments on professional evaluations. Regarding specific elements of the workshop, 44 of 59 (75%) respondents found the framework to compose effective comments to be “extremely/very valuable” and 36 of 59 (61%) found the activity of rewriting comments based on the scenario to be “extremely/very valuable.”Attendees commented that the framework can “decrease the cognitive load of providing oral and written feedback,” and specifically noted that the use of subjunctive tense alongside categories to reference when crafting comments (ie, physician as leader, clinician) “helps minimize the mental block often encountered when trying to come up with specific feedback.” Areas for workshop improvement include adding descriptions of how programs utilize trainee feedback to make changes and tips for peer-to-peer feedback. Additionally, participants recommended a similar workshop be offered to faculty to improve their narrative comments for trainees.Our study shows a workshop teaching a framework to improve trainees’ comments on faculty evaluations is an effective, feasible, and acceptable way to teach this skill for all levels of learners and specialties. Next, we plan to investigate if better-composed comments truly change faculty behavior and improve teaching.

Leveraging the Electronic Health Record to Measure Resident Clinical Experiences and Identify Training Gaps: Development and Usability Study

JMIR Medical Education · 2024-08-19 · 2 citations

Background: Competence-based medical education requires robust data to link competence with clinical experiences. The SARS-CoV-2 (COVID-19) pandemic abruptly altered the standard trajectory of clinical exposure in medical training programs. Residency program directors were tasked with identifying and addressing the resultant gaps in each trainee's experiences using existing tools. Objective: This study aims to demonstrate a feasible and efficient method to capture electronic health record (EHR) data that measure the volume and variety of pediatric resident clinical experiences from a continuity clinic; generate individual-, class-, and graduate-level benchmark data; and create a visualization for learners to quickly identify gaps in clinical experiences. Methods: This pilot was conducted in a large, urban pediatric residency program from 2016 to 2022. Through consensus, 5 pediatric faculty identified diagnostic groups that pediatric residents should see to be competent in outpatient pediatrics. Information technology consultants used International Classification of Diseases, Tenth Revision (ICD-10) codes corresponding with each diagnostic group to extract EHR patient encounter data as an indicator of exposure to the specific diagnosis. The frequency (volume) and diagnosis types (variety) seen by active residents (classes of 2020-2022) were compared with class and graduated resident (classes of 2016-2019) averages. These data were converted to percentages and translated to a radar chart visualization for residents to quickly compare their current clinical experiences with peers and graduates. Residents were surveyed on the use of these data and the visualization to identify training gaps. Results: Patient encounter data about clinical experiences for 102 residents (N=52 graduates) were extracted. Active residents (n=50) received data reports with radar graphs biannually: 3 for the classes of 2020 and 2021 and 2 for the class of 2022. Radar charts distinctly demonstrated gaps in diagnoses exposure compared with classmates and graduates. Residents found the visualization useful in setting clinical and learning goals. Conclusions: This pilot describes an innovative method of capturing and presenting data about resident clinical experiences, compared with peer and graduate benchmarks, to identify learning gaps that may result from disruptions or modifications in medical training. This methodology can be aggregated across specialties and institutions and potentially inform competence-based medical education.

Implementing a Pediatric Resident Physician/Bedside Nurse Shadow Program to Improve Interprofessional Collaboration and Understanding, and to Transform Practice

Health Interprofessional Practice and Education · 2023-07-25

INTRODUCTION Resident physician/nurse collaboration is essential for patient safety and optimal clinical outcomes. Interprofessional shadowing experiences can facilitate understanding of team roles; however, enhanced understanding may not transform individual practices leading to workplace improvement. This educational innovation utilized a resident/nurse shadow program to raise awareness of interprofessional roles and as a needs assessment to identify and educate about specific system-wide standards that can lead residents to transform their practices to improve nurse workflow. METHODS From 2018-2020, 44 first-year pediatric residents at a free-standing children’s hospital shadowed nine nurse preceptors. Each nurse received an orientation and checklist of topics to cover. Residents observed nursing responsibilities including admissions, line placement and medication administration. All participants completed pre-post surveys. RESULTS Post-surveys demonstrated a statistically significant improvement (p<.00001) in Likert scores of residents’ self-rated understanding of nursing cares and workflow. The experience uncovered three system-wide scheduling standards where resident modification of ordering practices could optimize nursing workflow: timing of morning lab draws, new medication administration, and delivery of dietary formula. The shadow program was modified to include education in these three areas and participants had statistically significant (p<.01) increased knowledge of timing of morning labs and new medication administration (35% and 39% improvement, respectively). Nurse understanding of resident workflow was not measured due to limited data.DISCUSSION A resident/nurse shadow program successfully improved resident awareness of nursing roles and was an innovative way to identify specific areas of workflow improvement. Program modifications are required to create a true, bidirectional, resident/nurse interprofessional shadow experience.

Eureka! Mining the Electronic Health Record to Measure Resident Physicians’ Clinical Experiences and Identify Training Gaps (Preprint)

2023

• Medical education
• Medicine
• Family medicine

<sec> <title>BACKGROUND</title> Competence-based medical education requires robust data to link competence with clinical experiences. The SARS-CoV-2 pandemic abruptly altered the standard trajectory of clinical exposure in medical training programs. Residency program directors were tasked with identifying and addressing the resultant gaps in each trainee’s experiences using existing tools. </sec> <sec> <title>OBJECTIVE</title> To demonstrate a feasible and efficient method to capture electronic health record (EHR) data that measures the volume and variety of pediatric resident clinical experiences from a continuity clinic; generate individual-, class-, and graduate-level benchmark data; and create a visualization for learners to quickly identify gaps in clinical experiences. </sec> <sec> <title>METHODS</title> This study was conducted in a large, urban pediatric residency program from 2016-2022. Through consensus, five pediatric faculty identified diagnostic groups pediatric residents should see to be competent in outpatient pediatrics. Institution business analysts used ICD-10 codes corresponding with each diagnostic group to extract EHR patient encounter data as an indicator of exposure to the specific diagnosis. The frequency (volume) and diagnosis types (variety) seen by active residents (classes of 2020-2022) were compared to class and graduated resident (classes of 2016-2019) averages. These data were converted to percentages and translated to a radar chart visualization for residents to quickly compare their current clinical experiences to peers and graduates. Residents were surveyed on utility of these data and the visualization to identify training gaps. </sec> <sec> <title>RESULTS</title> Patient encounter data about clinical experiences for 102 residents (N=52 graduates) were extracted. Active residents (N=50) received data reports with radar graphs biannually: three for the classes of 2020 and 2021 and two for the class of 2022. Radar charts distinctly demonstrated gaps in diagnoses exposure compared to classmates and graduates. Residents found the visualization useful in setting learning goals. </sec> <sec> <title>CONCLUSIONS</title> This pilot describes an innovative method of capturing and presenting data about resident clinical experiences, compared to peer and graduate benchmarks, to identify learning gaps that may result from disruptions or modifications in medical training. This methodology can be aggregated across specialties and institutions and potentially inform competence-based medical education. </sec>

Calling Consults: A Workshop to Teach Trainees Using Both Didactic and Small Group–Based Learning

ATS Scholar · 2020 · 1 citations

• Political Science
• Medical education
• Psychology

Training learners on the key components and etiquette of calling consults is crucial for the development of effective communication among providers. This training is generally lacking from undergraduate medical education; thus, it is important to provide education in calling consults during residency and fellowship.

Service: Retracted and Reframed

Journal of Graduate Medical Education · 2020

• Medical education
• Medicine
• Psychology

Do you feel that your education is compromised by service?As a residency program director, this is the question on the annual Accreditation Council for Graduate Medical Education resident/fellow survey I dread the most. When I read personal statements from applicants, they all speak of choosing medicine as a path of providing service to people or for a higher purpose. When did the term “service” become something detrimental that interferes with our education and ambitions?It reminds me of a time when I was in medical school. I was on my first rotation as a third-year medical student, sentenced to 4 weeks on the “malignant” pediatric surgery block, and quickly feeling as I anticipated—like the lowest rung on the ladder. No one seemed to care who I was, where I was, or what I was learning; but like many experiences, sometimes our greatest life lessons come when least expected.The surgery service was rounding on a premature newborn in the neonatal intensive care unit. The baby not only had immature lungs, but also had a tracheoesophageal fistula and required extracorporeal membrane oxygenation. Her small room was filled with huge machines and an intricate network of tubes, wires, and monitors, whose beeps and alarms served as a constant reminder of the technology that was keeping her alive. When she began to decompensate, the surgery team arrived to repair her fistula at the bedside.I stood at the periphery watching the instrument tables roll in and the surgeons gown up and transform the patient's room into a functioning operating room. They began their meticulous procedure. All was quiet until I heard some scuffling and an exchange of words, followed by the chief surgical resident moving back from the table and angrily stomping away.“You!” the surgeon said as he looked in my general direction. Was he talking to me? I had worked with him for nearly 4 weeks. He didn't know my name. I knew his name—in fact I remember it to this day. “Put on a gown and get in here.” I looked around to see if he was talking to someone else. “I need you in this space, to retract.” A nurse helped me on with a gown and pushed me forward until I saw the space I was supposed to be in—it was tiny, barely a square foot, and surrounded by a web of wires. “See all these wires?” the surgeon barked, “If you move an inch, this baby will die.” I squeezed into the corner, a miniature retractor in hand, my arm twisted completely behind my back, and did what was asked of me. At one moment of weakness, I craned my neck to sneak a peek at the surgical field, and the surgeon snapped, “Stay still—I didn't choose you to do this for the experience! I chose you because you're skinny!”So this is what it came down to: struggling through organic chemistry, memorizing the Krebs cycle, hospital volunteer work, student loans, the noxious smell of formaldehyde, board exams—just so I could be this anonymous skinny person facing a wall and retracting. It was grunt work. It was a scut. My education had been “compromised by service.”I have learned since then that most days in medicine you are valued for your brain, your thoughts, the way you link basic science and clinical clues and elements of your exam to come up with a diagnosis and a plan. Some days you are needed for your ability to reduce a fracture and relieve pain or to supervise and teach a trainee how to do it independently. Some days your job is to tell a family that their child has a terrible disease and to ready them for the long course ahead, and some days it is to tell them their tests are normal, and they will be fine. Those are the things we have trained for, that make us feel the true weight of our worth in this field.But there are other times when you are needed to run a CD down to radiology or complete FMLA paperwork or fax prescriptions to a pharmacy or redo a home health order for the tenth time since you still didn't get it right. It's not the glory of the job. It's not even close. But when you take a step back and see how that piece fits into the bigger picture, you may have indirectly avoided an additional x-ray, or prevented a mom from losing her job while her child recovered from an illness, or decreased the pharmacy wait time for a family that is already exhausted from a long hospital stay. Or you may have helped clear the field for a surgeon to sew up a tiny fistula. We are providing a service, in the most noble and altruistic definition of the word.I often think about this experience, my initial disappointment as an insignificant student holding a retractor, reframed over time, as pride in being able to help a fragile infant survive. These moments of retracting and faxing shrink beside the opportunity to witness bedside miracles, the chance to play a part, no matter how small, in the health of a child, and the privilege of being a part of this profession dedicated to service.

The Effect of Faculty Coaching on Resident Attitudes, Confidence, and Patient-Rated Communication: A Multi-Institutional Randomized Controlled Trial

Academic Pediatrics · 2018-10-24 · 31 citations

How Residents Learn From Patient Feedback: A Multi-Institutional Qualitative Study of Pediatrics Residents' Perspectives

Journal of Graduate Medical Education · 2018-03-20 · 29 citations

BACKGROUND: Residents may view feedback from patients and their families with greater skepticism than feedback from supervisors and peers. While discussing patient and family feedback with faculty may improve residents' acceptance of feedback and learning, specific strategies have not been identified. OBJECTIVE: We explored pediatrics residents' perspectives of patient feedback and identified strategies that promote residents' reflection on and learning from feedback. METHODS: In this multi-institutional, qualitative study conducted in June and July 2016, we conducted focus groups with a purposive sample of pediatrics residents after their participation in a randomized controlled trial in which they received written patient feedback and either discussed it with faculty or reviewed it independently. Focus group transcripts were audiorecorded, transcribed, and analyzed for themes using the constant comparative approach associated with grounded theory. RESULTS: Thirty-six of 92 (39%) residents participated in 7 focus groups. Four themes emerged: (1) residents valued patient feedback but felt it may lack the specificity they desire; (2) discussing feedback with a trusted faculty member was helpful for self-reflection; (3) residents identified 5 strategies faculty used to facilitate their openness to and acceptance of patient feedback (eg, help resident overcome emotional responses to feedback and situate feedback in the context of lifelong learning); and (4) residents' perceptions of feedback credibility improved when faculty observed patient encounters and solicited feedback on the resident's behalf prior to discussions. CONCLUSIONS: Discussing patient feedback with faculty provided important scaffolding to enhance residents' openness to and reflection on patient feedback.

Motivations for Volunteer Academic Faculty Involvement in Programmatic Administrative Tasks

Academic Pediatrics · 2018-07-01