At the 2017 CORD Academic Assembly, 30 participants will gather for the second CORD Medical Education Hack-a-thon. This challenging one day activity will explore design thinking and apply the tenets to a vexing problem in medical education: assessment!
During our session, we will be challenging a group Emergency Medicine educators from all across the nation to rethink their assessment and competency decision practices. But, we need YOUR help! And we’re asking it via the case below, modeled after the ALiEM MEdIC series (https://www.aliem.com/category/non-clinical/medic-series/) created by Teresa Chan and Brent Thoma.
We’re hoping that you can help us uncover facets of a problem facing all of us all the time:
How can we make our learner assessment meaningful for our residents?
Take a look at the following case and let us know what problems you think that you see in our case, but also share the problems you have encountered with the residency academic programs you remember (or are currently experiencing).
The Problem
“Ugh,” thought Colleen. The time had come for yet another round of clinical competency committee meetings. Colleen, a junior faculty member, had been invited to participate as a member of the committee last year. This would be her third round of meetings with the other committee members. Unfortunately, after her first meeting, Colleen began to question the process that the faculty used to make decisions about the residents progress within the milestones. The meetings typically went something like this: all of the faculty members would meet together in a closed room, and then the residents procedure logs, monthly evaluations, duty hours, and any potential complaints would be reviewed. The chair of the committee would query the members present for any concerns they had about the particular resident. Following this, they would display the milestones on the projector and the group would vote on the assigned score.
Having recently attended the CORD Academic Assembly, Colleen was concerned that they were not making valid decisions. The point was driven home during a recent clinical shift with one of her late third-year residents. A patient presented with impending respiratory arrest. Wanting to challenge her senior resident, she insisted on utilizing direct laryngoscopy instead of the favored video system. Standing at the side of the bed, Colleen watched with horror as a resident failed two attempts with direct laryngoscopy secondary to poor technique before allowing the resident to revert to their favored video system. Reviewing the residents’ milestone data later that afternoon, she found that the CCC placed him at a level 5 for PC10- Airway Management. Furthermore, on the last round of procedural evaluations, the committee determined that he was “capable of independently performing the procedures related to airway management without supervision.”
Colleen Meets with the Stakeholders
Colleen met with one of her chief residents, Michelle, to discuss her impression of the assessment system. Michelle felt the system was fair; she had not experienced any problems. When Colleen pushed about the milestones, Michelle began to fidget before admitting that she questioned her own competency with some of the skills that she was expected to do independently. She admitted that she had approached faculty with some of her concerns only to be given reassurance and unhelpful advice.
Colleen also met with the program director, Teresa. Teresa had been in the position for seven years and was responsible for the implementation of the milestones when they were released. When asked about assessment decisions, Teresa admitted that their system wasn’t perfect but it was the “best thing we could implement with limited time and budget.”
Colleen Seeks Advice
Colleen reached out to the CORD listserv for advice. Several faculty members sent articles about validity and reliability. A few even shared the tools that they were using at their sites. Despite the help, Colleen was still unsatisfied given the variety of tools and assessments that were being utilized across the country.
Key Questions
- What problems have you encountered with your clinical competency committee and determining competency decisions?
- What are the problems that you can identify in the above case? Are there any specific problems that resonate with you?
- What are some solutions you have encountered with regards to assessment decisions?
Wrap Up
The solutions to the above case will be presented via the CORD Blog upon the conclusion of this year’s #CORDHacks session at #CORD17.
All characters in this case are fictitious. Any resemblance to real persons, living or dead, is purely coincidental.
The Vocal CORD 