Authors: James Ahn, MD, MHPE (University of Chicago), Christine Babcock, MD, MSc (University of Chicago), and Navneet Cheema, MD (University of Chicago)

---

Introduction:

In 2016, three years after the Self-Study was introduced through the Next Accreditation System (NAS), the Accreditation Council for Graduate Medical Education (ACGME) offered select programs the opportunity to enroll in a pilot of the Self-Study program. There is limited literature outlining the optimal implementation of the Self-Study in EM, therefore forcing any program embarking on the process to create timely de novo hypothesized best practices.  The formation of this methodology informs this article which means to function as a template to the Self-Study process for EM programs.

 

Timeline:

As a program prepares for its Self-Study, it is important to have a projected timeline for organizational purposes.  Guralnick, et al., proposes a 20-week timeline for a typical large Internal Medicine program. Our process from formation of the Self-Study Committee (SSC) to submission of the summary spanned 12 months.  During this time, we had seven SSC meetings, three stakeholder meetings, conducted an electronic survey, and communicated by email regularly.  Our pilot Self-Study visit occurred three months after our document was submitted, for a total of a 15-month process.

 

Committee Formation:

The formation of a SSC is central to a successful Self-Study process. The ACGME suggests that the Program Evaluation Committee (PEC) be an ideal starting point given the substantial overlap in their data analysis and goals. Alternatively, the program could assemble a smaller, targeted committee for more efficient meetings and agile decision-making processes.  Broader representation to include additional stakeholders, such as recent graduates, nurses, and technicians can be beneficial as they contribute significantly to the environment and outcomes of the program.  We chose a smaller committee including the program leadership, chief residents, program coordinators, and selected core faculty.  This helped to facilitate productive discussions, meeting coordination, and task completion.  We engaged broader stakeholders through supplementary discussions and surveys.

 

Developing Aims:

An essential goal of the Self-Study is to develop program aims that are intended to describe the culture, product, and vision of the program.  They are in essence the core of what the program is and aspires to be when answering the question: “What type of physicians do we produce?”.   First, we had a facilitated idea-generating session at a residency-wide meeting.  We developed a word cloud from comments during that session that helped us focus our aims on recurring themes.  We next conducted a brief focus group during a faculty meeting.  Lastly, we performed a qualitative analysis on results from these focus groups and an alumni survey to triangulate our results.  This multi-pronged approach allowed for an iterative process of aim derivation that was reflective of our program.

 

Opportunities and Threats (SWOT analysis):

While the SWOT analysis has been a successful tool in the business world for decades, its use is newer in education.   It encompasses the process of identifying strengths, weaknesses, opportunities, and threats to the program. To gather data for our SWOT analysis, we had the chief residents hold a resident only meeting to facilitate candid feedback regarding the program.  Similarly, we had senior faculty members serve as objective facilitators and lead in-person and email discussions with the entire faculty to compile parallel data.  These two processes served as the basis for the SSC to finalize our SWOT analysis.  This process took place over months in an iterative manner.

 

Longitudinal Data Review:

One of the final critical components of the Self-Study is longitudinal data review.  This is an in-depth assessment of the yearly program evaluation data since the previous accreditation review.  This includes the ACGME resident and faculty surveys and a yearly program specific internal evaluation completed by both residents and faculty.  It explores which issues have been identified in the past and the action plans that have been implemented to address the shortcomings, as programs are expected to track their outcomes and improvements related to identified issues.

 

Self-Study Summary:

The SSC should aggregate their findings into a succinct Self-Study Summary (SSS).  This document is the only deliverable to the ACGME and has a 2500 word count limit.   As the ACGME requires transparency of the entire process, the program must be careful to document each step of the Self-Study contemporaneously. Notably missing from the SSS is any information about the program’s strengths and areas of improvement; the ACGME specifically excludes these items to allow programs to perform and report on these assessments during the 10-year visit.

 

Tips and Pitfalls:

After following the unique Self-Study process described above we successfully participated in the Self-Study Pilot Visit.  We have incorporated their feedback and the feedback of our key stakeholders in the process to identify the following tips to assist in the development and implementation of the Self-Study process.

• Engage all stakeholders: The aims define what makes your program unique and will be the basic platform for the rest of the Self-Study.  As such, it is imperative to have robust involvement from all stakeholders including current residents, faculty, alumni, and administrative staff.
• Develop measurable aims: It is important to have measurable aims with the ability to generate deliverables on an annual basis.  This will ensure that your program is still represented annually by the description generated by the Self-Study process.
• Establish a timeline: We recommend a longitudinal ten to twelve month timeline to tackle each portion of the Self-Study process on a scheduled basis.
• Set an agenda: Having a clearly defined agenda for each meeting and informing the committee in advance of the agenda and tasks to be completed at that meeting will allow for increased productivity. This will permit the actual meeting time for activities that mandate group participation such as consensus building, brainstorming, and discussion.
• Assign asynchronous tasks: Drafting final text, editing, and detailed work regarding data analysis and reporting should be assigned at the end of each meeting as tasks to be completed asynchronously. This model will provide a roadmap that dedicates appropriate time and discussion to each portion of the process while respecting time demands of busy residents and faculty.
• Approach the SWOT with transparency and embrace the opportunity to evolve: It is essential that there is an honest analysis into both internal and external stressors that affect the program. Throughout this process, we learned that every threat and opportunity offers the antithesis. Upon initial review, stressors may be identified that threaten the program in various capacities.  However, upon secondary review the committee may be able to find the silver lining that actually offers an opportunity in response to the threat.
• Have a balanced approach to change: When identifying weaknesses, we discovered that it is important to report on “low hanging fruit” on an annual basis. We recommend addressing fixable weaknesses via the APE annually and slowly chipping away at larger institutional or programmatic issues over time. The key is to demonstrate a dedication to continually evolve the program both on small and large-scale levels while generating positive momentum.

 

Conclusion:

Since the implementation of the ACGME Self-Study in 2013, no EM programs have formally participated in the Self-Study Site Visit. Our program opted into the Self-Study Pilot Visit and learned some valuable lessons regarding optimizing the Self-Study process. This commentary provides a description of our processes as well as valuable tips based on feedback from stakeholders and the Pilot Visit ACGME representatives.

References:

1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system–rationale and benefits. The New England journal of medicine 2012;366:1051-6.
2. Philibert I, Lieh-Lai M. A Practical Guide to the ACGME Self-Study. Journal of graduate medical education 2014;6:612-4.
3. Philibert I, Nasca TJ. The Program Self-Study and the 10-Year Site Visit: Rationale for a New Approach. Journal of graduate medical education 2015;7:310-2.
4. Coyle J, Martinez S, Robertson WW, Jr., Philibert I. Testing a site visit approach for the next accreditation system. Journal of graduate medical education 2013;5:349-51.
5. Self-Study: Eight Steps for Conducting The ACGME Program Self-Study. (Accessed May 30, 2017, at http://www.acgme.org/What-We-Do/Accreditation/Self-Study.)
6. Guralnick S, Hernandez T, Corapi M, et al. The ACGME Self-Study-An Opportunity, Not a Burden. Journal of graduate medical education 2015;7:502-5.
7. Robbins JB, Sarkany D. Self-Study: Practical Tips for a Successful and Rewarding Experience. Academic radiology 2017;24:721-4.