Authors: Liza Smith, MD, Zach Jarou, MD, and Ava Pierce, MD, on behalf of the Advising Student Committee in EM (ASC’EM, formerly SATF)

Background

The 2018-2019 residency application season will be the second year of the “operational pilot” of the Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI).  The SVI was conceived out of a desire to recognize applicant qualities beyond academic metrics that may not otherwise be represented in the existing Electronic Residency Application Service (ERAS) application.  The goal was to widen the pool of applicants invited for an in person interview by adding a nonacademic dimension to the application.

The SVI itself aims to provide a standardized assessment of an applicant’s ability in two Accreditation Council for Graduate Medical Education (ACGME) competencies: Interpersonal and Communication Skills and Knowledge of Professional Behavior.(1)  By scoring applicants on their responses to six validated question prompts in a uni-directional interview, the SVI generates an overall score from 6-30 (See Figure 1). This score is released to the student as well as to the programs to which he or she applies.  New this year, students can also allow reporting of scores to their medical schools. Scores will be reported September 1st, which will allow students 2 weeks prior to the opening of ERAS to work with their advisor to determine if they need to adjust their application strategy.(2)   Application reviewers additionally have the option of watching the applicant’s interview. (1)

Figure 1.

Figure 1. The distribution of Standard Video Interview scores for the 2017-2018 application cycle. (2)

Renee Overton, Senior Director of Residency & Fellowship Program Solutions at AAMC presented the Standardized Video Interview Update at the CORD Academic Assembly in San Antonio this year.   She presented a recap of the development process behind the SVI as well as some preliminary data from year one of the pilot. The data presented suggested that SVI scores did not have a strong correlation with USMLE scores, an important finding for making the argument that the inclusion of the SVI will add dimensionality to the overall ERAS application.  

How are SVI Scores and Videos Being Used?

At the Academic Assembly, Ms. Overton further offered some data on how Program Directors used SVI data for the initial year of the pilot.  She presented that 70% of Program Directors indicated that “SVI scores were not important in deciding whom to invite to the in-person interview.”  Approximately 80% of Program Directors watched at least one SVI video, with a very wide range of usage both in terms of how many videos were watched and how much of each video was viewed.   Ms. Overton noted that programs were more likely to view videos from applicants with a higher Step 1 score or applicants with a higher SVI score. She presented AAMC data that there were no differences in views for applicants by race, ethnicity, or gender.  She also included that only 57% of Program Directors indicated that they would be “somewhat or more likely to use [SVI score] in ERAS 2019 selection.”

In an abstract presented at this year’s Society of Academic Emergency Medicine (SAEM) conference in Indianapolis, Dr. Melissa White presented data obtained from nine ACGME residency programs showing that when SVI scores were considered in addition to traditional review of ERAS applications for approximately 2100 students, they did not change the likelihood of being interviewed in 93% of cases.  While the score was equally likely to increase or decrease the likelihood of being invited, on subgroup analysis, SVI scores in the 6-11 range decreased the likelihood of being invited by 64%.(3)

There will continue to be wide variation regarding whether SVI scores are taken into consideration by programs and how each individual program may or may not choose to integrate the new information into their application review process. For this reason, it is not clear how best to advise students who might score in this lower range, but it seems reasonable to address it as one might address a lower board score (see CORD “At-Risk” Applicant Applying Guide).

Preparing for the SVI

Students can begin to register for the SVI via the MyERAS portal on June 7, 2018, and must complete the SVI by July 15, 2018, unless they have applied for and been granted an extension by August 30, 2018.(2)

With respect to advising students regarding maximizing their SVI score and preparation for the interview itself, the AAMC Writing Group published their data from surveys administered to students who took part in last year’s pilot. Their results, published in April 2018 in EMRA’s EM Resident Magazine, assessed the time spent and methods used by students in preparation for the SVI experience.  They found that the 92% of students who prepared in some way for the SVI had a one point greater average SVI score than the 8% of students who did not prepare.  This one point difference existed regardless of the method of preparation or the time spent preparing.(4) They also demonstrated that the location the student used to complete the SVI did not influence the final SVI score, with 75% of students completing the SVI at home. (4)

Based on these findings, it seems reasonable to encourage students to engage in some degree of preparation for their SVI.  At the very least, they should read the AAMC Standardized Video Interview: Essentials, AAMC Standardized Video Interview: Applicant Preparation Guide, and the AAMC’s Tips for SVI Applicants.  Students should also be aware that there are sample questions available and they may wish to practice rehearsing their responses either with or without technology.  They will also have the opportunity to practice one or more interview questions on the interview platform HireVue, if they desire. It seems reasonable to advise students to complete the SVI in the location that is most convenient for them, provided that there is a strong, reliable internet connection.

The Future of the SVI

Per the AAMC’s website, the ongoing longitudinal study between the AAMC and the Emergency Medicine Standardized Video Interview (EMSVI) working group aims to evaluate the SVI in four areas: Psychometrics; Applicant and Program Director Reactions; Fairness and Preparation; and Predicting PGY1 Performance.(5)  The website also indicates that future SVI research will aim to address the relationship between SVI scores and Step 2 CS sub-scores, the relationship between SVI scores and the Standardized Letter of Evaluation (SLOE), as well as the use of computer scoring of the SVI.(5) Anyone interested in doing research regarding the SVI should reach out to a member of the AAMC EM-SVI Working Group to ensure that their project remains compliant with the terms of their ERAS Usage Agreement.

For more information, visit: aamc.org/svi

1. Association of American Medical Colleges. AAMC Standardized Video Interview: Applicant Preparation Guide. Available at https://aamc-orange.global.ssl.fastly.net/production/media/filer_public/50/d6/50d6f7d8-1fec-4c07-b55f-6bade45e4934/aamc_standardized_video_interview_applicant_preparation_guide_20180507.pdf. Accessed May 25, 2018.
2. Association of Medical Colleges. The AAMC Standardized Video Interview: Essentials for the ERAS 2019 Season. https://aamc-orange.global.ssl.fastly.net/production/media/filer_public/29/15/291528d7-dca3-426f-b098-504a37f50a6d/18-008_standardizedvideointerviewessentailsforeras2019_20180522.pdf Accessed May 27, 2018.
3. White M. The Standardized Video Interview: Does It Help or Hurt? [abstract]. SAEM Annual Meeting; 2018 May 16; Indianapolis, IN. Abstract 542.
4. Jarou, Z. et al on behalf of the AAMC Writing Group. Factors Affecting Standardized Video Interview Performance. EMResident. April/May 2018. vol 45. issue 2.
5. Association of American Medical Colleges. SVI Research and Data. Available at https://students-residents.aamc.org/applying-residency/article/svi-research-and-data/. Accessed May 25, 2018.