Evaluating the Utility of Webinars on the Radiation Oncology Residency Application Process in the COVID-19 Era

Abstract

Objective: Since the start of the COVID-19 pandemic, the number of digital resources available for medical students interested in radiation oncology (RO) has increased. Here, we evaluated the utility of webinars focused on educating medical students about the RO residency application process.

Materials and Methods: The American College of Radiation Oncology (ACRO) Resident Committee hosted webinars in 2021 and 2022 prior to the Electronic Residency Application Service (ERAS) application deadline. For each webinar, program directors gave short presentations about the ERAS application, interviews, and program ranking, and concluded with a question-and-answer session. Participant demographics were collected using live poll questions, and understanding was assessed using a Likert scale (range, 1-4). Recordings were available online for asynchronous viewing. Differences between groups were assessed using Chi-square statistics.

Results: Between both webinars, there were a total of 69 participants and 340 asynchronous views. A total of 86% and 71% of participants answered the demographics and understanding questions, respectively. The majority attended medical school within the US (75%), were in their third/fourth year (70%), were graduating with an MD degree (88%), and planned to apply to RO residency (78%). In terms of baseline knowledge of the application process, 49% believed they knew “a lot,” while 51% believed they knew “a little” or “nothing.” Most participants noted that the webinar improved their understanding of the general application process (mean 3.80), the ERAS application (mean 3.65), and the interview process (mean 3.90). When stratified by baseline understanding (n = 39), participants who knew “a little” about the application process reported higher scores than participants that knew “a lot.” However, these differences were not statistically significant.

Conclusions: ebinars can improve medical student understanding of the RO residency application process. Given the recent decline in applications to RO, engaging with medical students through dedicated webinars is a unique strategy worth continued utilization.

Keywords: radiation oncology, residency, education, webinar

Since the start of the COVID-19 pandemic, the number of webinars and virtual sessions dedicated to medical student education has increased. In radiation oncology (RO), virtual disease, site-specific educational sessions were shown to significantly improve medical students’ understanding of the role of RO.¹ The utility of webinars to prepare medical students to apply to residency, however, is largely unexplored.^2,3 Here, we present the preliminary outcomes from a webinar series dedicated to educating medical students on the RO residency application process.

Methods

The American College of Radiation Oncology (ACRO) Resident Committee hosted webinars for RO residency applicants using the Zoom videoconferencing platform. Webinars were held in August of 2021 and 2022 prior to the initial submission deadline of the Electronic Residency Application Service (ERAS) application. Webinars were advertised by multiple methods including direct email to ACRO and the Association of Residents in Radiation Oncology (ARRO) listservs and social media (Twitter and Instagram). Emails and/or social media posts were performed weekly starting 1 month in advance of the sessions. Each webinar consisted of 2 resident moderators and a panel of 2 to 3 residency program directors. New moderators and program directors were used each year, representing 9 residency programs. For both webinars, program directors provided a general overview of the residency application process by presenting short lectures about the ERAS application, interview process, and residency program ranking. Sessions did not focus on the specifics of a particular residency program. The sessions concluded with an open question-and-answer segment. Each webinar lasted approximately 1 hour. Afterward, a recording of each webinar was posted to the ACRO YouTube channel for asynchronous viewing.

Baseline participant demographics were assessed using the questions listed in Table 1. These questions were presented as a live poll within the Zoom videoconferencing platform prior to the didactic presentations, and participants were given 2 minutes to respond. At the conclusion of the webinar, after the question-and-answer segment, participant perceptions were evaluated using a Likert scale ranging from 1 (No, not at all) to 4 (Yes, definitely). These questions were also presented as a live poll within the Zoom videoconferencing platform and participants were given 2 minutes to respond (Table 2). Both sets of questions were developed by consensus with the ACRO Resident Committee. Given the use of live polling, a limited number of demographics and perceptions questions (and choices for each question) were used. Results of both questionnaires were combined and reported using descriptive statistics. While the structure of each webinar was the same, comparisons between each year were performed to assess for changes in applicant demographics and perceptions. A subset analysis was also conducted to assess whether participants’ baseline knowledge level of the residency application process impacted their perceptions of the webinar. Comparisons between groups were performed using the Chi-squared test in SAS (Carey, NC). A P value < 0.05 was considered statistically significant. The study was approved by the Wake Forest University School of Medicine institutional review board.

Results

Baseline Characteristics of Webinar Participants

The baseline characteristics of webinar participants are outlined in Table 1. Between the 2 webinars there were a total of 69 participants (36 in 2021 and 33 in 2022) and more than 340 asynchronous views online. Of all participants, 59 (86%) responded to the demographics questions (31 in 2021 and 28 in 2022). The majority of participants (70%) were in their third or fourth year of medical school, while nearly one quarter (24%) were either resident/fellows in another specialty or not currently students/trainees. When examined by year, there was a trend toward a significant increase in nonmedical student participation: 16% vs 32% of participants in 2021 and 2022, respectively (Chi-square, P = 0.05). Additionally, most participants attended medical school in the US (75%) and were planning to graduate with an MD degree (88%). While most participants were planning on applying to RO for residency (78%), baseline knowledge about the residency application process was relatively split: 44% identified as only knowing “a little” about the process, 49% identified as knowing “a lot” about the process, and only 7% identified as knowing “nothing” about the process. Other than training level, there were no statistically significant differences in baseline characteristics between each year.

Changes in Participant Understanding

Participant perceptions at the end of the webinar are outlined in Table 2. Of the 69 total webinar participants, 49 (71%) answered these questions (27 in 2021 and 22 in 2022). Overall, participants had favorable perceptions of the webinar with the majority noting a definitive improvement in their general understanding of the application process (82%) as well as their understanding of the ERAS application (69%) and interview process (89%). The webinar appeared to have the highest impact on participants’ understanding of the interview process (mean 3.90), followed by general understanding of the application process (mean 3.80) and ERAS application (mean 3.65) (Figure 1A). However, these differences were not statistically significant (Chi-square, P = 0.12). For a subset of participants (n = 39), survey results were able to be stratified by a baseline understanding of the application process (“a little” vs “a lot”). Compared with participants who felt they knew “a lot” about the application process, those that knew “a little” about the process reported higher scores in general understanding (mean 3.80 vs 3.60), ERAS application (mean 3.72 vs 3.50), and interview process (mean 3.93 vs 3.8) at the end of the webinar (Figure 1B). These differences, however, were not statistically significant by the Chi-square test. Finally, at the end of the webinar, the majority (96%) of participants indicated they plan to apply to RO residency within the next 2 years.

Discussion

Since the start of the COVID-19 pandemic there has been an increase in remote learning opportunities for RO trainees, including virtual clerkships,^4-7 online educational videos,^8,9 and webinar series.¹ Moreover, a recent systematic review identified 47 free digital educational resources specific to RO.¹⁰ As the number of digital resources in RO increases, we must examine their utility and whether they meet the needs of their target audience.

Overall, our results suggest that a national webinar dedicated to the RO residency application process is both feasible and has utility, as most participants noted improved understanding of the general application process, the ERAS application, and the interview process. Since the webinar recordings were placed online, they have garnered more than 340 views. We believe a webinar is an ideal format to educate medical students about the RO residency application process because it utilizes both attending and resident physicians—two sources of information that medical students consider highly trustworthy for residency advice.¹¹ Additionally, webinars are an opportunity for active participation as medical students are able to ask program directors specific questions of interest.

While participant perceptions were similar each year, there was a greater proportion of non-medical student participants in 2022 than 2021 (32% vs 16%). This potentially reflects the ongoing changes in the educational and training backgrounds of RO applicants. For example, in the 2023 Main Residency Match, 24% of RO PGY2 positions were filled by international medical school graduates or non-US senior medical students.¹² While these webinars were intended to educate medical students, who comprise the majority of RO residency applicants, they can also be informative to applicants with other training backgrounds who are looking to become radiation oncologists in the US. Given that these webinars discussed the nuances of applying to RO, and not just the residency application process in general, this change in participant training background is unlikely to impact our study’s assessment of participant perceptions.

Our preliminary findings are also in line with the results from other studies on residency application webinars. Within RO, for example, a 2016 webinar on medical student applications was noted to have “positive feedback” from participants.¹³ Dedicated residency webinars also have had favorable results in other fields. In plastic surgery, Serebrakian et al² found that a webinar led by a single institution increased medical student confidence levels about matching into residency. Similarly, Fereydooni and colleagues³ found that a webinar led by recently matched medical students improved participants’ understanding of the vascular surgery application process (eg, number of applications needed). Thus, our findings add to the limited body of literature that demonstrates the utility of webinars dedicated to the residency application process.

Our study is not without its limitations. For example, we used built-in poll questions during the webinar in place of postwebinar surveys to increase participation. While the response rate to the polls was high, particularly for the initial question set, it limited the number of demographic and understanding domains we could evaluate. Additionally, because perceptions were assessed at the end of the webinar, it is likely that some participants left before answering these questions. Because these webinars were held prior to ERAS application submission, we were unable to assess whether participants retained the knowledge they learned and applied it to the application process. Additionally, since postwebinar surveys were not conducted, we were unable to assess whether participants applied (or matched) into RO. In the future, detailed pre- and postwebinar questionnaires could be used to address these limitations. The Zoom live poll questions could also be distributed throughout the webinar, which could help increase response rate. In terms of accounting for participants’ baseline knowledge, we did not find a difference in understanding when stratified by baseline knowledge due to the smaller number of participants who answered both sets of questions. Additionally, because the webinars were advertised digitally, it is possible that this self-selected for participants who proactively sought out information on residency applications and were already well informed about the process.

Conclusions

A national webinar with program directors and residents can improve medical students’ understanding of the RO application process. This resource should continue to be offered for future applicants given the current landscape of the RO residency application and match process.

References

1. Kahn JM, Sandhu N, von Eyben R, et al. Radiation oncology virtual education rotation (ROVER) for medical students. Int J Radiat Oncol Biol Phys. 2021;111(1):29-35. doi:10.1016/j.ijrobp.2021.03.057
2. Serebrakian AT, Ortiz R, Christensen JM, et al. Webinar during COVID-19 improves knowledge of changes to the plastic surgery residency application pro- cess. Plast Reconstr Surg Glob Open. 2020;8(10):e3247. doi:10.1097/gox.0000000000003247
3. Fereydooni A, Ramirez JL, Dossabhoy SS, Brahmandam A, Dardik A, Ochoa Chaar CI. A national post-Match webinar panel improves knowledge and preparedness of medical students interested in vascular surgery training. J Vascul Surg. 2020;71(5):1733-1740.e5. doi.org/10.1016/j.jvs.2019.08.282
4. Pollom EL, Sandhu N, Frank J, et al. Continuing medical student education during the coronavirus disease 2019 (COVID-19) pandemic: development of a virtual radiation oncology clerkship. Adv Radiat Oncol. 2020;5(4):732-736. doi.org/10.1016/j.adro.2020.05.006
5. Sandhu N, Frank J, von Eyben R, et al. Virtual radiation oncology clerkship during the COVID-19 pandemic and beyond. Int J Radiat Oncol Biol Phys. 2020;108(2):444-451. doi.org/10.1016/j.ijrobp.2020.06.050
6. Kahn JM, Fields EC, Pollom E, et al. Increasing medical student engagement through virtual rotations in radiation oncology. Adv Radiat Oncol. 2021;6(1):100538. doi.org/10.1016/j.adro.2020.07.015
7. Franco I, Oladeru OT, Saraf A, et al. Improving diversity and inclusion in the post–coronavirus disease 2019 era through a radiation oncology intensive shadowing experience (RISE). Adv Radiat Oncol. 2021;6(1):100566. doi.org/10.1016/j.adro.2020.09.006
8. Kim TH, Kim JS, Yoon HI, et al. Medical student education through flipped learning and virtual rotations in radiation oncology during the COVID-19 pandemic: a cross sectional research. Radiat Oncol. 2021;16(1):204. doi:10.1186/s13014-021-01927-x
9. Oertel M, Pepper NB, Schmitz M, Becker JC, Eich HT. Digital transfer in radiation oncology education for medical students—single-center data and systemic review of the literature. Strahlentherapie und Onkologie. 2022;198(9):765-772. doi:10.1007/s00066-022-01939-w
10. Culbert MM, Brisson RJ, Oladeru OT. The landscape of digital resources in radiation oncology. Technic Innovat & Pati Supp Radiat Oncol. 2022;24:19-24. doi. org/10.1016/j.tipsro.2022.08.006
11. Maas JA, Burnett IO, Marcrom SR. Reasons for declining applicant numbers in radiation oncology from the applicants’ perspective: results from the Appli- cant Concerns and Radiation Oncology Sources Survey (ACROSS). Int J Radiat Oncol Biol Phys. 2021;111(2):317-327. doi:10.1016/j.ijrobp.2021.05.007
12. National Resident Match Program. Advanced Data Tables - 2023 Main Residency Match. March 17, 2023. Accessed May 10, 2023. https://www.nrmp.org/wp- content/uploads/2023/04/Advance-Data-Tables-2023_FINAL-2.pdf
13. Sura K, Lischalk JW, Leckie J, Grills IS, Vapiwala N. Applying for radiation oncology residency: webinar-based medical student mentorship outreach. Int J Radiat Oncol Biol Phys. 2017;97(1):11-12. doi.org/10.1016/j.ijrobp.2016.09.024