An Assessment of the Comprehensiveness of Radiation Oncology Residency Websites

Abstract

Objective: Medical students rely on the internet as a resource to gather information about residency programs, although little data exist on the quality or completeness of these websites. Therefore, we sought to evaluate the accessibility of educational and recruitment content of radiation oncology residency websites in the US.

Methods and Materials: The names of radiation oncology residency programs were obtained from the Electronic Residency Application Service. Websites were evaluated for the presence of 20 unique features related to categories of application process, work incentives, educational instruction, research, clinical training, and program leadership introduction. Programs were organized by geographic location, size and ranking for further analysis. Univariate logistic regression was performed to assess predictors of matching in the 2019 cycle.

Results: A total of 92 analyzable websites were identified. Individual program websites contained a mean (SD) of 9.75 (3.8) of the 20 factors sought (49%). Ten (11%) program websites had at least 75% of the 20 features. In addition, 37 (40%) programs had less than 50% of the features listed on their websites. When evaluated by geographic region, no differences in the amount of information available on each website were noted (p = 0.102). Furthermore, there was no significant difference in mean number of features reported by large or small programs (10.80 vs 9.15, p = 0.114) and by ranking (9.96 vs 9.68, p = 0.760). Large programs were more likely to fill all their spots in the 2019 match (OR 3.85, p = 0.013) and there was a nonsignificant trend in increased likelihood of matching with 6 to 15 features on program websites (OR 2.07-2.14).

Conclusion: With the recent high unmatched rate in radiation oncology residency programs, methods to improve the recruitment process are of even greater importance. Many radiation oncology residency websites appear to be incomplete. Improvement in the comprehensiveness and accessibility of radiation oncology websites may improve the recruitment process and allow for medical students to make more informed decisions.

Medical students interested in obtaining a radiation oncology residency position often use the internet as a resource to gather information about residency programs. Although there have been no surveys specific to radiation oncology applicants, studies in other medical specialties have confirmed the importance of online program information.^1-8 Therefore, it is important for residency programs to maintain informative and comprehensive websites for prospective radiation oncology applicants.

Prior studies in other medical specialties have demonstrated that residency program websites are often suboptimal and that missing information can be crucial for applicants to determine which programs are a better “fit” for them.^9,10 Given that program websites may be the only novel program-specific resource medical students have before applying, completeness of program information may be a significant factor in allowing residency programs to remain competitive for applicants, particularly with the recent increase in unmatched radiation oncology program spots in 2019.¹¹

Previous studies in other medical specialties have found deficiencies in online content on residency program websites; however, similar information regarding the availability and quality of current radiation oncology residency website content is not available. We sought to evaluate the accessibility of educational and recruitment content of radiation oncology residency websites in the US.

Methods and Materials

A list of US radiation oncology residency programs was obtained from the Electronic Residency Application Service (ERAS). All websites were publicly available. This study did not require institutional review board approval per the SUNY Downstate Medical Center.

The program websites were accessed through the link provided by ERAS or through an online search. Websites were evaluated for the presence of 20 unique features related to categories of application process, work incentives, educational instruction, research, clinical training, and program leadership introduction (Table 1). The 20 features were derived from published studies in other specialties evaluating residency websites and were considered to be relevant to the field of radiation oncology by the authors.^4,5,12 Inclusion of information required its presence directly on the radiation oncology residency or department website. However, information on salary, benefits, parking, and faculty listing was considered present if it was accessible by a direct link from the residency website.

Programs were organized by geographic location and residency size for further analysis. Programs were divided into institutions based in the Northeast (Maryland, Pennsylvania, Delaware, New Jersey, New York, Connecticut, Massachusetts, Vermont, Rhode Island, New Hampshire, Maine, and the District of Columbia), South (Virginia, Kentucky, Arkansas, Oklahoma, Texas, North Carolina, South Carolina, Georgia, Florida, Louisiana, Mississippi, Alabama, Tennessee), West (New Mexico, Colorado, Washington, Oregon, Nevada, Utah, Arizona, California, Hawaii), and Midwest (Nebraska, Kansas, Minnesota, Iowa, Missouri, Wisconsin, Illinois, Michigan, Indiana, Ohio, West Virginia). States/territories without programs included Alaska, Wyoming, Montana, Idaho, North Dakota, Puerto Rico, and South Dakota. Program size was determined by the median number of residents in the programs, with ≤ 7 residents considered to be small and ≥ 8 considered large. Programs were then categorized by the ranking of the cancer program per US News and World Report as a “Top 25” vs a “Not Top 25” program and further categorized using the National Resident Matching Program data as having filled or not filled in the 2019 cycle. Chi-square and Mann-Whitney/Kruskal-Wallis tests were used to compare categorical and continuous variables, respectively. Univariate logistic regression was performed to assess predictors of matching in the 2019 cycle. SPSS version 21.0 (IBM Inc., Armonk, New York) was used for statistical analysis.

Results

A total of 94 radiation oncology residency programs was obtained from ERAS. Of the 94 programs, 92 websites were evaluated (2 websites were inaccessible or nonexistent). Individual program websites contained a mean (SD) of 9.75 (3.8) of the 20 factors sought (49%).

Only 10 (11%) of the program websites had at least 75% of the 20 features. In addition, 37 (40%) programs had < 50% of the features listed on their websites. Websites ranged from having 15% of the features to as much as 90%. Most sites had contact e-mail (98%), research opportunities (86%), and a comprehensive faculty listing (82%), while fewer than a quarter of programs had a message from the chairperson (24%), call schedule (15%), or parking information (9%) (Table 1).

After organizing by geographical location, the Northeast contained 29% of programs, the South 29%, the West 19%, and the Midwest 23%. There were no differences in the amount of information available on each website based on geographic region (p = 0.102). There were 45 large programs and 47 small programs, and there was no significant difference in mean number of features reported by large or small programs (10.90 vs 9.15, p = 0.114). Comparison of Top 25 vs Not Top 25 programs by the US News and World Report found no difference in mean number of features reported (9.96 vs. 9.68, p = 0.760) (Table 2).

There were 88 programs that entered the NRMP in 2019 of which 22 (25%) did not fill all positions. On univariate logistic regression, large programs were more likely to fill their spots in the 2019 match (OR 3.85, p = 0.013) and there was a nonsignificant trend in increased likelihood of matching with 6 to 15 features on program websites (OR 2.07-2.14)(Table 3).

Discussion

The 2019 Match Day results, with 22 programs (25%) going unfilled for the first time in many years, was a surprising development after years of a competitive match in radiation oncology. The decline in applications is likely multifactorial, with causes including an anticipated future oversupply of radiation oncologists and a much higher-than-usual failure rate on the 2018 radiation biology and physics qualifying board examinations.^13,14

As future generations of medical trainees undoubtedly will continue to use the internet as a resource for investigating residency specialties and individual programs, having comprehensive program websites will continue to grow in importance. In this study, we evaluated current program websites based on 20 criteria and note that on average, programs met about half of these predefined criteria. Furthermore, the geographical location and program size were not related to website completeness, implying that the issue is widespread and overlooked among a variety of programs.

While the deficiencies in online content available for other medical specialties has been reported extensively,^1-8 information on availability and quality of online information regarding radiation oncology programs is limited.¹² We found that important information for prospective residents, including the number of match spots, selection criteria, and alumni job placement, is omitted in the majority of websites. Previous studies have noted that websites influence prospective applicants’ decisions^5,7 and that an easily navigable site may be an important factor in deciding where to apply.³ The lack of information on radiation oncology residency websites may leave applicants with insufficient information with which to gauge their interest in a particular program.

It is likely that another resource programs use is social media, which may be used in recruitment. A survey study of prospective anesthesia residents showed that the majority (52.8%) felt a residency-based social media account impacted their evaluation of programs. Specifically, the most popular platforms included Doximity and Facebook.¹⁵ With an increasing Twitter presence in oncology,¹⁶ its utilization by programs may also be an emerging trend in resident recruitment.

This study has several limitations. First, the choice of program website features by the study team was completed through extensive literature review of desired features in other specialties as well as consensus on factors specific to radiation oncology; however, additional factors of interest to medical students may not have been included. Second, due to website variability, available features may have been overlooked despite thorough review. There was also no official way to verify the accuracy of the information posted on the websites. Furthermore, intangible factors such as website design and ease of use were not assessed in this study. Nonetheless, these results highlight several areas for potential improvement.

Conclusion

The recent match results indicate that individual residency programs, and even our field as a whole, cannot be complacent when it comes to attracting the best medical students. We demonstrate that residency program websites, a medical student’s first and sometimes final look at a program, often lack completeness. Enhancing the quality and completeness of residency program websites may be a very high-yield first step toward optimizing future matches and reversing the recent concerning increase in unfilled radiation oncology residency spots.

References

1. Mulcahey MK, Gosselin MM, Fadale PD. Evaluation of the content and accessibility of web sites for accredited orthopaedic sports medicine fellowships. J Bone Joint Surg Am. 2013;95(12):e85.
2. Chu LF, Young CA, Zamora AK, et al. Self-reported information needs of anesthesia residency applicants and analysis of applicant-related web sites resources at 131 United States training programs. Anesth Analg. 2011;112(2):430-439.
3. Mahler SA, Wagner M, Church A, Sokolosky M, Cline DM. Importance of residency program web sites to emergency medicine applicants. J Emerg Med. 2009;36(1):83-88.
4. Kumar A, Sigal Y, Wilson E. Web sites and pediatric residency training programs in the United States. Clin Pediatr (Phila). 2008;47(1):21-24.
5. Gaeta TJ, Birkhahn RH, Lamont D, Banga N, Bove JJ. Aspects of residency programs’ web sites important to student applicants. Acad Emerg Med. 2005;12(1):89-92.
6. Reilly EF, Leibrandt TJ, Zonno AJ, Simpson MC, Morris JB. General surgery residency program websites: usefulness and usability for resident applicants. Curr Surg. 2004;61(2):236-240.
7. Embi PJ, Desai S, Cooney TG. Use and utility of web-based residency program information: a survey of residency applicants. J Med Internet Res. 2003;5(3):e22.
8. Hansberry DR, Bornstein J, Agarwal N, et al. An assessment of radiology residency program websites. J Am Coll Radiol. 2018;15(4):663-666.
9. Svider PF, Gupta A, Johnson AP, et al. Evaluation of otolaryngology residency program websites. JAMA Otolaryngol Head Neck Surg. 2014;140(10):956-960.
10. Skovrlj B, Silvestre J, Ibeh C, Abbatematteo JM, Mocco J. Neurosurgery residency websites: a critical evaluation. World Neurosurg. 2015;84(3):727-733.
11. National Resident Matching Program, Results and Data: 2018 Main Residency Match. National Resident Matching Program, Washington, DC. 2018.
12. Wakefield DV, Manole BA, Jethanandani A, et al. Accessibility, availability, and quality of online information for US radiation oncology residencies. Pract Radiat Oncol. 2016;6(3):160-165.
13. Pan HY, Haffty BG, Falit BP, et al. Supply and demand for radiation oncology in the United States: updated projections for 2015 to 2025. Int J Radiat Oncol Biol Phys. 2016;96(3):493-500.
14. Royce TJ, Katz MS, Vapiwala N. Training the radiation oncology workforce of the future: course correction to supply the demand. Int J Radiat Oncol Biol Phys. 2017;97(5):881-883.
15. Renew JR, Ladlie B, Gorlin A, Long T. The impact of social media on anesthesia resident recruitment. J Educ Perioper Med. 2019;21(1):E632.
16. Sedrak MS, Attai DJ, George K, Katz MS, Markham MJ. Integrating social media in modern oncology practice and research. Am Soc Clin Oncol Educ Book. 2018;38:894-902.