Initial report of a simulation-based medical education curriculum for radiation therapy treatment planning

Abstract

Purpose/Objective(s): Varying degrees of treatment planning education are provided across radiation oncology residency programs. The specific aim of this project was to implement and evaluate a self-directed simulation-based treatment planning curriculum developed to teach residents key treatment planning concepts in a series of interactive, “hands-on” modules.

Methods/Methods: A curriculum was developed consisting of 3 treatment planning modules including anteroposterior/posteroanterior (AP/PA) spine, 3-field breast, and intensity-modulated radiation therapy (IMRT). Participants completed anonymous evaluations after completing each module assessing their treatment planning proficiency using Likert-type scales (1 = “not at all” to 5 = “extremely”; reported as median [interquartile range]). Nonparametric statistical tests were used.

Results: Eleven residents in their postgraduate year (PGY-2 to PGY-5) at a single academic medical center completed the curriculum during the 2014-15 academic year. Completion of the modules was associated with improved resident comfort with AP/PA treatment planning (p = 0.03), 3-field breast treatment planning (p < 0.01), and IMRT planning (p = 0.03). Resident self-reported understanding of the following treatment planning concepts was significantly improved after completing the modules: dose grid (p < 0.01); beam energy selection (p = 0.03); calculation point (p = 0.04); iterations (p = 0.01); segments (p = 0.02); optimization (p < 0.01); and ring structure (p < 0.01).

Conclusion: Radiation oncology residents experience significantly improved comfort with treatment planning and understanding of treatment planning concepts after completion of self-directed treatment planning modules. Development of additional modules as part of a formal treatment planning curriculum is warranted.

In the era of highly conformal and image-guided radiation therapy, optimal therapeutic response requires accurate delineation of target volumes and adequate dose coverage of those volumes while sparing normal tissues. However, review of plans submitted for multi-institutional clinical trials reveals that many plans fail to adequately cover the target volume or exceed normal dose constraints resulting in minor or major deviations. Such variability has been shown to lead to worse clinical outcomes including decreased overall survival and increased normal tissue toxicity.^1,2

As members of an interprofessional team, radiation oncologists must understand fundamental principles of treatment planning. However, radiation oncology residents experience varying degrees of formal treatment planning education.^3,4 Furthermore, there is minimal literature describing effective educational methods or interventions for teaching treatment planning during residency.⁵ Inconsistent training may contribute to the larger problem of treatment planning and dosimetric variability observed between centers in multi-institutional randomized trials.

Simulation-based medical education (SBME) is an increasingly recognized form of effective and easily disseminated educational intervention across multiple medical fields including radiation oncology.⁶ The primary objective of this project was to develop, implement, and evaluate a novel self-directed SBME treatment planning curriculum to teach residents fundamental radiation treatment planning concepts in a series of interactive, hands-on modules. With improved treatment planning knowledge, radiation oncology residents will function more effectively as members of an interprofessional radiation therapy team.

Methods and Materials

Development of the Intervention

A pilot curriculum was developed consisting of 3 treatment planning modules that could be completed independently or in small groups using the Pinnacle treatment planning system (Philips Healthcare, Andover, Massachusetts).⁷ The modules are self-directed and each take approximately 1 hour to complete. Since these workshops teach basic treatment planning concepts and skills, they are completed in order: first anteroposterior/posteroanterior (AP/PA) spine, then 3-field breast, and finally intensity-modulated radiation therapy (IMRT). Specific goals and educational objectives of each module are described in Tables 1, 2, and 3. A screenshot sample from the IMRT module is provided in Figure 1. The modules can be downloaded at https://www.mededportal.org/publication/9297/

Curriculum Implementation

During the 2014-15 academic year, 11 residents (postgraduate year PGY-2 to PGY-5) from a single institution participated in the pilot curriculum. Participants were exposed to various aspects of treatment planning, including plan design, dose prescription, and plan evaluation. Participants were asked to complete evaluations before and after completing the modules assessing their treatment planning proficiency using Likert-type scales (1 = “not at all,” 2 = “somewhat,” 3 = “moderately,” 4 = “quite,” 5 = “extremely”; reported as median [interquartile range]). Free responses were also collected for qualitative analysis. Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at the https://redcap.uchicago.edu/.⁸ REDCap is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for importing data from external sources. The Wilcoxon rank-sum statistical test was used to compare pre- and post-workshop evaluation scores. Qualitative responses were analyzed for common themes. This study was approved as exempt by the University of Chicago Institutional Review Board.

Results

All 11 residents completed the treatment planning modules and submitted anonymous evaluations. Pre- and postmodule ratings are summarized in Table 4. Module completion was associated with improved resident comfort with AP/PA treatment planning (pre 3 [2-3] vs post 4 [3-4], p = 0.03), 3-field breast treatment planning (pre 3 [2-3] vs post 4 [3-4], p < 0.01), and IMRT planning (pre 3 [2- 4] vs post 4 [3-4], p = 0.03). Resident self-reported understanding of the following treatment planning concepts significantly improved after completing the modules: dose grid (pre 2 [2-3] vs post 4 [3-4], p < 0.01); beam energy selection (pre 3 [2-4] vs post 4 [3-4], p = 0.03); calculation point (pre 2 [1-3] vs post 4 [3-5], p = 0.04); iterations (pre 1 [1-3] vs post 4 [3-4], p = 0.01); segments (pre 3 [2-4] vs post 4 [3-4], p = 0.02); optimization (pre 2 [2-3] vs post 4 [4-4], p = 0.01); and ring structure (pre 2 [1-3] vs post 3 [3-4], p < 0.01).

In the qualitative analysis, residents described acquiring important skills and knowledge through the modules. Additionally, they gave feedback regarding module strengths as educational tools and suggested future curriculum improvements. Learning the basics of treatment planning software and functionality (6/11), practicing beam setup and modification (6/11), understanding the treatment planning dose grid (4/11), and evaluating treatment plans (2/11) were cited as important treatment planning concepts taught in the modules. The most commonly described curriculum strength was the explicit step-by-step instructions that guided learners through the modules and allowed for future reference and practice (3/11). Suggested improvements included providing the curriculum earlier in the academic year, particularly for junior residents (2/11), and creating more complex cases as additional learning opportunities (1/11).

Discussion

Formal treatment planning education during training varies significantly across US residency programs.^3,4 Given the importance of optimal treatment planning in the delivery of highly conformal radiation therapy, residency programs should work to implement a formalized treatment planning curriculum for residents in training. This study reports the pilot implementation of a self-directed SBME curriculum designed to teach fundamental treatment planning and dosimetry concepts to radiation oncology residents. Completion of the SBME modules was associated with positive learning outcomes and increased self-reported competence for multiple predetermined educational objectives. This report provides evidence that self-directed modules can be effective tools to teach foundational treatment planning concepts to residents at varying stages of training (PGY-2 through PGY-5).

SBME is increasingly recognized as an effective form of educational intervention for teaching a wide variety of skillsets across numerous medical specialties including radiation oncology.^6,9,10 This pilot curriculum is an example of radiation oncology SBME designed to teach treatment planning and dosimetry concepts to residents. Through a simulated treatment planning environment and screen-based simulator, residents can experiment and learn with real-time performance feedback. Importantly, SBME allows for acquisition and practice of new skills with no potential patient harm. This contrasts with traditional apprenticeship-based education in radiation oncology, which often involves practicing skills such as contouring and treatment planning on real patients, leading to inconsistent exposure to treatment planning concepts along with the potential risk of patient harm.

Online and self-directed teaching modules offer advantages compared to traditional didactic approaches. As an experiential form of learning, this pilot hands-on treatment planning curriculum allows residents to experience, reflect, think, and act in a recursive manner based on Kolb’s experiential model to effectively assimilate knowledge and master new concepts.¹¹ Importantly, because these modules are self-directed, learners can complete them at their own optimal pace. Lecture-based didactics are designed to teach groups of students and, therefore, are not as adaptable for individual variations in background knowledge or skill sets. The pilot treatment planning curriculum also allows learners to tailor their experience to address specific knowledge gaps without focusing on already mastered topics. Lastly, residents can complete individual self-directed learning modules more efficiently in noncontinuous intervals, allowing a more personalized schedule.

While this investigation provides evidence that the self-directed treatment planning modules are effective teaching tools for residents in training, in their current form these modules are not independently sufficient for ensuring an integrated and fully contextualized understanding of radiation therapy treatment planning. Individual variation in patient anatomy and treatment systems can significantly impact treatment planning and delivery. Learning the idiosyncrasies of treatment planning on a case-by-case basis is difficult to teach in general self-directed modules. This type of knowledge might better be acquired through communication and apprenticeship-based teaching approaches in which mentor expertise guides learning. Therefore, the present curriculum is not intended to replace, but rather to complement, other educational methods. As an example, the self-directed modules might be used to teach fundamental principles of treatment planning to residents while practical application of that knowledge can be honed under the tutelage of an experienced teacher.

Potential drawbacks to this treatment planning curriculum must also be recognized. The most important obstacle to self-directed learning is the potential for lack of individualized feedback and communication between learner and educator. As described above, in traditional lecture-based didactics, learners can ask questions on knowledge gaps and receive personalized feedback from experts. However, self-directed learning limits this interchange, and thus may be less efficient for teaching challenging concepts that would be better communicated through educator-learner dialogue. One potential method to mitigate this is to pair a senior resident (PGY4-5) with a junior resident (PGY2-3) to work through each module in a near-peer mentor dyad.¹² The junior resident “drives” the treatment planning system while the senior resident provides instruction and feedback. For example, at our institution one junior resident described how “working together in pairs, especially with a senior resident, was particularly helpful … and put the case into clinical context.” Another potential disadvantage of this curriculum is that successful teaching is predicated upon residents taking responsibility for educating themselves without protected time to do so. Given the high demands of residency training, it may be difficult for residents to thoroughly complete the entire curriculum and they may miss important opportunities to learn. Our program, therefore, protects time after clinic every 2 to 3 months to complete each module

The findings of this study may be limited by several factors related to the experimental design. First, this study was conducted with a sample of only 11 residents at a single institution. Ideally, the curriculum will be evaluated in the setting of a randomized controlled trial, which would increase the statistical power and limit biases, making findings more generalizable. While the results met statistical significance despite limited power, the findings may not be applicable to all residency programs or individual residents. Further investigation will characterize the efficacy of these modules when implemented in a larger and more diverse setting. Another limitation is the difficulty in interpreting self-reported competence as an educational end point. For example, it is possible for learners to feel more confident with a particular skill set after an educational intervention even though an objective performance evaluation may not indicate improvement. While objective measures of learner performance (eg, impact on patient outcomes) would be the ideal educational endpoints to assess the efficacy of any medical education curriculum, such evaluation is outside the scope of this initial pilot study. Given the promising results of our institutional implementation of the modules, expansion and objective evaluation of this and similar SBME curricula is a logical and important future direction of this work.

Given the public availability of the modules, this curriculum and future similar self-directed SBME treatment planning modules can be adopted for widespread use by radiation oncology training programs. Importantly, dissemination of the curriculum may lead to more consistent treatment planning education for radiation oncology residents on a large scale. This might help address the issue of inter-observer treatment planning variability described earlier and potentially improve clinical outcomes for patients. Additionally, given the positive pilot data for the self-directed modules in this study, further development of similar curricula to offer residents more in-depth and comprehensive exposure to treatment planning concepts is warranted. Finally, expansion of these modules to cover more advanced treatment planning material and other radiation oncology skill sets may enhance the continuing medical education of practicing radiation oncologists.

Conclusion

This study reports the pilot implementation of a self-directed SBME treatment planning curriculum for radiation oncology residents. Successful completion of the curriculum was associated with significant increases in self-reported learner competence in multiple predetermined educational objectives relating to fundamental treatment planning concepts. Further development and evaluation of similar curricula is warranted given the discrepancies in formal treatment planning educational methods across radiation oncology training programs.

References

1. Peters LJ, O’Sullivan B, Giralt J, et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol. 2010;28:2996-3001.
2. Abrams RA, Winter KA, Regine WF, et al. Failure to adhere to protocol specified radiation therapy guidelines was associated with decreased survival in RTOG 9704—a phase III trial of adjuvant chemotherapy and chemoradiotherapy for patients with resected adenocarcinoma of the pancreas. Int J Radiat Oncol Biol Phys. 2012;82:809-816.
3. Klein EE, Gerbi B, Meli J, Marsden D. 1995 survey of physics teaching efforts in radiation oncology residency programs. Int J Radiat Oncol Biol Phys. 1997;3:441-446.
4. Nabavizadeh N, Burt LM, Mancini BR, et al. Results of the 2013-2015 association of residents in radiation oncology survey of chief residents in the United States. Int J Radiat Oncol Biol Phys. 2016;94;228-234.
5. Alfieri J, Portelance L, Souhami L, et al. Development and impact evaluation of an e-learning radiation oncology module. Int J Radiat Oncol Biol Phys. 2012;82:e573-e580.
6. Rooney MK, Zhu F, Gillespie EF, et al. Simulation as more than a treatment planning tool: a systematic review of the literature on radiation oncology simulation-based medical education. Int J Radiat Oncol Biol Phys. 2018;102(2):257-283. doi:10.1016/j.ijrobp.2018.05.064
7. Golden D, Stepaniak C, Chmura S. Radiation oncology self-directed dosimetry workshops: AP/PA Spine, 3-field breast, and IMRT. MedEdPORTAL Publ. 2012. doi:10.15766/mep_2374-8265.9297
8. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009:42,377-381.
9. Gaba DM. The future vision of simulation in healthcare. Simul Healthc J Soc Simul Healthc. 2007;2:126-135.
10. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27:10-28.
11. Kolb DA. Experiential Learning: Experience as the Source of Learning And Development. Englewood Cliffs, NJ: Prentice Hall; 1983.
12. Ten Cate O, Durning S. Peer teaching in medical education: twelve reasons to move from theory to practice. Med Teach. 2007;29,591-599.