Enhancing Classroom Management Skills: Efficacy of a Supplemental Multi-Platform Intervention for Preservice Teachers

Many teachers are ill-prepared to apply practices that can both preempt and address challenging behaviors that interfere with academic instruction. We evaluated the efficacy of a supplemental multi-platform (direct instruction, guided practice, and mixed reality simulations) intervention designed for preservice teachers who do not have opportunities to participate in formal classroom management courses. Efficacy of the intervention was assessed through the quality of classroom management plans, the presentation and implementation of the plan in the virtual classroom, and classroom management self-efficacy. Results indicated that preservice teachers successfully developed quality proactive classroom management plans but struggled to present and implement their plans. However, self-efficacy increased indicating that participants believed they were better able to manage classroom behavior than they did prior to the intervention. Implications of these outcomes are discussed with an emphasis on how adequate opportunities to practice acquired classroom management skills are included in budget-challenged teacher preparation programs.

teacher evaluations, positive student outcomes, and decreased rates of attrition from the profession (Scheuermann & Hall, 2016). Unfortunately, many teachers believe they lack the skills to manage a classroom and attribute their shortcomings to deficiencies in their teacher preparation programs (Flower et al., 2017). Research has validated this perception: Although most states require accredited educator preparation programs (EPPs) to provide instruction on classroom management, many programs do not offer a specific course on classroom management and when a course or infused activities are provided, demonstrations of and opportunities to apply evidence-based practices are often in short supply (Flower et al., 2017;Freeman et al., 2014;Oliver & Reschly, 2010). Consequently, far too many teachers enter the classroom with minimal preparation in effective classroom management yet are expected to understand and address the diverse behavioral needs of students and facilitate academic achievement (Begeny & Martens, 2006;Mitchell et al., 2017). Not surprisingly, insufficient preparation in managing classrooms often leads to increased teacher stress, exhaustion, burnout, and attrition from the profession (Aloe et al., 2014). As noted by Scott (2017), this creates a recurring vicious cycle in which poor preparation leads to high rates of teacher burnout, leading to teacher shortages that are filled with even less qualified personnel who eventually leave the profession.
With so much at stake, why do many EPPs forego explicit training in classroom and behavior management? A full exploration of this important issue is beyond the scope of this paper. However, the number of credits allocated to pedagogical courses in the typical education major or concentration is extremely limited (typically no more than 24 credits) with multiple constituencies vying for instructional time. As observed by Sindelar and Rosenberg (2000), EPPs are servants to many masters, both inside and outside the academy. Specifically, state certification standards, licensure standards of professional organizations, departmental politics, and myriad hot button issues (bullying, reading failure, STEM {science, technology, engineering and mathematics, etc.) individually and together drive how much content can fit into a program's limited bandwidth. Regardless, given the limited classroom management preparation delivered to prospective teachers, there is a clear need for alternative, innovative ways of providing basic evidence-based training in how to, at the very least, develop, implement, and sustain a classroom management plan (Rosenberg & Jackman, 2003).
The purpose of this study was to assess the efficacy of a multi-platform intervention to promote inclusive or universal evidence-based classroom management practices among a cohort of preservice teachers who do not have an opportunity to participate in a specific classroom management course. Inclusive or universal practices, sometimes referred to as Tier 1 activities, focus on enhancing preventive and protective factors in ways that encourage students to meet behavioral expectations. At this level, classrooms are organized by developing a mission statement (a purpose statement linking appropriate behavior to academic success), rules, procedures, and consequences for appropriate and inappropriate behavior (e.g., Lewis & Sugai, 1999;McLeskey et al., 2018;Rosenberg & Jackman, 2003). Typically, 80-85% of students respond positively to inclusive or universal classroom management (Bradshaw et al., 2012). In our study, we developed and evaluated the efficacy of a 12hour instructional module made up of three components: (1) a direct instruction seminar that directly taught the inclusive or universal evidence-based classroom management concepts; (2) guided practice through on-line validated IRIS Center modules (e.g., The IRIS Center, 2021;Sayeski et al., 2015); and (3) direct application of skills with feedback through mixed reality simulations (TeachLive; e.g., Dieker et al., 2008;Pas et al., 2016). Both IRIS and TeachLive provide opportunities for on-line practice and application. The ultimate outcome was for the participants to develop, present, and implement an evidence-based, inclusive or universal classroom management plan. Scenarios for applied practice were developed through procedures developed by TeachLive researchers and are detailed in the methodology.

Participants and Setting
Twenty volunteer undergraduate students were recruited from the Elementary Education program at a medium-sized mid-Atlantic comprehensive public college. All participants were in the final year before student teaching of their 4-year preparation program; none had an opportunity to take a course in classroom management as it was not a required part of their already credit intensive preparation program. All students in this cohort were sent an email invitation to participate in the study, and one of the researchers visited a common class for all members of the cohort to raise awareness of the study and increase interest in study participation. Participants were predominantly female (n = 18) and between the ages of 18 and 23. Two of the participants dropped out of the study prior to the direct instruction seminar, and three more dropped out of the study prior to the first direct application session. This left 15 participants, 14 of whom were female. Participants received $100 for their involvement in the study.

Independent Variable
The multi-platform instructional intervention was delivered over the course of one month and was comprised of three components: (1) a direct instruction seminar; (2) guided practice; and (3) application of skills with feedback through mixed reality simulations.
Direct instruction seminar. Upon being accepted into the study, participants were asked to sign up to attend one of two four-hour seminars. The seminars, delivered by the principal investigator, highlighted inclusive or universal Tier 1 evidence-based practices for designing, implementing, and maintaining a positive classroom management system. Specific topics addressed included (a) the development of mission statement, rules, positive and negative consequences, procedures (e.g., Alter & Haydon, 2017;Mitchell et al., 2017;Rosenberg & Jackman, 2003;Rosenberg et al., 2006); (b) the connection between instruction and student behavior (Cooper & Scott, 2017) and (c) maintaining an appropriate management perspective by demonstrating care, respecting students, communicating as a professional, using surface management techniques, and being prepared (Scott et al., 2010). Participants were instructed in these concepts through an in-person classroom presentation supplemented by two video-based activities: (1) evaluating teachers' presentations of classroom management systems on the first day of school; and (2) appraising the implementation (i.e., delivery of positive and negative consequences) of existing management systems or strategies.
Guided practice. Guided practice on the key concepts was delivered through two IRIS Star Legacy modules, Classroom Management (Part 1): Learning the Components of a Comprehensive Behavior Management Plan (The IRIS Center, 2012a) and Classroom Management (Part 2): Developing Your Own Behavior Management Plan (The IRIS Center, 2012b). The second module concludes with an assignment in which participants design their own classroom management plan. Star Legacy modules (IRIS has developed 70 of them on myriad topics) and IRIS support materials are used widely in teacher education; 75% of all colleges and universities that offer special education preparation programs use them with their candidates (Smith & Bryant, 2014). Most important, IRIS materials are effective. Collectively, they receive high ratings for trustworthiness and reliability as sources for evidence-based practices (Test et al., 2015;Torres et al., 2014) and, in regard to the classroom management modules, they have been found to enhance participant knowledge (Sayeski et al., 2015).
Direct application. Two TeachLive sessions were customized to simulate (1) the first day of class, in which the teacher (each of our participants) was presenting the classroom management system and (2) an instructional session in which the participant was teaching a procedure (e.g., entering the classroom, cafeteria behavior, requesting a lavatory pass) while applying their management system. TeachLive was selected because of its efficacy in enhancing preservice teacher knowledge as well as providing effective practice and feedback on foundation teaching skills in sophisticated immersive classroom settings (e.g., Dawson & Lignugaris/Kraft, 2017;Pas et al., 2016;Sayeski et al., 2015). The first session provided opportunities to present the management plan and the second allowed for opportunities for the teachers to react to situations in ways that the classroom management plan would be utilized. Following each of the sessions, students were provided with feedback by a member of the research team. It was anticipated that the feedback provided after the first simulation would assist participants as they prepared for their second session.
Scenario development was completed by researchers at the University of Central Florida. In order to promote consistency for behavioral feedback from the virtual avatar students, the interactors, human operators that control the virtual avatars during a session, attended two, two-hour trainings on identifying classroom management strategies and selecting appropriate escalation or de-escalation responses. For the first portion of this training, interactors were given a coding rubric that described a wide selection of classroom management behaviors and key qualities of teacher self-presentation. Interactors were then asked to view videos of teaching and code the behaviors. After the coding activity, inconsistencies in coding were discussed and modifications were suggested for the performance rubric. For example, the category of "setting high expectations" was removed due to the difficulty in observing evidence of expectations in a short segment of teaching. Additionally, the qualities of vocal tone and emotional expression could be coded and considered as influencing factors on other observed classroom management behaviors. Interactors practiced applying the standardized performance rubric in mock teaching sessions with fellow interactors stepping in as practice teachers.

Dependent Variables
The efficacy of the intervention was assessed through four dependent measures: quality of the classroom management plan, presentation of the plan, plan implementation, and self-efficacy.
Quality of management plan. As part of the direct instruction and guided practice participants developed a comprehensive management plan containing five specific elements: (a) a mission statement, (b) rules, (c) positive consequences, (d) negative consequences, and (e) at least two procedures. Each element of the plan was evaluated on a five-point scale ranging from 1 (weak) to 5 (strong) by members of the research team. A rating of 3 was regarded as moderate. A grand mean or aggregate score was computed to provide a measure of the entire plan.
Introduction/Presentation of the management plan. A TeachLive scenario designed to reflect the first day of school allowed for assessment of participant introductions of their management plans to students. Using the observation instrument (See Appendix A), one member of the research team assessed participant performance on four elements of effective behavior management including the presentation of the mission statement, rules, as well as consequences for rule compliance and noncompliance. As with the assessment of plan development, each element was evaluated on a five-point scale and an aggregate score was computed.
Plan implementation. Within 10 days of introducing their plans, participants were assessed on the implementation of their plans through a TeachLive classroom scenario. This involved the direct teaching of at least one classroom procedure (e.g., entering the classroom, requesting lavatory use, cafeteria routines) and responding to appropriate behaviors and minor disruptions (e.g., calling out, speaking out of turn, inattention). Similar to the assessment of plan introduction, a member of the research team used the observation instrument (See Appendix A) to evaluate participant performance on the teaching of the procedures, respect for students, communicating with students, and responding to students appropriately (e.g., use surface management techniques, directed praise, and address misbehavior in a positive fashion). As with the other measures, each element was evaluated on a fivepoint scale and an aggregate score was computed.
Self-Efficacy. Self-efficacy is confidence or a belief in one's ability to execute specific actions or complete tasks (Bandura, 1977). Teacher self-efficacy has been investigated in terms of general teaching confidence as well as in specific aspects of successful teaching such as classroom management (Larson et al., 2018). In the current study, participants responded to a 10-item Self-Efficacy in Behavior Management Scale (SEBMS). The first five items were adopted from the validated General Teaching Efficacy scale (Hoy & Woolfolk, 1993). The second set of five questions was adopted from the validated Teacher Self-Efficacy in Behavior Management (Larson et al., 2018). Items were presented on a 5-point Likert scale (1 = Strongly Disagree through 5 = Strongly Agree) and an aggregate score was computed. Survey items are provided in Figure 1. Self-Efficacy in Behavior Management Scale (SEBMS) Items 1. When I really try, I can get through to most difficult students. 2. If a student in my class becomes disruptive and noisy, I feel assured that I know some techniques to redirect him or her quickly. 3. If I try hard, I can get through to even the most difficult or unmotivated students. 4. I can effectively work with disruptive students 5. I can manage almost any student behavior problem 6. The amount a student can learn is primarily related to family background. 7. A teacher is very limited in what he or she can achieve because a student's home environment is a large influence on his or her achievement. 8. When it comes right down to it, a teacher really can't do much because most of a student's home environment is a large influence on his or her achievement. 9. If students are not disciplined at home, they aren't likely to accept any discipline. 10. If parents would do more for their children, I could do more.

Calibration and Inter-rater Agreement of Observational Instrument
The observation instrument (see Appendix A) consists of eight items and uses a 5-point rating scale. Observers participated in an initial calibration exercise in which they were paired and watched a brief video recording of classroom teachers presenting their classroom management plans. The observers independently rated the teachers in these video recordings on the eight items in the observation instrument. If the observer did not feel that an item was represented in the video, they left that item blank on the observation instrument. After each exercise, the observers stopped and discussed their ratings and their reasons for assigning those ratings. In a subsequent session observers again independently rated teachers in video recordings and their ratings were compared with their paired partner's ratings to compute inter-rater agreement. Table 1 presents percent agreement across the pairings of the three observers. Deviation of one point (or 80% agreement) was considered adequate for this observation study, and this level of agreement was observed for nearly every comparison.

Management Plan Quality, Introduction, and Implementation
Participant performance on the three management plan metrics (quality of management plan, introduction to the plan in the first Teach Live session, and implementation of the plan in the second Teach Live session) is presented in Table 2. Results indicate that participants developed above average (moderate to strong) management plans with an aggregate score of 4.1. Procedures and mission statements were generally stronger than rules as well as consequences for compliance and non-compliance. However, performance fell sharply to an aggregate score of 3.3 when the participants introduced the plan; the most difficult area was presenting consequences for inappropriate behavior. Similarly, with an aggregate score of 3.6 participants were only moderately successful implementing their plans. Most challenging was responding to students which involved employing surface management techniques, using directed praise, addressing misbehavior in a positive fashion, and remaining consistent with their plans. Two-tailed paired-samples t-tests against an a-priori alpha of 0.05 determined that the mean participant observation ratings for the creation of the plan were significantly higher than those for the introduction of the plan (t = 4.61, p = 0.0001, 95% CI for the mean difference = 0.45 to 1.12), as well as those for the implementation of the plan (t = 2.97, p = 0.0417, 95% CI for the mean difference = 0.09 to 0.88). The mean differences for the observations were not statistically significantly different between the introduction of the plan and implementation of the plan (t = -1.72, p = 0.1449, 95% CI for the mean difference = -0.79 to 0.13). Cohen's d effect size for the difference between creating the plan and introducing the plan was calculated to be 1.36 (a large effect), and the effect size for the difference between creating the plan and implementing the plan was calculated to be 0.626 (a moderate effect). Taken together, participants were better able to create a classroom management plan than they were able to explain or implement the plan to the TeachLive avatars.

Self-Efficacy
To assess participants' levels of classroom management self-efficacy, the 10 item SEBMS was administered both prior to and after the intervention (see Table 3). Ideally, we would have collected pre-post self-efficacy data from matched non-participant peers. However, the timing of the study made such an effort an unreasonable request for the non-participants. We were able to collect preintervention data from 36 non-participant peers and established that there was no difference in participants' self-reported self-efficacy as compared with their non-participant peers (t = 0.46, df = 26, p = 0.650, 95% CI = -2.70 to 4.26). For intervention participants, self-efficacy increased after completing the intervention (t = 2.898, df = 14, p = 0.012, 95% CI = 1.01 to 6.73). Cohen's d effect size was calculated to assess the magnitude of this increase. The effect size (d = 0.61, a moderate effect) suggests that the participants' self-reported self-efficacy was more than half a standard deviation higher than their self-reported self-efficacy prior to beginning the intervention.

Discussion
Although effective classroom management is critical for student success (Scheuermann & Hall, 2016), many EPPs provide prospective teachers little formal preparation in this critical area (Begeny & Martens, 2006;Mitchell et al., 2017). The purpose of this study was to assess the efficacy of a multiplatform intervention to prepare preservice teachers who do not have the opportunity to participate in a specific classroom management course in evidence-based inclusive proactive classroom management practices. Specifically, participants received a 12-hour instructional module comprised of a direct instruction seminar that presented key management concepts (e.g., Lewis & Sugai, 1999;McLeskey et al., 2018;Rosenberg & Jackman, 2003); (2) guided practice through on-line IRIS Center modules (e.g., The IRIS Center, 2021;Sayeski et al., 2015); and (3) direct application of skills with feedback through mixed reality simulations (TeachLive; e.g., Dieker et al., 2008;Pas et al., 2016). Concepts emphasized in the intervention included formation of a management mission statement, development of rules and procedures, surface management of minor misbehavior, consequences for compliance and noncompliance, and displaying an appropriate management perspective when communicating and responding to student behavior. The impact of the intervention was assessed employing four dependent measures: quality of the classroom management plan, presentation of the plan, plan implementation, and self-efficacy. The results suggest that preservice teachers in an EPP that does not offer a formal course in classroom management can successfully develop quality proactive classroom management plans that include a mission, rules, procedures, and consequences. However, when put in applied simulated situations, participants struggled to present and implement their well-developed plans. Interestingly, even though the participants did not perform strongly on presenting and implementing their plans, SEBMS data increased significantly indicating that participants believed they were better able to manage classroom behavior than they did prior to the intervention. These findings are gratifying, sobering, and instructive. Specifically, our data indicate that a relatively brief supplemental intervention can provide opportunities for preservice teachers to acquire the necessary skills to develop a strong inclusive or universal management plan and increase their confidence or self-efficacy in addressing student behavior. Unfortunately, we also found that limited opportunities for practice and feedback when applying management skills in simulated or real situations do not fully sustain the impact of having strong plans. There was a significant decrease in performance when participants were presenting and implementing their plans in a simulated setting that replicated the critical first days of a school year, a time when presentations of behavioral expectations tend to be most impactful. Nonetheless, participant self-efficacy regarding classroom management increased even though they were, as a group, better able to create a classroom management plan than they were able to explain or implement the plan to the TeachLive avatars.

Acquisition and Application of Evidence-based Classroom Management Practices
One major positive outcome of the intervention was that participants acquired several evidence-based practices associated with inclusive universal classroom management. When we speak of acquiring these practices, we are referring to participants being able to understand what the practice looks like as well as how and under what conditions the practice is to be applied (Myers et al., 2017). Given that large numbers of preservice teachers either (a) do not have an opportunity to complete a specific course on classroom management or (b) receive infused activities that do not emphasize evidence-based practices (Flower et al., 2017;Freeman et al., 2014;Oliver & Reschly, 2010), this is a gratifying outcome. Specifically, after completing half of the intervention, participants were well versed in the importance of rules, procedures, and consequences, and were able to demonstrate their knowledge by designing their own creative plans and lessons to present the plans to their students. The direct instruction seminar and guided practice provided by the IRIS modules provided structure for students to develop their plan in a well-sequenced and logical fashion.
Unfortunately, data reflecting the presentation and implementation of the classroom management plans was sobering. Participants were unable to adequately present and implement their plans due to planned challenges in student behavior encountered with the TeachLive avatars. Although well-versed in what they wanted to present, participants seemed to be thrown by minor discipline issues (e.g., inattention, calling out, lack of interest), and had difficulty involving students in discussions and delivering basic surface management techniques. There were also several missed opportunities for using directed praise statements and in some cases, participants were unable to remain poised when questioned by the avatars. In terms of the content of the presentations, participants had the greatest amount of difficulty presenting consequences for inappropriate behavior and non-compliance. Although well-developed, participants appeared uncomfortable responding to a range of avatar questions about "what would happen if?" and "why is this important?" Participants only made modest performance increases during their second TeachLive sessions. Even with directed feedback, they still had difficulty implementing their plans as they taught classroom procedures, once again having difficulty responding to minor instances of student misbehavior and often neglecting to use directed praise statements. Several participants, caught up in the immediacy of responding to avatar behavior, neglected to align their responses to misbehavior with their plan.
Although gains were made (participants did exhibit moderate levels of performance), the current intervention did not appear to provide either an adequate amount of time for simulated practice or enough potent feedback of participant performance. In contrast to our effort to enhance the management skills of a cohort of students, most simulation efforts reported in the education literature are generally research prototypes evaluated in controlled experimental settings (Kaufman & Ireland, 2016). Evidence promoting enhanced fluency are mostly case studies and focused single subject multiple baseline design studies (e.g., Dawson & Lignugaris/Kraft, 2017;Peterson-Ahmad, 2018;Garland et al., 2012) that involved a limited number of preservice participants who received extensive coaching and follow-up activities. Our effort focused more on breadth of coverage, intervening with a larger group of volunteer undergraduate preservice teachers who agreed to participate in a supplemental learning activity. This breadth of coverage likely had consequences. Providing limited opportunities and brief feedback failed to result in adequate fluency (Myers et al., 2017) indicating that future efforts will require more occasions for application and substantive feedback. This could involve securing and investing more time and money into preservice simulation technologies (Kaufman & Ireland, 2016) or integrating explicit supervised classroom management application into fieldwork and student teaching activities. Such activities will require strategic programmatic reviews of how field experience opportunities align with classroom management competencies as well as how college and university resources are prioritized in a time of diminished financial support in higher education generally and for EPPs specifically. We know that already stretched public higher education budgets are suffering due to COVID-19, that enrollments will decline 15-25%, and revenue garnered from student tuition is plummeting (Dennis, 2020). In addition to having far less money to spend, many colleges and universities will continue to provide costly resources to support students by pivoting to online/remote education and/or making the face-to-face environment compliant with CDC safety standards.
Consequently, funding for EPP programmatic reviews as well as innovations such as simulations with targeted scenario development may be difficult to secure. That said, we believe supplemental programmatic interventions in classroom management may, in the long term, be cost-effective investments. For the current effort, a modest state funded grant provided resources for targeted scenario development, participant simulation time, and modest release time for faculty to observe and provided feedback on observations. Arguably, without such investments, it would be difficult to reduce future needs for costly and time consuming inservice professional development activities related to inappropriate school behavior (e.g., Mitchell et al., 2017) or reduce teacher attrition due to shortcomings in behavior management (e.g., Scott, 2017). That said, the cost of providing effective simulations remains prohibitive and, unfortunately, out of reach for many EPPs. Challenges remain as to how best to design and deliver cost-effective targeted application opportunities with adequate feedback and/or coaching.

Self-Efficacy
Despite uneven performance in presenting and implementing their management plans, participant levels of self-efficacy improved significantly with a moderate effect size. There are several possible explanations for this finding. Although studies (e.g., Reinke et al., 2013) have found that self-efficacy in behavior management is associated with improved management of inappropriate behavior, our findings are more consistent with those of Larson et al. (2018) and Debnam et al. (2015) who found that selfefficacy in behavior management was negatively or inconsistently associated with observed teacher use of proactive behavior management strategies. Questions remain regarding the validity of self-efficacy measures of classroom management performance and how one's perceptions of competence translate into actual performance. Regardless, it is likely that our participants experienced greater confidence in their ability to manage classroom behavior simply as a result of acquiring skills in what was a first sustained activity related to the topic. In fact, all the participants indicated that they were satisfied with the training and would recommend it to others in their EPP. Even though they received feedback indicating that fluency in management would require more practice, it is possible they found comfort in acquiring key basic skills and the opportunity that they would be able to practice these skills in subsequent field experiences. Given this possibility, it is incumbent on EPPs to ensure that adequate opportunities to practice inclusive universal classroom management are part of both in-person clinical activities as well as synchronous pandemic related remote fieldwork alternatives (Cave, 2020).

Limitations and Conclusions
Taken together, our findings suggest that a supplemental multi-platform intervention to promote inclusive or universal evidence-based classroom management practices can benefit preservice teachers who do not have an opportunity to participate in a specific classroom management course or sustained infused activities in methods courses. Given limited allocations of time (12 hours) and personnel available to provide feedback, these benefits are most potent in terms of acquisition and modest in terms of performance. Although self-efficacy does improve, it may be associated more with satisfaction associated with learning basic classroom management skills than with successfully meeting challenges associated with the presentation and implementation of management plans. That said, generalizing from our results requires attention to two methodological limitations. Specifically, in this study participants were undergraduate preservice teachers from one public comprehensive EPP within a comprehensive state university system. The programs enrolling these students do not require a formal course in classroom management and provide only limited infused instruction on behavior management in methods courses. Caution must be exercised when generalizing beyond these participant characteristics. Second, due to instructional time and programmatic considerations, we were unable to collect post-intervention self-efficacy data from control students. Consequently, we do not know if controls would have SEBMS scores that were similar or different than the participants.
In summary, many beginning teachers enter their classrooms and realize they are spending far too much academic time on discipline (Rosenberg & Jackman, 2003). Often these teachers lack the rudimentary skills to manage a classroom and attribute their shortcomings to deficiencies in their EPPs (Flower et al., 2017). With minimal credits available for preparation in the myriad areas (i.e., literacy, content area instruction, STEM), many EPPs are forced to make difficult curricular decisions. In situations where courses in classroom management are not offered, supplemental learning activities can be developed to promote acquisition of key management techniques. However, if we are to maximize the application of these techniques it is essential that EPPs make simulation platforms available for adequate amounts of time and with appropriate levels of feedback and/or coaching (Peterson-Ahmad, 2018). Future research should investigate the precise dosage and intensity for each of these components as well as the cost effectiveness of these expensive program enhancements. Clearly, EPPs have a road map for maximizing the effectiveness of their learning activities. What is needed is a strategy for scaling up and delivering these opportunities in an accessible and cost-effective manner.

Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
This project was funded through a TeachNY Implementation grant from the State University of New York (SUNY). ORCID iD Kate Ingraham https://orcid.org/0000-0003-4197-2335