Teacher residencies are a promising model for teacher preparation. One such program is Georgia State University’s (GSU) CREATE Teacher Residency program. CREATE seeks to raise student achievement in local high‐needs schools by increasing the effectiveness and retention of both new and veteran educators. CREATE aims to achieve this by developing critically-conscious, compassionate, and skilled educators who are committed to teaching practices that prioritize racial justice and interrupt inequities. Empirical Education has partnered with CREATE since 2014 to serve as a thought-partner and external evaluator on several U.S. Department of Education grants.
Our first quasi-experiment to study the efficacy of CREATE was funded by an Investing in Innovations (i3) grant. The grant followed the first two cohorts of study participants (Cohort 1 and 2), both containing CREATE and comparison participants. We followed each cohort for three years, with Cohort 1 starting in the 2015–16 school year and Cohort 2 starting in the 2016–17 school year.
We conducted confirmatory analyses on Teacher Assessment on Performance Standards (TAPS) ratings. There were no statistically significant effects of CREATE on Instructional Strategies (p = .221) or Positive Learning Environment (p = .192). In terms of student achievement, there was no statistically significant effect of CREATE on students’ ELA scores (p = .454), math scores (p = .569), and general achievement (p = .234).
However, exploratory analyses showed very promising results for teacher retention. We found undisrupted retention of CREATE residents over a three-year time period, relative to the comparison group (p =.038). This time period spanned from participants’ final year in a teacher credentialing program at Georgia State University through their second year as a teacher of record in a Georgia Public School.
The favorable impact on teacher retention was driven by higher continuous retention of Black educators in the CREATE program, relative to the comparison group (p = .021). The percentages of Black educators in CREATE, averaged across the two study cohorts, who maintained an uninterrupted trajectory of graduating from GSU CEHD, and taught in their first and second year were 98.6%, 96.3%, and 95.5%, respectively. In the matched comparison group of Black educators, the corresponding values were 85.7%, 68.5%, and 62.8%.
These results were further corroborated by statistically significant or marginally statistically significant differential impacts favoring Black teachers on several potential mediators: resilience, self-efficacy, and stress management and empathy related to teaching. The positive findings on retention, particularly for Black educators, is important given that 44% of teachers in Georgia leave the profession within the first five years, with evidence in the literature indicating that Black teachers experience higher turnover rates than non-Black teachers (Carver-Thomas & Darling-Hammond, 2017).
The authors discuss a number of factors that could have led to null impacts on confirmatory outcomes. The lack of variation in teacher performance ratings, which is a known issue in Georgia and in the literature at large (Weisberg et al., 2009; Kraft & Gilmour, 2017), could have contributed to the results for the TAPS ratings. Studies of teacher residency programs also suggest that impact on student achievement might not be present or detected in earlier years of teaching. Additionally, analyses of student achievement in this study were limited by small sample sizes of teachers in tested grades and subjects. The authors are addressing these limitations and investigating the promising outcomes on retention, especially for Black educators, in follow-on studies of CREATE (see below for research through the tenth cohort of study participants).
This first study was presented at the 2020 virtual conferences for the American Educational Research Association and the Society for Research on Educational Effectiveness (SREE) in September 2020. Download the CREATE 2020 slide deck or watch the video recording of the virtual presentation at AERA in 2020. Read the final report using the blue buttons below.
Our second quasi-experiment to study the efficacy of CREATE was funded by a Supporting Effective Educator Development (SEED) Grant. These funds supported evaluation activities for three additional cohorts of study participants (Cohorts 3–5). Cohort 3 joined the study during the 2017-18 school year, Cohort 4 during the 2018–19 school year, and Cohort 5 during the 2019–20 school year. Like with the first study, each cohort was comprised of CREATE and comparison participants. We followed each cohort for three years spanning from their final year in a teacher credentialing program at Georgia State University through their second year as a teacher of record in a Georgia Public School.
By the end of their second year as a teacher of record, CREATE residents reported lower levels of teacher burnout, higher levels of self-compassion (Standardized Effect Size [ES] = .51, p = .06), and higher levels of stress management and empathy related to teaching (ES = .34, p = .25), relative to the comparison group. These outcomes are theoretically posited to mediate longer-term outcomes of teacher effectiveness, retention, and student achievement. The variability in responses across cohorts and over time provides additional context for these results, especially for the period of program adaptations necessitated by COVID-19. For example, as part of descriptive exploration, we found the largest differences favoring CREATE for Cohort 5, which is the cohort that entered teaching during the onset of the COVID-19 pandemic. Typical year-end trends in mean scores by cohort are displayed in these two figures.
This evaluation of Cohorts 3-5 demonstrates a predominantly successful implementation of CREATE, robust treatment-control contrasts on several program components, and trends towards favorable impacts on key potential mediators of impact. However, COVID-19 curtailed the opportunity to carry out the intended evaluation of impacts of CREATE on important distal outcomes. Notably, in response to COVID-19, the state suspended collection of teacher performance data (including on project-critical dimensions of quality of instruction and educational environment). Several districts also reasonably prohibited the alternative of using in-person classroom observations. Further, suspension of state testing in 2020 limited the sample of students for evaluating impacts on achievement. Further, the study did not replicate the positive and statistically significant impacts of CREATE on teacher retention, especially for Black educators, as was observed in the previous i3 grant. This may reflect the effects of COVID-19 on GSU CEHD programming, recruitment and hiring mechanisms. We will continue to explore these outcomes in future grants
Read the final report using the blue buttons below.
Our third quasi-experiment to study the efficacy of CREATE was funded by another SEED grant in 2020. This SEED award funds new work with Georgia State University’s CREATE program by adding three additional cohorts of study participants (Cohorts 6–8) and conducting a quasi-experiment to study the efficacy of CREATE Cooperating Teachers (that is, the mentor teachers who host CREATE residents in their classrooms). The study examines impacts on teacher effectiveness and retention, student achievement, as well as other mediating outcomes. This new SEED award also supports rapid cycle formative feedback through plan-do-study-act cycles, and our first economic evaluation of CREATE through a cost effectiveness study.
We will post the final report of the third study here in 2023 or 2025, depending on continual funding.
Our fourth quasi-experiment to study the efficacy of CREATE was funded by another SEED grant in 2022. It supports Cohort 9 and Cohort 10 of study participants (CREATE and comparison) for three years per cohort, with Cohort 9 joining the study for the 2023–24 school year. Cohort 10 will join for the 2024–25 school year.
Using a quasi-experimental design and mixed-methods process evaluation, we aim to understand the impact of CREATE on teachers’ equitable and effective classroom practices, student achievement, and teacher retention. We will continue to study this impact with residents (pre-service and early career teachers), cooperating teachers (mentor teachers), and other experienced educators. We are also expanding our research to include more work unpacking the role of principals and other school/district leadership. We will continue to explore teacher well-being, self-compassion, and the cost effectiveness of CREATE. Adding a cost-benefit component to our work with this new funding is important. Specifically, we want to understand the economic benefit CREATE offers to local stakeholders, centering this work in the Atlanta community.
We will post the final report of the third study here in 2024 or 2026, depending on continual funding.
