Teacher residencies are a promising model for teacher preparation. One such program is Georgia State University’s (GSU) CREATE Teacher Residency program. CREATE seeks to raise student achievement in local high‐needs schools by increasing the effectiveness and retention of both new and veteran educators. CREATE aims to achieve this by developing critically-conscious, compassionate, and skilled educators. Empirical Education has partnered with CREATE since 2014 to serve as a thought-partner and external evaluator on several U.S. Department of Education grants.
Our first quasi-experiment to study the efficacy of CREATE was funded by an Investing in Innovations (i3) grant. The grant followed the first two cohorts of study participants (Cohort 1 and 2), both containing CREATE and comparison participants. We followed each cohort for three years, with Cohort 1 starting in the 2015–16 school year and Cohort 2 starting in the 2016–17 school year.
We conducted confirmatory analyses on Teacher Assessment on Performance Standards (TAPS) ratings. There were no statistically significant effects of CREATE on Instructional Strategies (p = .221) or Positive Learning Environment (p = .192). In terms of student achievement, there was no statistically significant effect of CREATE on students’ ELA scores (p = .454), math scores (p = .569), and general achievement (p = .234).
However, exploratory analyses showed very promising results for teacher retention. We found undisrupted retention of CREATE residents over a three-year time period, relative to the comparison group (p =.038). This time period spanned from participants’ final year in a teacher credentialing program at Georgia State University through their second year as a teacher of record in a Georgia Public School.
The favorable impact on teacher retention was driven by higher continuous retention of Black educators in the CREATE program, relative to the comparison group (p = .021). The percentages of Black educators in CREATE, averaged across the two study cohorts, who maintained an uninterrupted trajectory of graduating from GSU CEHD, and taught in their first and second year were 98.6%, 96.3%, and 95.5%, respectively. In the matched comparison group of Black educators, the corresponding values were 85.7%, 68.5%, and 62.8%.
These results were further corroborated by statistically significant or marginally statistically significant differential impacts favoring Black teachers on several potential mediators: resilience, self-efficacy, and stress management and empathy related to teaching. The positive findings on retention, particularly for Black educators, is important given that 44% of teachers in Georgia leave the profession within the first five years, with evidence in the literature indicating that Black teachers experience higher turnover rates than non-Black teachers (Carver-Thomas & Darling-Hammond, 2017).
The authors discuss a number of factors that could have led to null impacts on confirmatory outcomes. The lack of variation in teacher performance ratings, which is a known issue in Georgia and in the literature at large (Weisberg et al., 2009; Kraft & Gilmour, 2017), could have contributed to the results for the TAPS ratings. Studies of teacher residency programs also suggest that impact on student achievement might not be present or detected in earlier years of teaching. Additionally, analyses of student achievement in this study were limited by small sample sizes of teachers in tested grades and subjects. The authors are addressing these limitations and investigating the promising outcomes on retention, especially for Black educators, in follow-on studies of CREATE (see below for research through the tenth cohort of study participants).
This first study was presented at the 2020 virtual conferences for the American Educational Research Association and the Society for Research on Educational Effectiveness (SREE) in September 2020. Download the CREATE 2020 slide deck or watch the video recording of the virtual presentation at AERA in 2020. Read the final report using the blue buttons below.
Our second quasi-experiment to study the efficacy of CREATE was funded by a Supporting Effective Educator Development (SEED) Grant. These funds supported evaluation activities for three additional cohorts of study participants (Cohorts 3–5). Cohort 3 joined the study during the 2017-18 school year, Cohort 4 during the 2018–19 school year, and Cohort 5 during the 2019–20 school year. Like with the first study, each cohort was comprised of CREATE and comparison participants. We followed each cohort for three years spanning from their final year in a teacher credentialing program at Georgia State University through their second year as a teacher of record in a Georgia Public School.
By the end of their second year as a teacher of record, CREATE residents reported lower levels of teacher burnout, higher levels of self-compassion (Standardized Effect Size [ES] = .51, p = .06), and higher levels of stress management and empathy related to teaching (ES = .34, p = .25), relative to the comparison group. These outcomes are theoretically posited to mediate longer-term outcomes of teacher effectiveness, retention, and student achievement. The variability in responses across cohorts and over time provides additional context for these results, especially for the period of program adaptations necessitated by COVID-19. For example, as part of descriptive exploration, we found the largest differences favoring CREATE for Cohort 5, which is the cohort that entered teaching during the onset of the COVID-19 pandemic. Typical year-end trends in mean scores by cohort are displayed in these two figures.
This evaluation of Cohorts 3-5 demonstrates a predominantly successful implementation of CREATE, robust treatment-control contrasts on several program components, and trends towards favorable impacts on key potential mediators of impact. However, COVID-19 curtailed the opportunity to carry out the intended evaluation of impacts of CREATE on important distal outcomes. Notably, in response to COVID-19, the state suspended collection of teacher performance data (including on project-critical dimensions of quality of instruction and educational environment). Several districts also reasonably prohibited the alternative of using in-person classroom observations. Further, suspension of state testing in 2020 limited the sample of students for evaluating impacts on achievement. Further, the study did not replicate the positive and statistically significant impacts of CREATE on teacher retention, especially for Black educators, as was observed in the previous i3 grant. This may reflect the effects of COVID-19 on GSU CEHD programming, recruitment and hiring mechanisms. We will continue to explore these outcomes in future grants
Read the final report using the blue buttons below.
Our third quasi-experimental study—examining the efficacy of CREATE—was supported by a second SEED grant awarded in 2020. This grant expanded the evaluation of GSU's CREATE program by adding three new participant cohorts (Cohorts 6–8) and introducing a study of CREATE Cooperating Teachers (CTs)—the mentor teachers who host CREATE residents in their classrooms during the pre-service year. The third study also examines fidelity of implementation (FOI), program impacts on teacher well-being and retention, and the extent to which CREATE has scaled within partner schools and across APS.
Results from the CT study showed a greater proportion of CREATE CTs being very confident or extremely confident in their understanding of how to effectively mentor pre-service or novice teachers (87%, n = 48), compared with comparison CTs (77%, n = 10). CREATE CTs met with their residents more frequently, one-on-one, outside of active classroom teaching time, with 85% (n = 39) reporting meeting either daily (46%) or more than one day per week (39%). Only half (50%, n = 3) of comparison CTs reported the same.
The topics discussed during these interactions were notably different between the two groups. CREATE CTs most frequently reported discussing (1) curriculum standards and/or lesson planning, (2) feedback for their resident, and (3) relationships with students, parents, other teachers, and/or administrators. Comparison CTs, by contrast, most often reported discussing (1) curriculum standards and/or lesson planning, and (2) classroom procedures and logistics. Overall, findings suggest that CREATE CTs may engage residents in more frequent mentoring interactions and conversations that extend beyond lesson planning and classroom logistics to include feedback, relationships, reflection, well-being, pedagogy, and identity.
As outlined in the final report, FOI analyses included six core components of CREATE. Progressive Core Classroom Roles (Key Component 1) was implemented with fidelity across all years. Recruitment and Training of CTs (Key Component 2) and Multiple Forms of Mentoring while Teaching (Key Component 3) fell just below fidelity thresholds due to slightly lower-than-target participation in required mentor training. Fidelity for the Summer Residency Academy (Key Component 4) was achieved for second-year residents but not third-year residents. Equity-Centered Professional Learning (PL) (Key Component 5) met fidelity expectations for Years 1 and 2 but not Year 3, primarily because of lower resident participation in Together Time meetings and non-Together Time PL. Similarly, Compassion-Based PL (Key Component 6) met fidelity expectations for Year 2 residents but not Years 1 or 3, due to lower than expected attendance at Together Time meetings.
Overall, CREATE set ambitious fidelity thresholds. FOI findings indicate strong implementation of CREATE's core clinical experiences, with the primary implementation challenges related to mentor training participation and engagement in PL outside of Together Time. Despite these implementation challenges, CREATE produced several promising indicators of program success based on participant experiences. By teachers' second year as the sole teacher of record (Year 3), all CREATE residents (100%) continued to have a mentor teacher, compared with 78% of comparison teachers. CREATE participants also reported substantially higher levels of school-based support, with 37% indicating they felt more than moderately or very supported at their school site, compared with 11% of comparison teachers. Reflecting CREATE's co-teaching model, only 14% (n = 3) of CREATE participants served as the sole teacher of record throughout their entire first year of teaching, compared with 55% (n = 11) of comparison participants. Qualitative findings identified this gradual transition to independent teaching as a contributor to participants' sense of support. In addition, 96% of CREATE residents reported that they would have experienced moderate or significant financial hardship without the monetary stipends provided by the program.
CREATE also had significant positive impacts on multiple dimensions of teacher well-being. Participants reported a stronger sense of purpose, value, and direction in life, as measured by the mPERMA (ES = 0.90, p = .017). CREATE significantly improved teachers' sense of equity in their roles and experiences—including their ability to connect with students from diverse backgrounds, pursue equity goals, discuss equity-related issues, and feel valued and respected within their schools (ES = 0.77, p = .032). In addition, CREATE increased teachers' perceived opportunities and support to engage with issues of social justice and racial equity (ES = 1.49, p = .008) and strengthened their confidence in understanding concepts related to race and equity (ES = 0.75, p = .025).
Although no statistically significant differences were found for on-time graduation or teacher retention, both CREATE and comparison teachers achieved high rates on these outcomes. Over 90% of CREATE and comparison teachers graduated on time from GSU, over 80% in both groups completed their first year of teaching, and over 70% in both groups completed their second year of teaching.
Qualitative findings suggested that CREATE contributed to meaningful program scaling at the individual and classroom levels by influencing teachers' mindsets, relationships with students and families, and instructional practices. School-level scaling was supported by leadership, planning time, financial resources, substitute coverage, and aligned policies, while participants and alumni extended the program's reach through peer collaboration, recruitment, and PL communities. At the same time, limited leadership buy-in, competing priorities, time constraints, and resistance to equity-oriented initiatives constrained implementation. There was limited evidence of district-level scaling.
Planned analyses of student achievement and teacher effectiveness could not be conducted because complete student outcome data and Teacher Assessment on Performance Standards (TAPS) data were unavailable during these study years. The 2020 SEED grant also supported our first economic evaluation of CREATE through a cost-effectiveness study. Although this analysis remains incomplete due to challenges with the cost analysis portion, it has provided important insights into the methodological challenges of conducting economic evaluations of constantly evolving and equity-centered teacher residency programs, outlined in the appendix of the final report.
Read the final report using the blue buttons below.
Our fourth quasi-experimental study of CREATE’s efficacy was funded through a second SEED grant awarded in 2022. The study was designed to follow two cohorts of CREATE and comparison participants, Cohorts 9 and 10, for three years each. Cohort 9 entered the study during the 2023–24 school year and Cohort 10 during the 2024–25 school year. However, the federal government terminated the SEED grant in February 2025.
Using a quasi-experimental design and mixed-methods process evaluation, the study aimed to examine CREATE’s impact on teachers’ equitable and effective classroom practices, student achievement, and teacher retention. These outcomes were to be examined across residents (pre-service and early-career teachers), CTs (mentor teachers), and other experienced educators. We also planned to continue investigating teacher well-being and self-compassion.
An important addition to this phase of the research was a planned cost-benefit analysis focused on understanding the economic value CREATE provides to local stakeholders, with the work centered in the Atlanta community. This would have required us to complete the cost analysis work we started during our third study (see above). With our own funds, we could have provided the cost-effective estimates we aimed to deliver. We also planned and initiated a classroom observation study using Danielson's Framework for Teaching (FFT) instead of TAPS data, which would have provided rich data about teachers’ learning environments and experiences intended to deepen our examination of the role that principals and other school and district leaders play in supporting teachers and shaping CREATE’s implementation and impact. However, the early termination of the grant prevented us from completing these planned lines of inquiry.
