blog posts and news stories

The Evaluation of CREATE Continues

Empirical Education began conducting the evaluation of Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness (CREATE) in 2015 under a subcontract with Atlanta Neighborhood Charter Schools (ANCS) as part of their Investing in Innovation (i3) Development grant. Since our last CREATE update, we’ve extended this work through the Supporting Effective Educator Development (SEED) Grant Program. The SEED grant provides continued funding for three more cohorts of participants and expands the research to include experienced educators (those not in the CREATE residency program) in CREATE schools. The grant was awarded to Georgia State University and includes partnerships with ANCS, Empirical Education (as the external evaluator), and local schools and districts.

Similar to the i3 work, we’re following a treatment and comparison group over the course of the three-year CREATE residency program and looking at impacts on teacher effectiveness, teacher retention, and student achievement. With the SEED project, we will also be able to follow Cohort 3 and 4 for an additional 1-2 years following residency. Surveys will measure perceived levels of social capital, school climate and community, collaboration, resilience, and mindfulness, in addition to other topics. Recruitment for Cohort 4 began this past spring and continued through the summer, resulting in approximately 70 new participants.

One of the goals of the expanded CREATE programming is to support the effectiveness and social capital of experienced educators in CREATE schools. Any experienced educator in a CREATE school who attends CREATE professional learning activities will be invited to participate in the research study. Surveys will measure similar topics to those measured in the quasi-experiment and we conduct individual interviews with a sample of participants to gain an in-depth understanding of the participant experience.

We have completed our first year of experienced educator research and continue to recruit participants, on an ongoing basis, into the second year of the study. We currently have 88 participants and counting.

2018-10-03

Determining the Impact of CREATE on Math and ELA Achievement

Empirical Education is conducting the evaluation of Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness (CREATE) under an Investing in Innovation (i3) development grant awarded in 2014. The CREATE evaluation takes place in schools throughout the state of Georgia.

Approximately 40 residents from the Georgia State University (GSU) College of Education (COE) are participating in the CREATE teacher residency program. Using a quasi-experimental design, outcomes for these teachers and their students will be compared to those from a matched comparison group of close to 100 teachers who simultaneously enrolled in GSU COE but did not participate in CREATE. Implementation for cohort 1 started in 2015, and cohort 2 started in 2016. Confirmatory outcomes will be assessed in years 2 and 3 of both cohorts (2017 - 2019).

Confirmatory research questions we will be answering include:

What is the impact of one-year of exposure of students to a novice teacher in their second year of teacher residency in the CREATE program, compared to the Business as Usual GSU teacher credential program, on mathematics and ELA achievement of students in grades 4-8, as measured by the Georgia Milestones Assessment System?

What is the impact of CREATE on the quality of instructional strategies used by teachers, as measured by the Teacher Assessment of Performance Standards (TAPS) scores, at the end of the third year of residency, relative to the business as usual condition?

What is the impact of CREATE on the quality of the learning environment created by teachers, as measured by Teacher Assessment of Performance Standards (TAPS) scores, at the end of the third year of residency, relative to the business as usual condition?

Exploratory research questions will address additional teacher-level outcomes including retention, effectiveness, satisfaction, collaboration, and levels of stress in relationships with students and colleagues.

We plan to publish the results of this study in fall of 2019. Please check back to read the research summary and report.

2017-06-06

Meeting Long-time Friends and New Partners at #i3PD2015

Empirical sent five staff members to the Invest in Innovation (i3) Project Director’s meeting in Washington to support the five i3 evaluations we are currently conducting. Denis Newman, Andrew Jaciw, Jenna Zacamy, Megan Toby and Adam Schellinger. The meetings were filled with formal and informal meetings with partners, members of the i3 technical assistance teams, and old friends. Projects we are currently evaluating are RAISE, Aspire Public Schools, iRAISE, Making Sense of Science, and CREATE. We are currently at work on proposals for the 2016 round of awards.

2015-06-10

Empirical Education Focuses on Local Characteristics at the 14th Annual CREATE Conference

Empirical Education staff presented at the National Evaluation Institute’s (NEI) 14th annual CREATE conference in Wilmington, North Carolina. Both presentations focused on the local characteristics of the evaluations. Dr. Denis Newman, president of Empirical Education, and Jenna Zacamy, research manager, presented a randomized experiment which evaluated the impact of a pre-algebra curriculum (Carnegie Learning’s Cognitive Tutor Bridge to Algebra) being introduced in a pilot program in the Maui School District. The district adopted the program based in part on previous research showing substantial positive results in Oklahoma (Morgan & Ritter 2002). Given the unique locale and ethnic makeup in Maui, a local evaluation was warranted. District educators were concerned in particular with their less experienced teachers and with ethnic groups considered at risk. Unlike in prior research, we found no overall impact although for the algebraic operations subscale, low scoring students benefited from being in the Cognitive Tutor classes indicating that the new program could help to reduce the achievement gaps of concern. We also found for the overall math scale that uncertified teachers were more successful with their Cognitive Tutor classes than their conventional classes. Dr. Newman also presented work co-authored with Marco Muñoz and Andrew Jaciw on a quasi-experimental comparison, conducted by Empirical Education and Jefferson County (KY) schools, of an activity-based middle-school science program (Premier Science) to more traditional textbook programs. All the data were supplied by the district including a rating of quality of implementation. The primary pretest and outcome measures were tests of science and reading achievement. While there was no discernible difference overall, poor readers gained more from the non-textbook approach, helping to diminish an achievement gap of concern to the district.

2008-12-15
Archive