blog posts and news stories

Two New Studies for Regional Education Laboratory (REL) Southwest Completed

Student Group Differences in Arkansas Indicators of Postsecondary Readiness and Success

It is well documented that students from historically excluded communities face more challenges in school. They are often less likely to obtain postsecondary education, and as a result see less upward social mobility. Educational researchers and practitioners have developed policies aimed at disrupting this cycle. However, an important factor necessary to make these policies work is the ability of school administrators to identify students that are at risk of not reaching certain academic benchmarks and/or exhibit certain behavioral patterns that are correlated with future postsecondary success.

Arkansas Department of Education (ADE), like education authorities in many other states, is tracking K-12 students’ college readiness and enrollment and collecting a wide array of student progress indicators meant to predict their postsecondary success. A recent study by Regional Education Laboratory (REL) Southwest showed that a logistic regression model that uses a fairly small number of such indicators, measured as early as in seventh or eighth grade, predicts with a high degree of accuracy whether students will enroll in college four or five years later (Hester et al., 2021). But does this predictive model – and the entire “early warning” system that could rely on it – work equally well for all student groups? In general, predictive models are designed to reduce average prediction error. So, when the dataset used for predictive modeling covers several substantially different populations, the models tend to make more accurate predictions for the largest subset and less accurate for the rest of the observations. Meaning, if the sample your model relies on is mostly White, it will most accurately predict outcomes for White students. In addition, predictive strength of some indicators may vary across student groups. In practice, this means that such a model may turn out to be less useful to forecast the outcome for those students who should benefit the most from it

Researchers from Empirical Education and AIR teamed up to complete a study for REL Southwest that focuses on the differences in predictive strength and model accuracy across student groups. It was a massive analytical undertaking based on nine years of tracking two cohorts of six graders from the whole state, close to 80,000 records and hundreds of variables including student characteristics (including gender, race/ethnicity, eligibility for the National School Lunch Program, English learner student status, disability status, age, and district locale), middle and high school academic and behavioral indicators, and their interactions. First, we found that several student groups—including Black and Hispanic students, students eligible for the National School Lunch Program, English learner students, and students with disabilities—were substantially less likely (by 10 percentage points or more) to be ready for or enroll in college than students without these characteristics. However, our main finding, and a reassuring one, is that the model’s predictive power and predictive strength of most indicators is similar across student groups. In fact, the model often does a better job predicting postsecondary outcomes for those student groups in most need of support.

Let’s talk about what “better” means in a study like that. It is fair to say that statistical model quality is seldom of particular interest in educational research and is often limited to a footnote showing the value of R2 (the proportion of variation in the outcome explained by independent variables). It can tell us something about the amount of “noise” in the data, but it is hardly something that policy makers are normally concerned with. In the situation where the model’s ability to predict a binary outcome—whether or not the student went to college—is the primary concern, there is a clear need for an easily interpretable and actionable metric. We just need to know how often the model is likely to predict the future correctly based on current data.

Logistic regression, which is used for predicting binary outcomes, produces probabilities of outcomes. When the predicted probability (like that of college enrollment) is above fifty percent, then we say that it predicts success (“yes, this student will enroll”), and it predicts failure (“no, they will not enroll”) otherwise. When the actual outcomes are known, we can evaluate the accuracy of the model. Counting the cases in which the predicted outcome coincides with the actual one and dividing it by the total number of cases yields the overall model accuracy. The model accuracy is a useful metric that is typically reported in predictive studies with binary outcomes. We found, for example, that the model accuracy in predicting college persistence (students completing at least two years of college) is 70% when only middle school indicators are used as predictors, and it goes up to 75% when high school indicators are included. These statistics vary little across student groups, by no more than one or two percentage points. Although it is useful to know that outcomes two years after graduation from high school can be predicted with a decent accuracy in as early as eighth grade, the ultimate goal is to ensure that students at risk of failure are identified while schools still can provide them with necessary support. Unfortunately, such a metric as the model accuracy is not particularly helpful in this case.

Instead, a metric called “model specificity” in the parlance of predictive analytics lets us view the data from a different angle. It is calculated as a proportion of correctly predicted negative outcomes alone, ignoring the positive ones. The model specificity metric turns out to vary across student groups a lot in our study but the nature of this variation validates the ADE’s system: for the student groups in most need of support, the model specificity is higher than for the rest of the data. For some student groups, the model can detect that a student is not on track to postsecondary success with near certainty. For example, failure to attain college persistence is correctly predicted from middle school data in 91 percent of cases for English learner students compared to 65 percent for non-English learner students. Adding high school data into the mix lowers the gap—to 88 vs 76 percent—but specificity is still higher for the English learner students, and this pattern holds across all other student groups.

The predictive model used in the ADE study can certainly power an efficient early warning system. However, we need to keep in mind what those numbers mean. For some students from historically excluded communities, their early life experiences create significant obstacles down the road. Some high schools are not doing enough to put these students on a new track that would ensure college enrollment and graduation. It is also worth noting that while this study provides evidence that ADE has developed an effective system of indicators, the observations used in the study come from the two cohorts of students who were in the sixth graders in 2008–09 and 2009–10. Many socioeconomic conditions have changed since then. Thus, the only way to assure that the models remain accurate is to proceed from isolated studies to building “live” predictive tools that would update the models as soon as a new annual batch of outcome data becomes available.

Read complete report titled “Student Group Differences In Arkansas’ Indicators of Postsecondary Readiness and Success” here.

Early Progress and Outcomes of a Grow Your Own Grant Program for High School Students and Paraprofessionals in Texas

Shortage and high turnover of teachers is a problem that rural schools face across the nation. Empirical Education researchers have contributed to the search for solutions for this problem several times in recent years, including two studies completed for REL Southwest (Sullivan, et al., 2017; Lazarev et al., 2017). While much of the policy research is focused on the ways to recruit and retain credentialed teachers, some states are exploring novel methods to create new pathways into the profession that would help create new local teacher cadres. One such promising initiative is Grow Your Own (GYO) program funded by the Texas Education Agency (TEA). Starting in 2019, TEA provides grants to schools and districts that intend to expand the local teacher labor force through one or both of the following pathways. The first pathway offers high school students an early start in teacher preparation through a sequence of education and training courses. The second pathway aims to help paraprofessionals already employed by schools to transition into teaching positions by covering tuition for credentialing programs, as well as offering a stipend for living expenses.

In the joint project with AIR, Empirical Education researchers explored the potential of the first pathway for high school students to address teacher shortages in rural communities and to increase the diversity of teachers. Since this is a new program, our study was based on the first two years of implementation. We found promising evidence that GYO can positively impact rural communities and increase teacher diversity. For example, GYO grants were allocated primarily to rural and small town communities, and programs were implemented in smaller schools with a higher percentage of Hispanic students and economically disadvantaged students. Participating schools also had higher enrollment in the teacher preparation courses. In short, GYO seems to be reaching rural areas with smaller and more diverse schools, and is boosting enrollment in teacher preparation courses in these areas. However, we also found that fewer than 10% of students in participating districts completed at least one education and training course, and fewer than 1% of students completed the full sequence of courses. Additionally, white and female students are overrepresented in these courses. These and other preliminary results will help the state education agency to fine-tune the program and work toward a successful final result: a greater number and increased diversity of effective teachers who are from the community in which they teach. We look forward to continuing research on the impact of “Grow Your Own.”

Read complete report titled “Early Progress and Outcomes of a Grow Your Own Grant Program for High School Students and Paraprofessionals in Texas” here.

All research Empirical Education has conducted for REL Southwest can be found on our REL-SW webpage.


Hester, C., Plank, S., Cotla, C., Bailey, P., & Gerdeman, D. (2021). Identifying Indicators That Predict Postsecondary Readiness and Success in Arkansas. REL 2021-091. Regional Educational Laboratory Southwest.

Lazarev, V., Toby, M., Zacamy, J., Lin L., & Newman, D. (2017). Indicators of Successful Teacher Recruitment and Retention in Oklahoma Rural Schools (REL 2018–275). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.

Sullivan, K., Barkowski, E., Lindsay, J., Lazarev, V., Nguyen, T., Newman, D., & Lin, L. (2017). Trends in Teacher Mobility in Texas and Associations with Teacher, Student, and School Characteristics (REL 2018–283). U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.


Evidentally Rejoins Empirical Education

On 10/31/18, we formed Evidentally from a set of projects that Empirical Education called Evidence as a Service (EaaS). The idea was to build a set of products that would automate much of the labor-intensive portions of the research process, e.g., statistical analysis, data cleaning, and similar efforts. Building on the education technology (edtech) efficiencies, particularly the collection of edtech usage data, would make it possible for non-researchers such as school administrators to conduct efficacy research.

By lowering the cost of research and introducing an ease into the research process, we are able to increase the number of valid studies that could be combined in a meta-analysis for generalizable results. The notion was that, as a product company, Evidentally could attempt to get investments unavailable to services companies such as Empirical Education. Unfortunately, for several reasons, Evidentally was unable to get that investment.

The intellectual property and projects of Evidentally have been returned to Empirical Education as of 12/7/22, and Evidentally (as its own entity) was dissolved. While the team is still committed to building the Evidence as a Service suite of tools, it will be conducted as a project of Empirical Education, under the branding of the Evidentally Evidence Suite. The Evidentally product is just one piece of Empirical Education’s education evidence offerings, when an education application or curriculum needs evidence of efficacy meeting any of the Every Students Succeeds Act (ESSA) Tiers of evidence.


Happy New Year from Empirical Education

To ring in the new year, we want to share this two-minute video with you. It comprises highlights from 2022 from each person on our team. We hope you like it. Cheers to a healthy and prosperous 2023!

My colleagues appear in this order in the video.

Happy New Year photo by Sincerely Media


Studying the Impacts of CAPIT Reading: An Early Literacy Program in Oklahoma

Empirical Education’s Evidentally recently conducted a study to evaluate the impact of CAPIT Reading on student early literacy achievement. The study utilized a quasi-experimental comparison group design using data from 12 elementary schools in a suburban school district in Oklahoma during the 2019–20 school year.

CAPIT Reading is a comprehensive PK–2 literacy solution that includes a digital phonics curriculum and teacher professional development. The program is a teacher-led phonemic awareness and phonics curriculum that includes lesson plans, built-in assessments, and ongoing support.

Four schools used CAPIT to supplement their literacy instruction for kindergarten students (treatment group) while eight schools did not (comparison group). The study linked CAPIT usage data and district demographic and achievement data to estimate the impact of CAPIT on the Letter Word Sounds Fluency (LWSF) and Early Literacy Composite scores of the aimsweb reading assessment, administered by the district in August and January.

We found a positive impact of CAPIT Reading on student early reading achievement on the aimsweb assessment for kindergarten students. This positive impact was estimated at 4.4 test score points for the aimsweb Early Literacy Composite score (effect size = 0.17; p = 0.01) and 7.8 points for the LWSF score (effect size = 0.29; p < 0.001). This impact on the LWSF score is equivalent to a 29% increase in growth for the average CAPIT student from the fall to winter tests.

We found limited evidence of differential impact favoring student subgroups, meaning that this positive impact for CAPIT users did not vary according to student characteristics such as eligibility for free and reduced-price lunch, race, or gender. We found that the impact on aimsweb overall was marginally greater for special education students by 4.9 points (p = 0.09) and that the impact on LWSF scores was greater for English Language Learners by 7.4 points (p = 0.09). Impact of CAPIT reading does not vary significantly across other student groups.

Read the CAPIT Reading Student Impact Report for more information on this early literacy research.


SREE 2022 Annual Meeting

When I read the theme of the 2022 SREE Conference, “Reckoning to Racial Justice: Centering Underserved Communities in Research on Educational Effectiveness”, I was eager to learn more about the important work happening in our community. The conference made it clear that SREE researchers are becoming increasingly aware of the need to swap individual-level variables for system-level variables that better characterize issues of systematic access and privilege. I was also excited that many SREE researchers are pulling from the fields of mixed methods and critical race theory to foster more equity-aligned study designs, such as those that center participant voice and elevate counter-narratives.

I’m excited to share a few highlights from each day of the conference.

Wednesday, September 21, 2022

Dr. Kamilah B. Legette, University of Denver

Dr. Kamilah B Legette masked and presenting at SREE

Dr. Kamilah B. Legette from the University of Denver discussed their research exploring the relationship between a student’s race and teacher perceptions of the student’s behavior as a) severe, b) inappropriate, and c) indicative of patterned behavior. In their study, 22 teachers were asked to read vignettes describing non-compliant student behaviors (e.g., disrupting storytime) where student identity was varied by using names that are stereotypically gendered and Black (e.g., Jazmine, Darnell) or White (e.g., Katie, Cody).

Multilevel modeling revealed that while student race did not predict teacher perceptions of behavior as severe, inappropriate, or patterned, students’ race was a moderator of the strength of the relationship between teachers’ emotions and perceptions of severe and patterned behavior. Specifically, the relationship between feelings of frustration and severe behavior was stronger for Black children than for White children, and the relationship between feelings of anger and patterned behavior showed the same pattern. Dr. Legette’s work highlighted a need for teachers to engage in reflective practices to unpack these biases.

Dr. Johari Harris, University of Virginia

In the same session, Dr. Johari Harris from the University of Virginia shared their work with the Children’s Defense Fund Freedom Schools. Learning for All (LFA), one Freedom School for students in grades 3-5, offers a five-week virtual summer literacy program with a culturally responsive curriculum based on developmental science. The program aims to create humanizing spaces that (re)define and (re)affirm Black students’ racial-ethnic identities, while also increasing students’ literacy skills, motivation, and engagement.

Dr. Harris’s mixed methods research found that students felt LFA promoted equity and inclusion, and reported greater participation, relevance, and enjoyment within LFA compared to in-person learning environments prior to COVID-19. They also felt their teachers were culturally engaging, and reported a greater sense of belonging, desire to learn, and enjoyment.

While it’s often assumed that young children of color are not fully aware of their racial-ethnic identity or how it is situated within a White supremacist society, Dr. Harris’s work demonstrated the importance of offering culturally affirming spaces to upper-elementary aged students.

Thursday, September 22, 2022

Dr. Krystal Thomas, SRI

Dr. Krystal Thomas presenting at SREE

On Thursday, I attended a talk by Dr. Krystal Thomas from SRI International about the potential of open education resource (OER) programming to further culturally responsive and sustaining practices (CRSP). Their team developed a rubric to analyze OER programming, including materials and professional development (PD) opportunities. The rubric combined principles of OER (free and open access to materials, student-generated knowledge) and CRSP (critical consciousness, student agency, student ownership, inclusive content, classroom culture, and high academic standards).

Findings suggest that while OER offers access to quality instructional materials, it does not necessarily develop teacher capacity to employ CRSP. The team also found that some OER developers charge for CRSP PD, which undermines a primary goal of OER (i.e., open access). One opportunity this talk provided was eventual access to a rubric to analyze critical consciousness in program materials and professional learning (Dr. Thomas said these materials will be posted on the SRI website in upcoming months). I believe this rubric may support equity-driven research and evaluation, including Empirical’s evaluation of the antiracist teacher residency program, CREATE (Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness).

Dr. Rekha Balu, Urban Institute; Dr. Sean Reardon, Stanford University; Dr. Beth Boulay, Abt Associates

left to right: Dr. Beth Boulay, Dr. Rekha Balu, Dr. Sean Reardon, and Titilola Harley on stage at SREE

The plenary talk, featuring discussants Dr. Rekha Balu, Dr. Sean Reardon, and Dr. Beth Boulay, offered suggestions for designing equity- and action-driven effectiveness studies. Dr. Balu urged the SREE community to undertake “projects of a lifetime”. These are long-haul initiatives that push for structural change in search of racial justice. Dr. Balu argued that we could move away from typical thinking about race as a “control variable”, towards thinking about race as an experience, a system, and a structure.

Dr. Balu noted the necessity of mixed methods and participant-driven approaches to serve this goal. Along these same lines, Dr. Reardon felt we need to consider system-level inputs (e.g., school funding) and system-level outputs (e.g., rate of high school graduation) in order to understand disparities in opportunity, rather than just focusing on individual-level factors (e.g., teacher effectiveness, student GPA, parent involvement) that distract from larger forces of inequity. Dr. Boulay noted the importance of causal evidence to persuade key gatekeepers to pursue equity initiatives and called for more high quality measures to serve that goal.

Friday, September 23, 2022

The tone of the conference on Friday was to call people in (a phrase used in opposition to “call people out”, which is often ego-driven, alienating, and counter-productive to motivating change).

Dr. Ivory Toldson, Howard University

Dr. Ivory Toldson at a podium presenting at SREE

In the morning, I attended the Keynote Session by Dr. Ivory Toldson from Howard University. What stuck with me from Dr. Toldson’s talk was their argument that we tend to use numbers as a proxy for people in statistical models, but to avoid some of the racism inherent in our profession as researchers, we must see numbers as people. Dr. Toldson urged the audience to use people to understand numbers, not numbers to understand people. In other words, by deriving a statistical outcome, we do not necessarily know more about the people we study. However, we are equipped with a conversation starter. For example, if Dr. Toldson hadn’t invited Black boys to voice their own experience of why they sometimes struggle in school, they may have never drawn a potential link between sleep deprivation and ADHD diagnosis: a huge departure from the traditional deficit narrative surrounding Black boys in school.

Dr. Toldson also challenged us to consider what our choice in the reference group means in real terms. When we use White students as the reference group, we normalize Whiteness and we normalize groups with the most power. This impacts not only the conclusions we draw, but also the larger framework in which we operate (i.e., White = standard, good, normal).

I also appreciated Dr. Toldson’s commentary on the need for “distributive trust” in schools. They questioned why the people furthest from the students (e.g., superintendents, principals) are given the most power to name best practices, rather than empowering teachers to do what they know works best and to report back. This thought led me to wonder, what can we do as researchers to lend power to teachers and students? Not in a performative way, but in a way that improves our research by honoring their beliefs and first-hand experiences; how can we engage them as knowledgeable partners who should be driving the narrative of effectiveness work?

Dr. Deborah Lindo, Dr. Karin Lange, Adam Smith, EF+Math Program; Jenny Bradbury, Digital Promise; Jeanette Franklin, New York City DOE

Later in the day, I attended a session about building research programs on a foundation of equity. Folks from EF+Math Program (Dr. Deborah Lindo, Dr. Karin Lange, and Dr. Adam Smith), Digital Promise (Jenny Bradbury), and the New York City DOE (Jeanette Franklin) introduced us to some ideas for implementing inclusive research, including a) fostering participant ownership of research initiatives; b) valuing participant expertise in research design; c) co-designing research in partnership with communities and participants; d) elevating participant voice, experiential data, and other non-traditional effectiveness data (e.g., “street data”); and e) putting relationships before research design and outcomes. As the panel noted, racism and inequity are products of design and can be redesigned. More equitable research practices can be one way of doing that.

Saturday, September 24, 2022

Dr. Andrew Jaciw, Empirical Education

Dr. Andrew Jaciw at a podium presenting at SREE

On Saturday, I sat in on a session that included a talk given by my colleague Dr. Andrew Jaciw. Instead of relaying my own interpretation of Andrew’s ideas and the values they bring to the SREE community, I’ll just note that he will summarize the ideas and insights from his talk and subsequent discussion in an upcoming blog. Keep your eyes open for that!

See you next year!

Dr. Chelsey Nardi and Dr. Leanne Doughty


Navigating the Tensions: How Could Equity-Relevant Research Also Be Agile, Open, and Scalable?

Our SEERNet partnership with Digital Promise is working to connect platform developers, researchers, and educators to find ways to conduct equity-relevant research using well-used digital learning platforms, and to simultaneously conduct research that is more agile, more open, and more directly applicable at scale. To do this researchers may have to rethink how they plan and undertake their research. We wrote a paper identifying five approaches that could better support this work.

  1. Reframe research designs to form smaller, agile cycles that test small changes each time.
  2. Researchers could shift from designing new educational resources to determining how well-used resources could be elaborated and refined to address equity issues.
  3. Researchers could utilize variables that capture student experiences to investigate equity when they cannot obtain student demographic/identify variables.
  4. Researchers could work in partnership with educators on equity problems that educators prioritize and want help in solving.
  5. Researchers could acknowledge that achieving equity is not only a technological or resource-design problem, but requires working at the classroom and systems levels too.

We hope that this paper (Navigating the Tensions: How Could Equity-Relevant Research Also Be Agile, Open, and Scalable?) will provide insights and ideas for researchers in the SEERNet community.

Read the paper here.


Evidentally is a finalist in the XPRIZE Digital Learning Challenge

The XPRIZE Digital Learning Challenge encourages applicants to develop innovative approaches to “modernize, accelerate, and improve effective learning tools, processes and outcomes” for all learners. The overarching goal of this type of research is to increase equity by identifying education products that work with different subgroups of students. Seeing the Institute of Education Sciences (IES) move in this direction provides hope for the future of education research and our students.

For those of you who have known us for the last 5-10 years, you may be aware that we’ve been working towards this future of low-cost, quick turnaround studies for quite some time. To be completely transparent, I had never even heard of XPRIZE before IES funded one of their competitions.

Given our excitement about this IES-funded competition, we knew we had to throw our Evidentally hat into the ring. Evidentally is the part of Empirical Education—formerly known as Evidence as a Service (EaaS)—that has been producing low-cost, quick turnaround research reports for edtech clients for the past 5 years.

Of the 33 teams who entered the XPRIZE competition, we are excited to announce that we are one of the 10 finalists. We look forward to seeing how this competition helps to pave the road to scalable education research.


AERA 2022 Annual Meeting

We really enjoyed being back in-person at AERA for our first time in a few years. We missed that interaction and collaboration that can truly only come from talking and engaging in person. The theme this year—Cultivating Equitable Education Systems for the 21st Century—is highly aligned with our company goals, and we value and appreciate the extraordinary work of so many researchers and practitioners who are dedicated to discovering equitable educational solutions.

We met some of the team from ICPSR, the Inter-university Consortium for Political and Social Research, and had a chance to learn about their guide to social science data preparation and archiving. We attended too many presentations to talk about so we’ll highlight a few below that stood out to us.

Thursday, April 21, 2022

On Thursday, Sze-Shun Lau and Jenna Zacamy presented the impacts of Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness (CREATE) on the continuous retention of teachers through their second year. The presentation was part of a roundtable discussion with Jacob Elmore, Dirk Richter, Eric Richter, Christin Lucksnat, and Stefanie Marshall.

It was a pleasure to hear about the work coming out of the University of Potsdam around examining the connections between extraversion levels of alternatively-certified teachers and their job satisfaction and student achievement, and about opportunities for early-career teachers at the University of Minnesota to be part of learning communities with whom they can openly discuss racialized matters in school settings and develop their racial consciousness. We also had the opportunity to engage in conversation with our fellow presenters about constructive supports for early-career teachers that place value on the experiences and motivating factors they bring to the table, and other commonalities in our work aiming to increase retention of teachers in diverse contexts.

Friday, April 22, 2022

Our favorite session we attended on Friday was a paper session titled Critical and Transformative Perspectives on Social and Emotional Learning. In this session, Dr. Dena Simmons, shared her paper titled Educator Freedom Dreams: Humanizing Social and Emotional Learning Through Racial Justice and talked about SEL as an approach to alleviate the stressors of systemic racism from a Critical Race Theory education perspective.

We tweeted about it from AERA.

Another interesting session from Friday was about the future of IES research. Jenna sat in on a small group discussion around the proposed future topic areas of IES competitions. We are most interested in if/how IES will implement the recommendation to have a “systematic, periodic, and transparent process for analyzing the state of the field and adding or removing topics as appropriate”.

Saturday, April 23, 2022

On Saturday morning, there was a symposium titled Revolutionary Love: The Role of Antiracism in Affirming the Literacies of Black and Latinx Elementary Youth. The speakers talked about the three tenants of providing thick, revolutionary love to students: believing, knowing, and doing.

speakers from the Revolutionary Love symposium

Saturday afternoon, in a presidential session titled Beyond Stopping Hate: Cultivating Safe, Equitable and Affirming Educational Spaces for Asian/Asian American Students, we heard CSU Assistant Professor Edward Curammeng give crucial advice to researchers: “We need to read outside our fields, we need to re-read what we think we’ve already read, and we need to engage Asian American voices in our research.”

After our weekend at AERA, we returned home refreshed and thinking about the importance of making sure students and teachers see themselves in their school contexts - Dr. Simmons provided a crucial reminder that remaining neutral and failing to integrate the sociopolitical contexts of educational issues only furthers erasure. As our evaluation of CREATE continues, we plan to incorporate some of the great feedback we received at our roundtable session, including further exploring the motivation that led our study participants to enter the teaching profession, and how their work with CREATE adds fuel to those motivations.

Did you attend the annual AERA meeting this year? Tell us about your favorite session or something that got you thinking.


McGraw Hill Education ALEKS Study Published

We worked with McGraw Hill in 2021 to evaluate the effect of ALEKS, an adaptive program for math and science, in California and Arizona. These School Impact reports, like all of our reports, were designed to meet The Every Student Succeeds Act (ESSA) evidence standards.

During this process of working with McGraw Hill, we found evidence that the implementation of ALEKS in Arizona school districts during the 2018-2019 school year had a positive effect on the AzMERIT End of Course Algebra I and Algebra II assessments, especially for students from historically disadvantaged populations. This School Impact report—meeting ESSA evidence tier 3: Promising Evidence—identifies the school-level effects of active ALEKS usage on achievement compared to similar AZ schools not using ALEKS.

Please visit our McGraw Hill webpage to read the ALEKS Arizona School Impact report.

What is ESSA?

For more information on ESSA in education and how Empirical Education incorporates the ESSA standards into our work, check out our ESSA page.


Presenting CREATE at AERA in April 2022

Attending AERA 2022

We’re finally returning to in-person conferences after the COVID-related hiatus, and we will be presenting at the annual meeting of the American Educational Research Association (AERA). This year, the AERA meeting will be held in San Diego, our CEO Denis Newman’s new home turf since relocating to Encinitas from Palo Alto, CA.

Sze-Shun Lau and Jenna Zacamy will be attending AERA in person, joined by Andrew Jaciw virtually, to present impacts of Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness (CREATE) on the continuous retention of teachers through their second year.

  • When: Thursday, April 21, from 2:30 to 4:00pm PDT
  • Where: San Diego Convention Center, Exhibit Hall B
  • AERA Roundtable session: Retaining Teachers for Diverse Contexts
  • AERA Presentation: Impacts of “Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness” on the Continuous Retention of Teachers Through Their Second Year

Collaboration and Reflection to Enhance Atlanta Teacher Effectiveness (CREATE)

The work that is the basis of our AERA presentation examines the impact of CREATE—a teacher induction program—on graduation and subsequent retention of teachers through their first two years. The matched comparison group design involved 121 teachers across two cohorts. Positive impacts on retention rates were observed among Black educators only.

Retention rates after two years of teaching were 71% for non-Black educators in both CREATE and comparison groups. For Black educators the rates were 96% and 63% in CREATE and comparison, respectively. Positive impacts on mediators among Black educators, including stress-management and self-efficacy in teaching, provide a preliminary explanation of effects.

We have been exploring potential mechanisms for these impacts by posing open-ended survey questions to teachers about teacher retention. Based on their own conversations, experiences, and observations, early career teachers have cited rigid teaching standards, heavy and mentally taxing workloads, a lack of support from administration, and the low pay as common reasons teachers in their first three years of teaching leave the profession.

Factors that these teachers see as effective in retaining early-career teachers include recognition of the importance of representation in the classroom and motivation to work towards building less oppressive systems for their students. For early career teachers participating in CREATE, the access to professional learning around communication skills, changing one’s mindset, and addressing inequities are credited as potential drivers of higher retention rates.

We look forward to presenting these and other themes that have emerged from the responses these teachers provided.

We would be delighted to see you in San Diego if you’re planning to attend AERA. Let us know if we can schedule a time to meet up.

Photo by Lucas Davies