blog posts and news stories

New Research Project Evaluating the Impact of EVERFI’s WORD Force Program on Early Literacy Skills

Empirical Education and EVERFI from Blackbaud are excited to announce a new partnership. Researchers at Empirical will evaluate the impact and implementation of the WORD Force program, a literacy adventure for K-2 students.

The WORD Force program is designed to be engaging and interactive, using games and real-world scenarios to to teach students key reading and literacy skills and understand how to use them in context. It also provides students with personalized feedback and support, allowing them to work at their own pace and track their progress.

We will conduct the experiment within up to four school districts—working with elementary school teachers. This is our second project with EVERFI, and it builds on our 20 years of extensive experience conducting large-scale, rigorous randomized controlled trial (RCT) studies. (Read EVERFI’s press release about our first project with them.)

In our current work together, we plan to answer these five research questions. 1. What is the impact of WORD Force on early literacy achievement, including on spoken language, phonological awareness, phonics, word building, vocabulary, reading fluency, and reading comprehension, for students in grades K–2? 2. What is the impact of WORD Force on improving early literacy achievement for students in grades K-2 from low- to middle-income households, English Language Learner (ELL) students, by grade, and depending on teacher background (e.g., years of teaching experience, or responses to baseline survey about orientation to literacy instruction)? 3. What is the impact of WORD Force on improving early literacy achievement for students in grades K-2 who struggle with reading (i.e., those in greatest need of reading intervention) as determined through a baseline assessment of literacy skills? 4. What are realized levels of implementation/usage by teachers and students, and are they associated with achievement outcomes? 5. Do impacts on intermediate instructional/implementation outcomes mediate impacts on achievement ?

Using a matched-pairs design, we will pair teachers who are similar in terms of years of experience and other characteristics. Then, from each pair, we will randomize one teacher to the WORD Force group and the other to the business-as-usual (BAU) control group. This RCT design will allow us to evaluate the causal impact of WORD Force on student achievement outcomes as contrasted with BAU. EVERFI will offer WORD Force to the teachers in BAU as soon as the experiment is over. EVERFI will be able to use these findings to identify implementation factors that influence student outcomes, such as the classroom literacy environment, literacy block arrangements, and teachers’ characteristics. This study will also contribute to the growing corpus of literature around the efficacy of educational technology usage in early elementary classrooms.

For more information on our evaluation services, please visit our research services page and/or contact Robin Means.

All research Empirical Education has conducted for EVERFI can be found on our EVERFI webpage.

Studying the Impacts of CAPIT Reading: An Early Literacy Program in Oklahoma

Empirical Education’s Evidentally recently conducted a study to evaluate the impact of CAPIT Reading on student early literacy achievement. The study utilized a quasi-experimental comparison group design using data from 12 elementary schools in a suburban school district in Oklahoma during the 2019–20 school year.

CAPIT Reading is a comprehensive PK–2 literacy solution that includes a digital phonics curriculum and teacher professional development. The program is a teacher-led phonemic awareness and phonics curriculum that includes lesson plans, built-in assessments, and ongoing support.

Four schools used CAPIT to supplement their literacy instruction for kindergarten students (treatment group) while eight schools did not (comparison group). The study linked CAPIT usage data and district demographic and achievement data to estimate the impact of CAPIT on the Letter Word Sounds Fluency (LWSF) and Early Literacy Composite scores of the aimsweb reading assessment, administered by the district in August and January.

We found a positive impact of CAPIT Reading on student early reading achievement on the aimsweb assessment for kindergarten students. This positive impact was estimated at 4.4 test score points for the aimsweb Early Literacy Composite score (effect size = 0.17; p = 0.01) and 7.8 points for the LWSF score (effect size = 0.29; p < 0.001). This impact on the LWSF score is equivalent to a 29% increase in growth for the average CAPIT student from the fall to winter tests.

We found limited evidence of differential impact favoring student subgroups, meaning that this positive impact for CAPIT users did not vary according to student characteristics such as eligibility for free and reduced-price lunch, race, or gender. We found that the impact on aimsweb overall was marginally greater for special education students by 4.9 points (p = 0.09) and that the impact on LWSF scores was greater for English Language Learners by 7.4 points (p = 0.09). Impact of CAPIT reading does not vary significantly across other student groups.

Read the CAPIT Reading Student Impact Report for more information on this early literacy research.