Increasing the Reach of Promising Dropout Prevention Programs: Examining the Trade-offs Between Scale and Effectiveness

The University of Chicago Education Lab and Crime Lab New York research teams are carrying out a randomized controlled trial during the 2016-17 and 2017-18 academic years to build on previous collaborations with the Chicago Public Schools (CPS), the New York City Department of Education, and SAGA Innovations that have found that SAGA's intensive, individualized, during-the-school-day tutoring can generate very large gains in academic outcomes in a short period of time, even among students who are many years behind grade level. This research suggests the promise of this approach for improving the academic skills and educational attainment of disadvantaged youth, even once they have reached adolescence. However, to truly affect outcomes at the local and national level, SAGA would have to be rolled out on a much greater scale than researchers have been able to study in Chicago. Yet little is known about how to take promising interventions to scale. This study seeks to build the science of scale-up, by examining the extent to which this individualized tutoring program can be implemented at an even greater scale and by explicitly exploring the trade-offs between effectiveness and scale.

The SAGA Innovations program expands on the nationally recognized innovation of high- dosage, in-school-day tutoring developed in Match Education's charter school in Boston. The tutoring program meets as a scheduled course, Math Lab, once a day during the normal school day, and is provided in addition to a student's regular math class. Students work two-on-one (two students with one tutor) with the same full-time, professional tutor for the entirety of the school year. The content of the tutoring sessions is aligned with what students are learning in their regular math courses, but is also targeted to address individual gaps in math knowledge. Also following the original model developed by Match Education, SAGA tutors use frequent internal formative assessments of student progress to individualize instruction.

A previous randomized controlled trial conducted by the University of Chicago research team found that one year of this intervention, delivered in AY2013-14 in the Chicago Public Schools, generated between one and two extra years of academic growth in math, over and above what the normal U.S. high school student learns in one year (Cook et al., 2015; Reardon, 2011). The estimated effects for math achievement are on the order of 0.19 to 0.30 standard deviations, depending on the exact test and norming used. The intervention also improved student grades in math by 0.58 points on a 1-4 grade point scale, compared to a control mean of 1.77. These gains are particularly important because of the growing evidence on the importance of math specifically for short- and medium-term success in school, and for long-term life outcomes such as employment and earnings (Duncan et al., 2007).

This study aims to build upon the investigators' previous evaluations of the program, and will provide insight into the ability of this program to serve youth at a much larger scale. Specifically, this study aims to answer the following research questions:

1. What is the effect of implementing an evidence-based individualized tutoring program at larger scale?
2. What is the relationship between the effect of the program and the scale at which the program is implemented?