Reading First Statewide Program Evaluation
From 2003–04 through 2009–10, we were the prime contractor to the California Department of Education (CDE) to implement an evaluation of the State’s Reading First program. This statewide program evaluation included producing three different paper and online surveys for teachers, reading coaches, and principals of participating Reading First program schools. We prepared the survey questions, equated the surveys, administered the surveys statewide to four different cohorts of participating schools, and analyzed the data to produce a custom Reading First school “implementation index”, a measure of how well participating schools implemented the Reading First program. We collected student assessment data from the State’s assessment system, and to preserve student privacy, the student names and IDs were suppressed. Educational Data Systems collected Reading First End-of-Year student data directly from Reading First participating school districts through a secure download website; collected data from data vendors through secure website or secure FTP sites, and collected assessment and publicly available data sets from existing sources such as STAR, CBEDS, and CELDT statewide assessments. We created a custom Reading First “achievement index,” a measure of the growth in achievement of each participating school. We then used a psychometric procedure to combine the implementation and achievement indexes to indicate the effectiveness of the Reading First program. We analyzed the data and compiled the annual Reading First Evaluation Report as required by federal NCLB requirements and CDE specifications.