Home » Services » Research & Evaluation Services » Program Evaluation Services

Program Evaluation Services

Billions of dollars. That’s what school districts and state departments of education spend on educational programs. Education stakeholders require rigorous evidence that programs are effective—grant money, tax money, student and teacher time, and many other resources are all at stake. Whether your program is straightforward or complex, we customize our evaluation of educational programs to meet your specific needs. If you are planning for a grant that requires an independent education program evaluation, we are happy to partner with you to prepare an effective grant application, then assist you in ensuring all of the elements of the evaluation are in place from the start of your program. We develop measurement metrics for program implementation and outcome variables. We understand the quantitative and qualitative methods needed for an accurate and valid evaluation in the education industry. Our analysis yields insights into how to improve the program, whether it is effective as planned, and whether there are unintended consequences. Our services follow a prescribed proven methodology:

  • Frame program evaluation questions
  • Determine indicators that answer the program evaluation questions
  • Determine strategies to measure those indicators
  • Prepare for and gather the implementation indicators
    • Prepare new or modify existing instruments
    • Perform site visits or observations
    • Perform focus group interviews
    • Develop and administer surveys
    • Collect data and format for analysis
    • Analyze implementation data
  • Prepare for and gather the outcome indicators
    • Gather existing student achievement data (from state, district, or school tests)
    • Develop new or modify existing tests
    • Implement assessments
    • Process and score student assessments
    • Prepare and format data for analysis
    • Analyze student achievement data
  • Using statistics and psychometric measurement methodologies, establish the relationship between program implementation and student outcomes
  • Prepare findings, make recommendations, and write reports

Which schools or districts have fully implemented the program, and which have not? Is your program effective at all? If it is, how effective, and is it significant? Which schools show true growth in student achievement? Are those students at the schools that fully implemented the program? What is the relationship between implementation and achievement? These are all questions that we can answer by implementing a valid evaluation of your educational programs. Please contact us to see how we can help you design and implement an effective educational program evaluation.

Educational Data Systems developed and implemented the State of California’s Statewide Reading First program evaluation. See below to read a brief overview of that evaluation project.


Reading First Statewide Program Evaluation

From 2003 through 2010, Educational Data Systems was the prime contractor to the State of California to implement an evaluation of the State’s Reading First program. The program evaluation included producing three different paper and online surveys for Reading First teachers, reading coaches, and school principals at participating schools. To facilitate this, we:

  • Prepared the survey questions
  • Equated the three surveys
  • Administered the surveys statewide to four different cohorts of participating schools
  • Analyzed the data to produce a custom Reading First school “implementation index,” a measure of how well the school implemented the program
  • Collected student assessment data from the State’s assessment systems and, to preserve student privacy, suppressed the student names and IDs
  • Secured Reading First End-of-Year student data directly from participating school districts
  • Collected data from vendors through secure websites or secure FTP sites
  • Collected publicly available data sets from existing sources such as STAR, CBEDS, and CELDT statewide assessments

We also created a custom Reading First “achievement index,” a measure of the growth in achievement of each participating school. Using a psychometric procedure, we then combined the implementation and achievement indexes to indicate the effectiveness of the Reading First program.

Educational Data Systems was responsible for the Reading First Evaluation Report for each of the seven years of California’s participation in the Reading First program, as required by federal NCLB requirements and CDE specifications.

If you’d like to learn more about our Reading First program evaluation and methodologies we used, read one of our evaluation reports or call, email, or visit us at your convenience for more details.