Title of Study: Effectiveness of Reading and Mathematics Software Products:  Findings fromTwo Student Cohorts

Author(s): Larissa Campuzano, Mark Dynarski, Roberto Agodini, Kristina Rall, Audrey Pendleton

Summary:
This report presents findings of the second year of a two-year study on the effects of reading and mathematics software products on student test scores.  Ten products were tested the second year.  The products were Destination Reading, the Waterford Early Reading Program, Headsprout and Plato Focus for first grade reading; LeapTrack and Academy of Reading for fourth grade reading; Larson Pre-Algebra and Achieve Now for sixth grade math/ Cognitive Tutor Algebra I and Larson Algebra I for students taking algebra in middle or high school.  The objectives of the second year of the study were: 1) to assess whether a second year’s experience with a software product increased the effects products have on student test scores and 2) to assess the effectiveness of individual products in raising student test scores.  

Sponsoring entity(s): U.S. Department of Education

Date conducted: 2009

Location of the study: This study can be accessed through the Department of Education, Institute of Educational Sciences website.  

Setting(s) addressed:

• Classroom

Targeted population(s):

• First and fourth grade reading students
• Sixth grade pre-algebra students
• Middle and high school algebra students

Primary sources of evidence used in the study or report:

• Test scores
• Questionnaires

Primary Technology Application(s) Addressed:

• Educational software

Major education topic(s) addressed:

• Reading
• Math

Major findings/conclusions:

• In the first year of the study, the reading software had no effect on first grade or fourth grade reading scores.
• Overall, results of the second year of the study were not statistically different from the first year.
• LeapTrack was the only reading product to have a statistically significant positive effect on reading scores.
• Effects of the software on sixth grade math scores were slightly negative in the first year, although the effect was not statistically significant, and even more negative in the second year.
• Effects of the software on algebra scores were slightly negative the first year. Scores of students using the software in the second year were higher than in the first year, but not significantly higher than controls.
• None of the math software had a statistically significant positive effect on math scores.

Practical implications of the findings:

• Teachers surveyed the first year of the study felt they had been adequately trained with the software, so the lack of effect was not attributable to teacher unfamiliarity with the products tested.
• Student usage data indicated that the amount of time students spent with the software had no effect on test scores.

Reviewer Comments:
There were changes in experimental protocols from the first to second years of this study. The way students were tested varied among schools. Teachers volunteered for the study, and samples of teachers and students differ from year to year.  These factors may have had some small effect on results.