Title of Study:  An Open Source Platform for Internet-based Assessment: A Report on Education Leaders’ Perceptions of Online Testing in an Open Source Environment

Author(s): Grunwald Associates

Summary:
This report summarizes interviews with more than 80 individuals representing 27 states and public and private organizations.  The vast majority believes that online testing will become the norm for state assessments over the next few years. The study found many educators do not completely understand the nature of open source resources, but that there is strong interest in learning more about how an open source platform would operate for state accountability testing.  Case studies from states using online assessment discuss lessons learned during implementation of online assessment systems.

Sponsoring entity(s): Educational Testing Service

Date conducted: 2010

Location of the study: This study can be accessed through the Grunwald associates website.

Setting(s) addressed:

• State level

Targeted population(s):

• K-12

Primary sources of evidence used in the study or report:

• Telephone interviews with state assessment and technology leaders and leaders of public and private educational organizations

Primary Technology Application(s) Addressed:

• Online assessment

Major education topic(s) addressed:

• High stakes testing
• Formative assessment

Major findings/conclusions:

• 23 of the 27 states surveyed reported they were already pilot testing or offering online testing in some form or would be moving their tests to the internet within the next five years.
• States found that student test performance is comparable between online and paper tests, but that students and educators prefer the online tests.
• Although most respondents expressed interest in using an open source platform for assessment, many admitted they did not understand open source.

Practical implications of the findings:

• Respondents felt online testing improves the logistics of state testing
• Respondents felt online testing provides better data with fewer administrative errors
• Respondents felt online testing gives faster turnaround of data
• Respondents felt open source resources provided cost savings, but were concerned about hidden costs and potential security issues.
• Respondents felt open source platforms have the advantage of adaptability, and felt they might lead to sharing among states if common standards are adopted.
• Problems with online testing include infrastructure issues such as the need for sufficient and secure bandwidth and an adequate number of computers.
• Problems also include the need for technological expertise at the district level to support the testing
• Constructed response questions pose challenges to scaling up online assessments
• Testing schedules may have to be staggered to accommodate the internet traffic

Reviewer Comments:
The findings are based on interviews of assessment and technology leaders in 27 states and nationally known educational leaders. The report gives a good summary of perceived advantages and disadvantages of open source platforms for formative and summative assessment, with examples of case studies in early adopting states.  The report summarizes strategies that might be used in pursuing an open source platform to be used for assessments across state boundaries.