Oklahoma's track record with testing companies responsible for the state's high-stakes exams has been marred with errors and delays.
During the past decade, a combination of dissatisfaction and competitive costs has prompted the state to use five testing companies.
Every spring students in grades 3 to 12 sit take Oklahoma's standardized tests in a variety of subjects. Companies that have multimillion-dollar contracts with the state are responsible for developing, administering and scoring the exams. They also must analyze the results.
State schools Superintendent Janet Barresi announced last week that errors were made by the testing company Pearson Education Inc. when calculating school and district accountability under the federal No Child Left Behind Act.
Barresi requested a review of the $16.7 million contract with Pearson. That contract is for Pearson to develop, administer and score the state's third-grade through eighth-grade exams.
Barresi said the contract could be terminated.
Almost exactly a decade ago, Oklahoma fired the testing company Riverside Publishing after test scores for grades 3 to 8 were delivered late. The company also was fined $1,800 for each day test scores were late.
It was noted then that no testing company had an unblemished record. The same holds true today.
“Every company in the testing industry has had errors,” said Bob Shaefer, director of FairTest, a think tank that opposes the high-stakes testing mandated in 2001 with the No Child Left Behind Act.
“What we see across the nation is a perverted game of musical chairs, in which a test company will get fired in one state for messing up and hired in another state.”
Shaefer said the disconcerting thing is that these exams are increasingly being used to make huge decisions.
In just a few years, Oklahoma teachers and principals will be evaluated in part based on student performance on state exams, schools will be graded on an A to F grading scale, high school seniors will be prevented from graduating if they can't pass their end-of-instruction exams and third-grade students will be retained if they fail the reading exam.
Barresi said the high stakes associated with these exams are exactly why she's not accepting anything but the highest quality results from Oklahoma's testing companies.
Oklahoma has three testing contracts. Up until a few months ago, Pearson held all three. The contracts are for grades 3 to 8 criterion reference tests, both the regular exams and special exams for students on individualized education plans; for grades 9 to 12 end-of-instruction exams for both regular and special students; and for the portfolio assessment of the state's most cognitively disabled students.
The multimillion-dollar contracts are awarded in an official public bidding process through the Department of Central Services.
Spring 2011 was the first time Pearson administered the grades 3 to 8 exams, after outbidding the previous company, Data Recognition, in 2010 to take over administration of the tests.
However, Pearson has handled Oklahoma's high school level exams — commonly known as end-of-instruction exams — since 2007.
The $5.2 million contract for Oklahoma's high school exams expired this year.
Three companies submitted bids for the end-of-instruction testing contract, and the state Education Board voted in July to award the contract to Pearson.
But no contract has been awarded yet.
Barresi said the evaluation of those bids is ongoing and historical performance of companies will be taken into consideration.
According to officials with the state Education Department, the transition between testing companies can be a timely and difficult process.
While Oklahoma owns the rights to all testing materials created for the uniquely Oklahoma exams, it does not own propriety software used to create those exams.
It takes time and detailed work from psychometricians (experts in psychological measurement) to re-create the exams made specifically for Oklahoma based on a number of state standards.
If Oklahoma were to award the 2012 end-of-instruction testing contract today, the winner would have until early November to prepare the exams. That's less than three months, if a company other than Pearson wins, to re-create the state's end-of-instruction exams that are the basis for whether high school seniors can graduate.
In Oklahoma the test results for individual students are accurate.
Most of those results have been sent home to parents.
The errors and delays have come in the analysis of that data in conjunction with school demographics — a process that determines how schools perform on the federal Academic Performance Index.
That analysis is used to determine whether schools and districts are on the list of schools in need of improvement, an infamous designation that can come with federal sanctions.
Before it was announced that there were errors in those calculations, Barresi reached a $1.2 million settlement with Pearson for other delays and errors in testing results.
In 2010, Pearson paid Florida $14.7 million in fines for delivering the state's standardized test scores late and paid Wyoming $5.1 million for online testing issues, when the computer program froze during the testing process.
In 2004, Harcourt Assessments Inc. printed the incorrect answers to sample questions on state tests for Oklahoma eighth-graders, eliciting an apology from the company.
In 2001, Riverside Publishing was fired in Oklahoma for significant delays in student test results, and in 1997 Harcourt Publishing sent the wrong writing exams to 80,000 Oklahoma students in eighth and 11th grades.
Despite the error, which forced students to retake the writing exam, the state renewed the approximately $8 million contract with Harcourt.