Student test score gains in Texas during the last decade have been inflated, raising questions about the validity of academic progress in a state touted as a model of reform, according to a report released Tuesday by the Rand Corp.
In his quest for the White House, Texas Gov. George W. Bush regularly cites the rising test scores of white and minority students in his home state--what some call the "Texas miracle."
Bush campaign officials immediately attacked the Rand report as shoddy and "dead wrong," while calling its timing suspect. The release of the study comes just two weeks before the Nov. 7 election.
On ABC's Nightline on Tuesday night, Bush said, "I find it interesting that kind of a renegade report comes floating out two weeks to go in the campaign . . . and I reject their findings."
"The Rand controversy has no consequence in this election," he said, "because it's a report that's got no weight. I mean it's somebody's opinion, and when people look at the facts, they'll realize that it's just background noise."
Rand officials said the timing was coincidental and defended the integrity of the report.
"The soaring test scores in Texas do not reflect real improvement in students' ability to read and do math," said Stephen Klein, the lead analyst from Rand, a nonpartisan research organization that funded the report. "Texas is doing better than the rest of the country in some areas, but nowhere near the miracle. It's a myth."
The Rand report has important implications for California because the state has followed Texas' lead in basing its school accountability system heavily on a single test.
But on Tuesday, most reaction to the study focused on politics.
Bush communications director Karen Hughes pointed out that another Rand study, released in July, concluded that Texas was among the most effective states at helping students at all income levels excel.
The author of the July report also was critical of the new study, but Rand said both studies were valid.
Campaign officials for Vice President Al Gore, meanwhile, seized on the Rand report to deride the Texas governor's record.
"Bush's claim to have improved the state of education in Texas has been called into serious question," said Ron Klain, a senior advisor to Gore and his running mate, Sen. Joseph I. Lieberman. "The same independent group Bush once pointed to as proof that he has done an effective job on education as governor of Texas has now devastated his claims."
Tuesday's report compared the progress of Texas fourth- and eighth-graders on the state's own standardized tests with their performance on federally sponsored exams.
The state test is called the Texas Assessment of Academic Skills and the federal exam is known as the National Assessment of Educational Progress. Both covered reading and math. Researchers reviewed scores from the tests between 1992 and 1998.
The analysts found that students enjoyed far larger gains on Texas' exams than they did on the national tests. The scores also showed that African Americans and Latinos posted greater gains than whites on the state tests. And the historic gap in test scores between whites and minorities narrowed.
On the national test, however, whites, blacks and Latinos in Texas made comparable gains and the achievement gap grew slightly.
The findings led the analysts to suggest that the widely ballyhooed gains on Texas' own tests may be the result of extensive test preparation, a low standard for passing and some cheating prompted by pressure that the system puts on teachers and administrators.
The study noted that Texas uses new versions of the test every year. Questions on the tests are released after the exams are given each year, and teachers are known to use the materials for extensive test preparation. Schools with large percentages of minority and low-income students may be doing more test prep than other campuses, the experts said.
The researchers also said that the shrinking achievement gap on the state test may have arisen from higher-performing white students "topping out" on an exam that is too easy, thus artificially narrowing the gap between them and minorities.
Klein and his fellow researchers said all these issues spelled trouble.
"It raises very real questions about whether you can trust the gains. . . ," Klein said. "I think the gains are misleading and inflated."
Tuesday's report contrasted with the Rand report in July. That study, based exclusively on scores from the National Assessment of Educational Progress, painted a far more complimentary picture of achievement in Texas.
It found that the state, along with North Carolina and a handful of other states, posted gains at twice the national average rate between 1990 and 1996. When scores were adjusted to factor out the effects of students' backgrounds, Texas also came out on top.
That study of state-by-state test data also said that more resources--such as preschool education and smaller classes--made a bigger difference in Texas and elsewhere than tough accountability measures such as those Bush pushed in his home state and now supports in his race for the presidency.
Klein, the lead researcher on Tuesday's report, said the two studies are not comparable because they reviewed different periods and diverged sharply in scope.
But just as Bush and Gore clashed over the meaning of Tuesday's report, so did the Rand researchers.
David Grissmer, the lead analyst on the July study, criticized his colleagues on Tuesday. He said that comparing the state and national tests in Texas may be problematic because the two are designed with different purposes in mind.
He also said that his colleagues should have reviewed test data going back to 1990. Had they done so, he said, they would have found a smaller difference between the state and national test results.
"There may be nothing amiss about the [Texas] test," Grissmer said. "They may just be different tests designed to test different things."
Klein declined to comment on Grissmer's work but expressed confidence in his own. Grissmer's study was more in-depth and was published as a book.
Top Rand officials defended the quality of both studies, saying they each went through rigorous reviews by experts at the organization and elsewhere. The studies arrived at different but equally valuable conclusions, they said.
Rand officials also defended the timing of Tuesday's report, saying its release shortly before the general election was only coincidental. One Rand official pointed out that the July study, with its glowing appraisal of Texas, came out one week before the Republican National Convention.
"We don't produce findings for political reasons, we don't distribute them for political reasons and we don't sit on them for political reasons," Rand President James A. Thomson said. "This is a scrupulously nonpartisan institution."