Test scores don't add up

"UBSCT data is rich and full. But it's also confusing . . . because there are so many data points," said John Jesse, Alpine District's research and evaluation director. "It's tough even for us as a district to sort it out. . . . I don't know how you produce accurate data statewide."

The newspaper requested more state analyses of the numbers.

The State Office of Education denied the request, saying test data will never be perfect on the basic skills test until universal student IDs are created. Park says test-takers might have moved, changed their ethnicity designation from one enrollment to the next and provided limited information on answer sheets, further complicating the state office's efforts to track them.

"This information can only be used for aggregate purposes. Efforts to use this information to estimate graduation rates, dropout rates, or specific number of students who still need to pass the test to receive a basic diploma is inappropriate, based on the current methods of data collection and reporting," according to a statement issued by the State Office of Education and attributed to Park.

"Efforts to compare district data to state data is not possible because students who move may be counted in the previous and current district, and districts may configure their data with both the class of 2006 and the class of 2007, and may use different parameters for this configuration."

Complexities are many

Some school district testing chiefs have questioned state numbers. But they also offer their understanding.

They say the state education office is under tremendous pressure to churn out results of Utah's only high-stakes exam, and fast. Districts and schools must know how their students did so they can give them help over the summer and the following school year. Some also are training more teachers to tutor based on how many need help, and specifically, what kind of help.

"I don't think any of the data coming out was deliberately (reported) to misconstrue any facts," said Darrell Thomas, Granite District's research, evaluation and assessment director. "Knowing the difficulty and complexities of dealing with kids moving in and out, and these crazy numbers when we have three test (administrations) — heaven help us when we have five — . . . there will be some glitches along the way. I just don't think people understand the magnitude of databases that go into the millions of rows and columns." Indeed, State Superintendent of Public Instruction Patti Harrington says basic skills test reports were never intended to be absolute. Officials also stressed to legislators that participation- and enrollment-based data would not add up because they're answering two different questions.