I think everyone, based on our comments this morning, we all feel comfortable with the test. We don’t feel like it’s skewed, that there’s a political agenda. We feel like it’s safe, that the things we flagged with comments are being looked at. —Review committee member Alean Hunt
SALT LAKE CITY — Since Monday, Alyson Williams and 14 other parent volunteers have spent long hours at the State Office of Education, carefully combing through roughly 10,000 exam questions that will be used this spring in new computer adaptive tests.
"I have four kids to shuttle around to different baby sitters before I get up," Williams, of Spanish Fork, said. "It’s been a marathon from early, early in the morning to late at night."
That marathon ended Friday as the review committee participated in a debrief session with state education officials and policymakers. While roughly 3 percent of exam questions were flagged for review — based on a range of technical or content-related concerns — members of the committee were positive in their feedback to state officials.
"I came in with fairly low expectations about what the process would be and whether or not we’d be able to get through the number of questions that are in the pool," Williams said. "I'm really impressed. I am satisfied that they are taking our feedback seriously."
John Jesse, director of assessments for the State Office of Education, spoke highly of the review committee efforts. He said every flagged question will be reworked or discarded, but, overall, all 15 members of the committee suggested that the test is in good shape to continue.
"These parents worked so dang hard, they really took their job seriously," he said. "I don’t think we could have hoped for a better process or a better result."
The new test, which uniquely adjusts in difficulty based on a student's correct and incorrect responses, is slated to replace Criterion-Referenced tests this spring as Utah's end-of-level assessments in science, math and English language arts.
The tests have courted controversy, both due to their use in conjunction with new Common Core State Standards and because of their adaptive nature that sees no two students taking the same assessment. The adaptive questioning also makes it effectively impossible for a parent to review the exact test materials their students will be presented.
The Utah State Legislature mandated the creation of a review committee out of concerns that the computer adaptive tests could include biases that are objectionable to Utah families. For the same reason, education officials were directed to contract with a test provider to create a unique Utah test rather than using the national adaptive tests designed for the 46 states that have adopted the Common Core.
"I didn’t see any real social problems with the test or things that would be controversial," said review committee member Alean Hunt. "I think everyone, based on our comments this morning, we all feel comfortable with the test. We don’t feel like it’s skewed, that there’s a political agenda. We feel like it’s safe, that the things we flagged with comments are being looked at."
Hunt said she went into review process with concerns that Utah students are tested too much. But she said she found the adaptive test questions to be engaging and the review process highly informative.
"I feel like this test is a great formative test and I’m very comfortable with it," she said.
Williams said she appreciates that Utah has retained local control of its testing materials. She said she has been vocal about her criticisms regarding testing reform and parents are naturally skeptical when students are presented with new assessments.
"People want to know, is it going to be good what my children are seeing or not?" she said.
Williams said that she hopes parents recognize that even with the vetting of the review committee, the tests administered this spring will be a first-year pilot for the state. She said this first review went as well as could be hoped, but she added that she is not yet fully comfortable with her children taking the new test.
"I haven’t decided, but I have a lot more information about which to make that decision," she said.
Parents wishing to review the new test materials will be able to access a series of practice tests that will be posted to the State Office of Education website in December, Jesse said. The practice tests will not include the exact questions students will encounter in April but will be made up of similarly designed questions that are openly accessible to the public.
Email: [email protected], Twitter: bjaminwood