Computers to grade written portions of STAAR tests, parents and faculty question AI reliability

STAAR tests are underway in schools across North Texas but there are some questions about how the tests are being graded

NBC Universal, Inc.

As North Texas students continue with STAAR testing, there is a major change to the way part of the test will be scored. A computer program and technology will score written answers.

According to the Texas Tribune, the new grading program is similar to the popular artificial intelligence program Chat GPT.

“The STAAR test was redesigned recently with a new version in 2023 that has six to seven times more open-ended response questions than in previous years,” Texas Tribune reporter Keaton Peters said in a recent interview with KXAN-TV. “That takes a lot more time and also a lot more money to score all of those, so I think that's the main reason the TEA does expect to save anywhere from $15 million to $20 million this year by rolling out the automatic system.”

Some question the reliability of the computer scores.

“When it scores a test, it gives a confidence rating. Basically, based on its programming, if it has a low confidence in the score that it gave… that response is automatically routed to a human to double-check,” Peters said. “If the response has words the computer doesn't recognize, say slang or words in a language other than English, then it sends those out to a real person as well.”

Peters also said there’s a $50 fee parents can pay to have something rescored by a person. If the score is returned higher than the original score, the fee will be waived.

NBC 5 contacted Fort Worth ISD for comment about the scoring changes. A district representative provided the following comment.

“With the stakes so high for our campuses, we have a natural sense of uneasiness about a computer’s ability to score responses as well as a human can, even if the automated scoring engine works as technically intended. We have encouraged and supported our students to meaningfully engage in reading and express themselves with their unique voice and originality in their writing. Technology may not fully capture or reward this as a human rater would and penalize our students, especially our students whose first language is not English who may express themselves using second language acquisition writing conventions that only a human may be able to capture.  We just want our students to receive fair credit for their original thoughts and ideas and the unique way they express them in their writing responses. We understand the need for TEA to adapt and seek system efficiency and will continue to best align our instructional practices to best meet the needs of students, knowing how they will ultimately be expected to demonstrate content mastery.”

Contact Us