With so much recent scandal about test security on standardized exams like the SAT and the ACT, particularly ones involving international administrations of the exams, it’s no wonder that companies like ACT and the College Board are looking for ways to improve.
This summer, in hopes of increasing international test security, ACT announced it will require all international test takers to take the ACT online using a computerized based exam. Though the move does not affect most paper testing offerings in the US at this time, it does signal a move to computer based testing that many other standardized tests, including the GRE the TOEFL, have already taken.
But it’s the rollout, according to student reports, that did not go as smoothly as planned.
At the worse end of the spectrum, reports on social media have surfaced of multiple international testing centers (from South America to Europe) cancelling exams just hours before their administration. According to a release at ACT.org, over 39 test centers in 31 countries were cancelled. Students have reported that the ACT has not offered a replacement exam to many who were counting on these exam dates. These students invested significant money and time into this preparation process and now may not have the opportunity to obtain a score before early application deadlines.
On social media, one student relayed the message to me that according to her sources, delays in her area were due to a lack of “computer readiness” certification. While it is true that some test centers likely suffered issues that can be addressed with better technology requirements, other improvements need to be made including improved technical workflows and servers, staff training, and site monitoring on behalf of ACT officials.
For instance, one student reported that with the computer test, many students started the test at different times or accidentally early, which allowed surrounding students to receive breaks while others remained testing despite the noise and commotion. Clearly this is an issue ACT could address. Other students complained of system failures, troubleshooting teachers taking over their machines, mice that didn’t work, keys that stuck, and screens that lagged or froze. All these issues greatly reduced the time the students had to write essays or finish sections. For many students, these issues derailed scores because the tests could not be paused or effectively stopped.
Students also complained of technicalities in the administration. The reading and English sections boasted the ability to highlight in the passage onscreen, but apparently auto erase any highlights between questions. On paper, you highlight a passage and it stays. This discrepancy could cause some students to have more trouble tracking main ideas or important concepts. Other students complained of inconsistencies between announced policies (you’ll be given a dry erase board) and actualities (students were given scratch paper and did not bring their own pencils or erasers).
Computerized testing also brought up other issues which might be harder to mitigate.
For example, many students have reported that their scores on the computerized test are surprisingly lower than their paper test scores. Some of these students may plead “unfair.” And while I know that experience as a student is frustrating, I am also not surprised. That’s not to say these students don’t deserve better scores, but rather, that to broadly administer a new style of test across the world is no small feat. Additionally, computerized testing itself has inherent issues that unless a curve or scale is designed to mitigate these, it will likely result in most students receiving lower scores when taking the exam on a computer than they would on paper.
As a tutor, if my students have a choice, I always recommend the paper test. Copying your work down to scratch paper is time consuming which is the last issue you want during a timed exam. Reading from a screen, numerous studies have shown, makes deep learning more difficult and can take 20-30% longer to do. And yes, you do lose time as questions load. But that’s life, and that’s online testing.
I’m not ACT and I don’t know what their testing and design process entailed on this computerized test. But I do know that for the US students, ACT chose to leave the number of items on the test and the timing of the test the same as the paper exam. That alone is cause for concern. The problem with this grand ACT experiment is that there are too many variables involved with trying to make a test “fair” including administering it two different ways, taking choice away from students, and unforeseen technical problems that ACT did not think through. Thus, the application of this ACT experiment will create inequality of some sort.
For one, it’s going to be tough for ACT to standardize the computers used for the exam. Students may complain of old keyboards that are tough to type on or that outdated computers load questions more slowly than their home computers would. These complaints are unlikely to be avoided. Anytime you move to a fully computerized exam, there will be inequalities in equipment which are likely to exacerbate already existent socio-economic inequalities. Remember that the poorest countries and the poorest schools are less likely to have the best and fastest computers.
When all tests are given on a computer, the playing field is equalized as everyone is dealing with the same challenge. But as most US test takers still use old fashioned pencils and paper, and ACT likely created this test (which is reported to have the same test items and format as the paper test) imaning it would be administered on paper, it is understandable how these intense score drops occurred. What I think the ACT owes students is rigorous evidence-based administration of computerized tests and grading practices that allow for an equal playing field if the test is administered in different formats — if that’s possible.
ACT and the College Board have tough decisions to make. Will computerized tests be more secure? Maybe— but in this case it’s clear that other issues have emerged. Your best bet if you must confront one of these online tests is to be as informed as you can about the test your’e taking, practice with a simulator, and work from a computer rather than paper so you’re not handicapped by a transition from your normal practice routine.
