19 May 2022
Every 14/15-year-old in the country should sit a mock online GCSE as soon as next year, an academic has told a Cambridge University Press & Assessment event.
John Jerrim, Professor of Education and Social Statistics at University College London’s Institute of Education, said there was no time to lose because teething problems would need to be tackled early if online GCSEs were to go live in 2025.
In his talk to Cambridge Assessment Network, Prof Jerrim drew upon his experience from working with the OECD in 2015, when its Programme for International Student Assessment (PISA) test switched from pen-and-paper to digital.
Prof Jerrim focused on “mode effects” – the impact on students’ results of sitting a digital exam as opposed to a pen-and-paper exam.
Opening his talk, Maintaining comparability of results when GCSEs go digital - A reflection on lessons learnt from PISA, Prof Jerrim said the covid pandemic had naturally led to renewed policy focus and interest in high stakes digital assessment. While he dismissed the argument that digital assessment would make education more robust to shocks like the pandemic – “that horse has bolted”, he said – he argued there were significant benefits to moving online. These could include adaptive testing as well as new, interactive question types – beneficial particularly in a hands-on subject like science, he said.
Prof Jerrim focused on “mode effects” – the impact on students’ results of sitting a digital exam as opposed to a pen-and-paper exam. He noted how the OECD had to some extent been a trailblazer when it moved its PISA assessment online in 2015. But he acknowledged that there were teething problems, some of which “would have been a disaster” if it had been a high-stakes exam like the GCSE – including the accidental wiping of a memory stick with a school’s entire PISA results on it.
He said the OECD had investigated mode effects and found some negative impact on students’ results from the move to digital. One of the problems however was that there was no research to explain the impact and what was happening. If it were a high-stakes exam like the GCSE, much more research would need to be done on mode effects and that meant mocks would be needed as early as next year.
Tim Oates CBE, Group Director, Assessment Research and Development at Cambridge University Press & Assessment, said it had been “bang on topic, a really rich exploration of the issues of item and instrument design”.
“The government should be doing a lot of work on this now,” he said. “It’s going to take time to realise any benefits – maybe 10 years, maybe a bit sooner - so we’ve got to bite the bullet now. It does feel like now is the time to gradually make the transition, to get students used to taking digital assessments”.
Thanking Prof Jerrim for his talk, Tim Oates CBE, Group Director, Assessment Research and Development at Cambridge University Press & Assessment, said it had been “bang on topic, a really rich exploration of the issues of item and instrument design”.
“You then discussed the management challenge of delivering an online assessment, when you’re under pressure, and then extrapolated from that to high-stakes exams in national settings,” Tim said. “And you dealt with the practical issues of the skills of young people and their response to this kind of test. It was really insightful.”
Cambridge University Press & Assessment has a dedicated unit in charge of the move to digital high-stakes assessments. It has already held more than 2,000 onscreen mocks in three trial qualifications and marked and returned the results to students.
You can watch the talk back over on our YouTube channel.