There’s a lot of discussion about ‘getting rid of these qualifications here’ and ‘reforming those qualifications over there’; talk of reform, revision and ‘modernisation’.
There are many issues to consider in the balance of different modes of assessment, the sequencing of assessment and ‘national standards’ - detailed matters of the balance of teacher assessment and formal exams are important, there’s no doubt about that. But all the discussion about exams or no exams, final assessment versus on-going assessment, past papers versus new papers....all of this does take attention away from perhaps the most important aspect of assessment: it’s about good questions. I believe we need to focus in on this far more.
Encouraging deeper thinking
There are certainly bad practices in trying to ‘teach to the test’. A brilliant head once told me of a class he observed, where pupils were given copies of the poems of William Blake. One pupil immediately went head down into the text. The teacher spotted this and said ‘...No! Stop! We are not going to read the poems, we just need to focus on specific lines to memorise for the exam...’. This kind of practice has given exam questions a bad name.
Contrast Isaac Physics, which has selected good physics questions from the Cambridge Assessment archive - some of them over 100 years old - and made them available online. Contrary to the moves towards ‘accessibility’ and ‘scaffolding’ Isaac Physics has left the questions as demanding items, but with a succession of hints which can be unlocked if needed. Even the highest attaining pupils tend to look at the first hint, to reassure themselves that they are on the right lines in thinking about the question. The use of Isaac Physics has gone through the roof, and it’s now accessed over a hundred thousand times a day, by both teachers and pupils. They contribute as much to learning as they do to assessment - insight into what pupils know, understand and can do.
These carefully selected questions both stimulate thought and reveal the way in which pupils are thinking. They contribute as much to learning as they do to assessment - insight into what pupils know, understand and can do. And of course they familiarise pupils with the sorts of questions which will arise in exams. After tackling dozens, even hundreds of questions, unfamiliar questions just become one more challenge, not a strange and stressful experience. This level of use of questions broadens learning programmes - if doesn’t narrow them. Looking through a stack of past Macbeth GCSE questions introduces ideas of ambition, regicide, loyalty, hubris, witchcraft, female identity, and much, much more. It encourages wide and deep thinking about the play, not narrowness. Ask a demanding question at the end of a lesson and come back to it at the beginning of the next - this encourages thinking about a subject outside contact time; something which we know elevates attainment.
And the latest cognitive science tells us that this kind of use of demanding questions is fundamental to better learning. Kurt Fisher’s work emphasises that we need to commit facts, concepts and insights to long term memory, where they become ready resources for high level and creative thinking. Karpicke’s work builds on this, and shows how learning followed by ‘retrieval practice’ - answering questions on a topic or area - is better at consolidating learning than studying, and then re-studying something. I know my name automatically, since I have been asked about it so many times. The same is true of remembering the chemical formula for sodium hydroxide, the reason why Germanicus was called Germanicus, and the concept of constitutional monarchy. And if they have been consolidated in memory we have enduring, persistent learning. This research shows that using high quality questions effects exactly that important consolidation. And they have the benefit of both confirming whether someone has learned what they need to learn, and revealing misconceptions.
Flooding the system
Exam questions have picked up a bad reputation, since they are associated with surprise, ‘ambush’, stress, and ‘high stakes’. But it’s wrong to associate high quality questions only with these perspectives on assessment. Questions are designed to produce responses. And from those responses we can make inferences about what someone thinks and can do and finance.
Exam questions are designed to produce responses. From those responses we can make inferences about what someone thinks and can do.
That can lead to rich discussion, exploration of ideas, correction of misconceptions, and an opportunity to understand the mental lives of others. They are as important in vocational education and training as they are in academic education. Why something should be done a particular way, understanding why something is happening, understanding risks, limits and dangers is important in medicine, in engineering.
We do try to increase fairness by exposing pupils to questions in controlled settings - timed, secure exams - where superficial memorisation has been made impossible by keeping the questions secure. And we do need to know that it is genuinely who we think it is who is answering the question. Cheating can and does happen. Good questions, which clearly differentiate those who have mastered an idea or can demonstrate a skill, are expensive to create. Designing them, evaluating and refining them - this takes time and skill.
But I think these issues of security and technical precision have led us to treat exam questions as ‘special objects’. What if we used a different approach? What if we just flooded the whole education system with good questions?
Flooding the whole education system with good questions would help pupils better understand their progress and understanding.
No one would know exactly which ones would come up in a given exam. This would be entirely consistent with the latest research on high quality learning. It would help pupils better understand their progress and understanding. Teachers would know more about their pupils. Good questions would stimulate thinking and discussion, and develop enquiring minds. Far from narrowing the curriculum, it would deepen and enrich learning. Pandemic has accelerated some things which already were happening - on line topic tests, a greater flow of questions to support learning. So....
