One of our principal researchers, Jackie, looks at how psychology can be used to make assessment better.
Developing and using school exams draws from psychology. We do psychological research to inform practice. Recently Martin Johnson, Tori Coleman and I used psychological research to show how to make the best use of assessors’ judgement when assessing complex competence. Some years ago Irenka Suto and I studied the cognitive strategies used in examination marking and the implications for GCSE marking. So I keep up to date with psychology to see how the latest work informs practice. Here I focus on two recent research articles about school exams.
The first article, by David Putwain and team, is about how students interpret teachers' ‘fear appeals’ about GCSE Maths. A teacher makes a fear appeal when they warn that particular actions may lead to negative outcomes. For instance: “If you mess about in class you will not do well in your Maths exam and you won’t get a decent job”. Students might appraise fear appeals as a challenge or a threat. Students who believe they can achieve, and who value educational attainment view the fear appeal as a challenge, generally. Students who value educational attainment but have low confidence in their ability to achieve tend to view the fear appeal as a threat.
Putwain and colleagues studied how challenge and threat appraisals relate to GCSE Maths exam results, and whether the statistical relationship is mediated through engagement. They collected the GCSE Maths exam results of 579 students, and their responses to two questionnaires: one about engagement (on task behaviour, persistence and classroom participation) and dissatisfaction with learning, and the second about whether fear appeals were seen as threats or challenges.
The results were that:
- stronger engagement predicted GCSE Maths exam performance.
- when students viewed fear appeals as a challenge they predicted good performance through higher engagement.
- when students viewed fear appeals as a threat they predicted low performance through lower engagement.
These results are a generalisation, they are not necessarily true for each and every individual.
Given these and similar findings in other research, Putwain and team suggest that teachers only use fear appeals about GCSE Maths with small groups of students or individuals who value educational achievement and believe they can achieve.
The second article, by Steven Howard and team, focuses on what traits are tested by Australia's national assessments. They say that many standardised literacy and numeracy tests are criticised for potentially measuring working memory and nonverbal reasoning in addition to the intended traits. Given these criticisms Howard and team researched whether working memory and nonverbal reasoning contributed to performance on Australia's National Assessment Program measures of Literacy and Numeracy (NAPLAN). This is a national standardised test of reading, writing, numeracy and language (punctuation, grammar and spelling). In the study 91 students aged 7 or 8, from a variety of schools, took the national assessments, a working memory test and a nonverbal reasoning test. The results showed that students' nonverbal reasoning and working memory contributed to their marks on the literacy and numeracy tests.
This reminds us of the importance of:
- saying what abilities an assessment tests
- assessing the intended abilities
- saying how results from the test can be used
In conclusion, we can learn from both articles, as they have direct practical applications.
Dr Jackie Greatorex, AFBPsS
Principal Research Officer, Cambridge Assessment
References:
Greatorex, J., Johnson, M., & Coleman, V. (2017). A review of instruments for assessing complex vocational competence. Research Matters: A Cambridge Assessment Publication, 23, 35-42.
Howard, S., Woodcock, S., Ehrich, J., & Bokosmaty, S. (2017). What are standardized literacy and numeracy tests testing? Evidence of the domain‐general contributions to students’ standardized educational test performance. British Journal of Educational Psychology, 87(1), 108-122. doi: 10.1111/bjep.12138
Putwain, D. W., Symes, W., & Wilkinson, H. M. (2017). Fear appeals, engagement, and examination performance: The role of challenge and threat appraisals. British Journal of Educational Psychology, 87(1), 16-31. doi: 10.1111/bjep.12132
Suto, W. M. I., & Greatorex, J. (2008). What goes through an examiner's mind? Using verbal protocols to gain insights into the GCSE marking process. British Educational Research Journal, 34(2), 213-233. doi: 10.1080/01411920701492050