Validation is the process of collecting and analysing various forms of data to build an argument concerning the overall quality (validity) of your assessments. It is one of the most important ideas in educational assessment because it is an essential process to ensure confidence and trust in the assessment outcomes.
One analogy that can be drawn is that validation is much like a courtroom trial. We put our assessment ‘on the stand’ and use different sources of evidence to build cases for and against its validity.
The case
In a courtroom trial, a particular claim is made against a defendant (i.e. that they are guilty of a crime). Validation enquiries are also dependent on claims being made – generally about how assessments can be interpreted and used. For example, you might wish to claim that an assessment distinguishes between the highest and lowest-performing students, or that an assessment has sampled the course curriculum sufficiently.
In trials, evidence is presented both in favour of the claim that someone is guilty (the prosecution) and against the claim (the defence). In validation, assessors look to collect qualitative, quantitative and documentary evidence to either support the claims that they have made or to find possible ‘threats’ to validity - they are both the prosecutor and the defence.
Finally comes the verdict. Similar to trials, it is not possible to ‘prove’ the validity of an assessment. In reality, we are looking to build a strong enough case to prove validity beyond a reasonable doubt so that the assessment can be judged acceptable to stakeholders. In this sense, stakeholders such as students, teachers, parents and assessors act as the jury!
Collecting evidence
The importance of validation means that awarding organisations spend a lot of time and resources validating their assessments. Cambridge University Press & Assessment have dedicated teams that conduct in-depth research, analyse various forms of data, and apply complex statistical techniques so that we can learn more about how well our assessments have worked.
The scale and long-term nature of these investigations mean it can be difficult to know where to start with validation, particularly for busy practitioners who don’t have too much time to check the quality of their assessments.
That is why Cambridge Assessment Network has created a new and accessible Assessment Practitioner Workshop: Validation for beginners: How do I know my assessments have worked well? The aim of the workshop is to provide people designing and delivering assessments with the fundamental theoretical and practical knowledge to structure their validation enquiries.
Judging your assessments
As Head of Assessment Training at Cambridge Assessment Network, I enjoy translating and distilling the latest thinking in areas like validation and converting these ideas into digestible, practical, and applicable courses. It is important to me that assessment practitioners leave my courses knowing precisely what their next steps will be in making their assessments work better for them.
In the workshop, participants will learn about a five-point plan for evaluating the quality of their assessments and think about how they can best prioritise their time.
By the end of the workshop learners will have:
- Identified where validation is required and useful to their professional practice
- Understood key principles of validation enquiry
- Developed new strategies and methods to guide validation practice
- Have confidence justifying professional choices based on validation best practice
I wanted to introduce this workshop to the Cambridge Assessment Network training programme because I see it as introducing a key skill set and structure for thinking about validation that assessors can use again and again. The workshop is designed to be accessible to anyone who wants to find out how well their assessments have met their purpose. Or in other words, those who want to build a case for the validity of their assessments beyond a reasonable doubt.
If you are looking to interrogate the quality of your assessments, you can book a place on our workshop or contact thenetwork@cambridgeassessment.org.uk and we will answer any of your questions.