Launched in November, Cambridge Assessment Network’s assessment practitioner workshops focus on important elements of assessment design and practice. Aaron Mortlock, Business Development Manager for Cambridge Assessment Admissions Testing, recently attended the first series of workshops on Designing effective assessments and items. Below he gives his account of how the workshop series has improved his assessment knowledge and impacted his day to day practice.
Becoming an assessment advisor
I am always keen to keep learning about all things education and assessment for my personal development, and I decided to attend the Designing effective assessments and items workshop series in order to better support the schools, universities and other educational institutions I work with.
As the main customer-facing person, I always want to be an adviser, rather than just a provider, to potential customers so they can make informed decisions. That means being able to provide them with the information to help them decide whether our assessments are right for them or not. It can also have the benefit of reducing the burden on our assessment teams in answering some of the basic assessment-specific questions.
An engaging and accessible environment
The workshops started with the pre-question writing ‘groundwork’ to make sure you start things off right; for example, creating your assessment strategy and ‘blueprint’. We then progressed to core principles and issues in question (or ‘item’) design, and then onto good practice in mark schemes.
Each week progressed nicely and the short recaps were enough to refresh our memory from the previous week (otherwise I think lots of us would have forgotten things!). The presenters shared their knowledge and experiences each week, and also actively engaged us throughout. This also included group activities, which can initially feel daunting with new people but in this environment it worked very well.
We could engage with others with different backgrounds and experiences, sharing ideas rather than seeking ‘right answers’.
We could engage with others with different backgrounds and experiences, sharing ideas rather than seeking ‘right answers’. We could do this via a chat function or microphone, and on or off camera; this makes the whole experience feel very accessible to individuals with different personalities too.
Putting the pieces of the assessment puzzle together
The more I learn, the better I can support our customers. Each course I attend helps bring together the puzzle pieces that make up high quality assessments. It continues to provide those ‘aha’ moments – when I have been listening to presentations or reviewing publications and things didn’t quite make sense at the time and suddenly my understanding falls into place.
To provide a specific example, our assessments (in Admissions Testing) are typically in English but are assessing subject-specific knowledge and/or skills. The concept of ‘construct irrelevant variance’ is critical here because if the English level is too high (or higher than one’s competence), we are testing their English level first and it distracts from their ability to demonstrate the knowledge or skill being assessed. This is a critical point when speaking to universities, for example, as it does not replace an English Language qualification an applicant may be required to obtain.
It continues to provide those ‘aha’ moments – when I have been listening to presentations or reviewing publications and things didn’t quite make sense at the time and suddenly my understanding falls into place.
More broadly, I attend a lot of conferences where research is presented. Attending this training, as well as Introduction to statistics for assessment series of workshops, will allow me to have the confidence to ask informed and sometimes challenging questions that I would not have felt comfortable asking previously. For example, understanding where correlations are statistically significant and the factors that may have attributed to that correlation.
As of January 2021 I will become a teacher, teaching accountancy qualifications. This knowledge will provide me with a great foundation to understand the assessments the students will be working towards and to constructively challenge the Awarding Bodies who provide the assessments too!
Making assessments the best they can be
Examiners have an incredibly complex task and having subject knowledge is only the start of the journey to well-designed assessments. This series of workshops has opened my eyes and I now have a more empathetic view towards examiners and the rigour that is involved in item writing, and the focused effort on making the assessment the best it can be for the students on test day.
I have learnt a lot and there is much more to learn, but each course gives me more clarity on the challenges and best practices in designing valid, fair and reliable assessments.
For example, I see how important and impactful choice of language can be, such as indefinite vs definite language (a vs the), time ambiguity (‘immediately before’ or only ‘before’) and keeping consistent layouts and lexicon. It is so important to have an appreciation of all these small nuances and a clear understanding of their impact in order to deliver appropriate assessments.
I would highly recommend the assessment practitioner workshops from Cambridge Assessment Network as they provide a good balance of learning and engagement with others from different educational environments from all around the world. I have learnt a lot and there is much more to learn, but each course gives me more clarity on the challenges and best practices in designing valid, fair and reliable assessments.
Cambridge Assessment Network is an accredited provider of assessment training and professional development. The Assessment Network exists in order to provide high-quality assessment training, underpinned by research, to Cambridge Assessment staff and the wider assessment community.
Find out about our upcoming assessment practitioner workshops on our events pages.