| Speakers: |
Stuart Shaw (Principal Research Officer, Cambridge Assessment International Education) |
| Date: |
06 Feb 2019 |
Venue: |
Cambridge Assessment
The Triangle Building, Shaftesbury Road
Cambridge
CB2 8EA
|
| Time: |
09:30
-
12:30
|
| Type: |
Training event |
Fee: |
£70.00 |
This session will respond to two fundamental questions about validity in assessment:
- What does it mean to claim validity?
- How can a claim to validity be proven?
Workshop outline
Stuart will begin by tracing the evolution of validity from disparate and contested origins to a point where there now appears to be a strong professional consensus over its meaning - reflected in the maxim ‘all validity is construct validity’. Despite fairly general professional agreement over some of the fundamental principles of validity, there still remains considerable confusion over the scope of the concept. Even the consensus over the construct validity maxim seems problematic when scrutinised carefully.
Discontent with the broad perspective has been voiced by a number of notable validity theorists. In this workshop you'll be given an opportunity to explore issues surrounding the current consensus as well as the chance to consider alternate, divergent conceptions of validity.
The session was extremely valuable because in higher education we are always writing assessment questions but no one ever teaches us how to do this. The course made me think in a more constructive way about the purpose of a question and the purpose of the mark scheme."
Stuart will highlight the challenges faced when validating the intended interpretation of test scores and their relevance to the proposed uses of those scores.
Key learning outcomes
Specifically, he'll address a number of outstanding validation challenges:
- Where to start (identifying claims, purposes, interpretations and uses of test scores).
- How to proceed (determining the relevance and sufficiency of validation evidence).
- When to stop (evaluating validation arguments).
- How to report (identifying appropriate audiences and tailoring content to their requirements).
Resources are available for anyone who is unfamiliar with the concept of validity, and who may benefit from reading one or two articles prior to the workshop.
There is also literature available for those who are already engaged with the issues of validity theory and validation practice and for those who wish to explore the technical and philosophical foundations of validity in more detail.
Course trainer
Stuart Shaw began his career as an engineer, and holds an honours degree in Physics, a diploma in Applied Physics and a research degree in Metallurgy. His early experience, gained with an international plc, covered a range of engineering specialisms. Following his time in industry, he entered the TEFL world (Teaching English as a Foreign Language), gaining a certificate and diploma in TESOL and a Master degree in Applied Linguistics. He had several years of experience as an EFL teacher and Director of Studies. Stuart also holds a postgraduate degree in Theology.Stuart has worked for Cambridge Assessment since January 2001 where he is particularly interested in demonstrating how Cambridge Assessment seeks to meet the demands of validity in its assessments. Before leading a research team in the area of mainstream international examinations, Stuart worked on a range of Cambridge English products with specific skill responsibilities for assessing writing. He is an experienced presenter and has lectured for the Department of Theoretical and Applied Linguistics (University of Cambridge). He is currently an affiliated lecturer with the Faculty of Education (University of Cambridge).
He has a wide range of publications in English second language assessment and educational research journals. Recent assessment books include: Examining Writing: Research and practice in assessing second language writing (Shaw & Weir, 2007); The IELTS Writing Assessment Revision Project: towards a revised rating scale (Shaw & Falvey, 2008); Validity in Educational and Psychological Assessment (Newton & Shaw, 2014); and Language Rich: Insights from Multilingual Schools (Shaw, Imam & Hughes, 2015). He has recently contributed to validity debate through a Special Issue of Assessment in Education: Principles, Policy & Practice (2016). He is a Fellow of the Association for Educational Assessment in Europe (AEA-Europe) and member of the Professional Development Committee (AEA-Europe). He is also a Fellow of the Chartered Institute of Educational Assessors (CIEA).