| Date: |
Date to be confirmed |
Venue: |
Online
|
| Time: |
12:30 - 14:00
|
| Type: |
Workshop |
Fee: |
£95 (Members - £85.50) |
Register your interest
This workshop is accredited for continuing professional development (1.5 CPD hours), with certification post attendance.
This interactive workshop will provide an overview of the history, theory and practice of comparative judgement in educational assessment.
Comparative judgement as a method has been around since the 19th century, starting in psychophysics. It is essentially a method for ordering something on a numerical scale based on relative human judgements. In educational assessment, for example, candidate responses are put in order of the quality of work.
Comparative judgement currently has two main applications in educational assessment:
- As an alternative to conventional marking (rating/scoring)
- As a means of linking the mark scales of assessments that have been marked in the usual way
This workshop will explore how both work, and how to evaluate their advantages and disadvantages.
Relevant to anyone working in education or assessment who wants to know more about this innovative assessment method, this workshop will provide a comprehensive overview of the theory and practice of comparative judgement and an opportunity to engage in the ongoing debates surrounding the practice.
Course outline
This workshop will provide an overview of the history of comparative judgement and the areas where it is widely used. You will learn how to design a comparative judgement study, and the main issues that need to be considered.
It will also cover:
- The main statistical models that are used to analyse comparative judgement data and create the scale
- The main applications of comparative judgement in educational assessment:
- As an alternative to marking
- Linking mark scales on two assessments (standard maintaining)
- Different varieties of comparative judgement with advantages and disadvantages
- Paired comparisons vs ranks of more than 2 objects
- Adaptive comparative judgement vs non-adaptive
- ‘Simplified pairs/ranks’ for standard maintaining
- Ongoing debates about comparative judgement in assessment.
Key learning outcomes
By the end of the workshop you will have:
- An understanding of what comparative judgement is, and how it has been applied in educational assessment
- Explored issues involved in designing a comparative judgement study
- Knowledge of how to interpret the statistical information associated with the most common kinds of comparative judgement analysis
- An awareness of some of the ongoing debates about comparative judgement in assessment
Course trainers
Tom Bramley is the Director of Cambridge Assessment’s Research Division. His main areas of expertise are Rasch measurement and Comparative Judgement, and the psychometric issues associated with test development, standard setting, standard maintaining and comparability monitoring. He is a member of the editorial board of the journal ‘Assessment in Education: Principles, Policy and Practice’; and of Ofqual’s Standards Advisory Group and Research Advisory Group.
Gill Elliott is the Deputy Director of Cambridge Assessment’s Research Division. Gill's background is in Psychology, and she has a BSc from the University of York as well as an M.Ed from the University of Bristol. Gill joined the newly-formed research department of UCLES in January 1994, and has worked on a wide variety of projects including extensive comparability work, surveys of centres and candidates and detailed evaluation of individual specifications. Gill leads the comparability strand of the research programme, and has been involved for more than 20 years with the Aspects of Writing research which examines elements of 16+ candidates’ writing at various points in time. As Deputy Director in the Research Division Gill has close involvement in the management and oversight of the division alongside her own project work.
Sarah Hughes is Research Information Manager at Cambridge Assessment International Education. She has taught in primary schools and been a researcher for UK and international awarding bodies and government agencies. Her particular research interests are the validity of assessment and the impact of research on practice and policy. She currently carries out and applies research to support assessment practitioners and to improve assessment quality.