| Speakers: |
Dr Iasonas Lamprianou (European University-Cyprus) |
| Date: |
15 Mar 2011 |
Venue: |
Downing College
Regent Street
Cambridge
Cambridgeshire
CB2 1DQ
|
| Type: |
Seminar |
A frequent claim of awarding bodies is that standards are comparable across assessments in different subject areas, and if they are not, the awarding bodies can make them comparable through appropriate statistical techniques. It is not surprising that in many countries such brave statements are received with suspicion and mistrust by the stakeholders.
At a recent seminar hosted by the Cambridge Assessment Network, guest speaker Dr Iasonas Lamprianou of the European University in Cyprus warned how using a specific inter-subject comparability method can affect the chances of some students in gaining access to university.
As part of the admissions process, universities and colleges make comparisons between applicants with different qualifications. Should one try to distinguish between two candidates with the same A-level grades but in different subjects? And could the choice of method used in subject comparability exercises ultimately affect the chances of groups of students in accessing Higher Education?
Dr Lamprianou scored a group of students' performance using four different methods:
- Scaling method of the Cyprus Testing Service
- Natural Scores or National Ratings (hence, NR) of the Scottish Qualification Authority
- Rasch model
- Multilevel Models (MLM)
The findings showed how the different methods affect reported scores in a different way, although the correlations were high.
Lamprianou used recent empirical data in order to study the attempts of the Cyprus Testing Service to aggregate candidates' scores from various examination subjects. The aim is to compute a single index of 'academic performance' for each candidate, although different candidates may be assessed on very different examination subjects.
The study showed how using a specific inter-subject comparability method (the 'Cyprus scaling method') affects the chances of some candidates to get access to Universities. It also showed how using the Scottish 'National Ratings' method on the same data affects the chances of specific sub-groups (e.g. boys, students of vocational education schools etc.) to achieve access to higher education. Depending on the method used, different groups of candidates may be 'penalised'.
Although one would expect different models to produce different results, in the context of inter-subject comparability methods this raises questions regarding the ethical as well as the legal dimensions of making such comparisons. For example, how can one justify the use of a specific comparability method if it may reduce the chances of candidates from lower socioeconomic classes to gain access to higher education?
Lamprianou commented: "There are ethical, political, moral, legal issues…there are psychometrical, statistical issues. These are ‘real world’ problems. I think future research on this issue should focus a lot more on the consequences; on the political aspects, because these sorts of decisions are mainly political and not psychometric."
Related materials