-
Research Matters 13 - Foreword
Oates, T. (2012). Foreword. Research Matters: A Cambridge Assessment publication, 13, 1.
Innovation, creativity, learning to learn, critical thinking, 21st Century Skills. These are mentioned in many of the policy statements on education and training, in many nations - not just England. One of the contributions that an assessment organisation can make to the discussions of these ideas is that of clarity. Validity in assessment is vital. We need to understand what we are assessing and design our assessments to apprehend it in a reliable way.
Download
-
Research Matters 13 - Editorial
Green, S. (2012). Editorial. Research Matters: A Cambridge Assessment publication, 13, 1.
The themes addressed in this issue reflect the diversity of research carried out at Cambridge Assessment. Several of these articles report on studies that have been presented at national and international conferences this year.
Download
-
International assessment through the medium of English: analysing the language skills required
Shaw, S. (2012). International assessment through the medium of English: analysing the language skills required. Research Matters: A Cambridge Assessment publication, 13, 2-10.
English is the medium of instruction and assessment for Cambridge Assessment International Education programmes of learning and summative assessment. Cambridge International programmes are increasingly offered in a variety of educational and bilingual contexts, and for many international students English is an additional language – their second language, or perhaps their third language. The Cambridge International context raises a number of issues relating to language awareness (e.g., progression from basic interpersonal communication skills to cognitive academic language proficiency) and assessment (e.g., the level of English needed to access and succeed in international assessments). The focus of the study is the International General Certificate of Secondary Education (IGCSE). The study adopts a two-phase methodology and involves an analysis of language use in Geography, History and Biology in order to (a) build a ‘profile’ of the language skills required and evidenced by IGCSEs and (b) determine whether any identifiable linguistic patterns adhere to different content, non-language IGCSEs. The level of English language proficiency required for accessing and succeeding in IGCSE is tentatively mapped to the Common European Framework of Reference for Languages (CEFR).
Download
-
An investigation into the number of special consideration enhancements and their impact on examination grades
Vidal Rodeiro, C. L. (2012). An investigation into the number of special consideration enhancements and their impact on examination grades. Research Matters: A Cambridge Assessment publication, 13, 11-17.
Special consideration is a reasonable adjustment for students who were fully prepared for an examination and covered the course, but whose performance was affected by circumstances beyond their control (e.g., recent bereavement, serious accident, illness). This research investigated the number of special consideration requests (over time, by qualification and by school type) and the effect of the enhancements on the overall grades in a range of GCSE and A level subjects.
The research shows that the number of special consideration applications rose in the period of study, with more requests at A level than at GCSE. Furthermore, candidates in independent schools were more likely to submit a request for special consideration than candidates in state schools. The special consideration enhancements considered in this work were minor adjustments to candidates’ marks, with the most common tariff applied being 2% of the unit/component mark (normally due to minor illnesses at the time of the examination). As a result, the percentages of students improving their overall grades in GCSE and A level subjects were very small (lower than 1% in all subjects considered).
Download
-
The effect of manipulating features of examinees' scripts on their perceived quality
Bramley, T. (2012). The effect of manipulating features of examinees' scripts on their perceived quality. Research Matters: A Cambridge Assessment publication, 13, 18-26.
Expert judgment of the quality of examinees’ work plays an important part in standard setting, standard maintaining, and monitoring of comparability. In order to understand and validate methods that rely on expert judgment, it is necessary to know what features of examinees’ work influence the experts’ judgments. The controlled experiment reported here investigated the effect of changing four features of scripts from a GCSE Chemistry examination: i) the quality of written English; ii) the proportion of missing as opposed to incorrect responses; iii) the profile of marks in terms of fit to the Rasch model; and iv) the proportion of marks gained on the subset of questions testing 'good Chemistry'. Expert judges ranked scripts in terms of perceived quality. There were two versions of each script, an original version and a manipulated version (with the same total mark) where one of the four features had been altered. The largest effect was obtained by a combination of iii) and iv): increasing the proportion of marks gained on ‘good Chemistry’ items, and increasing the number of correct answers to difficult questions at the expense of wrong answers to easy questions. The implications of the findings for operational standard maintaining procedures are discussed.
Download
-
Starting them young: research and project management opportunities for 16 to 19 year olds
Suto, I. and Nadas, R. (2012). Starting them young: research and project management opportunities for 16 to 19 year olds. Research Matters: A Cambridge Assessment publication, 13, 27-31.
Several educational routes have been developed which entail project work with a specific focus on independent learning and research. In this article we outline some of the options that exist at Level 3, primarily for 16 to 19 year olds (Years 12 and 13) in the UK and internationally. We then conduct a more detailed comparison of two routes: the Extended Project Qualification, and the International Baccalaureate Extended Essay. Many stakeholders may be unaware of the differences in the aims, structure, and scope of these routes. It is important for students and teachers to be conscious of the differences so that they can make informed decisions about what is most suitable for them. End-users such as higher education admissions tutors and employers also need to understand the differences in order to weigh up the experiences and achievements of applicants fairly.
Download
-
An investigation into the impact of screen design on computer-based assessments
Haigh, M. (2012). An investigation into the impact of screen design on computer-based assessments. Research Matters: A Cambridge Assessment publication, 13, 31-37.
This article reports on part of a study that investigated the impact of item format on the difficulty of test items. Two parallel forms of a computer-based test were developed; each test consisted of 15 items based on the GCSE Science curriculum. Five items were identical in both forms of the test to act as a control. The remaining items were modified in the parallel forms to investigate the effect of manipulating ten aspects of item format, including, for example, drag and drop categorisation vs. tick-box categorisation; and static graphic vs. animated graphic. The outcomes indicated that there was little effect on quantitative measures of item difficulty when the item format was changed.
Download
-
Making the most of our assessment data: Cambridge Assessment's Information Services Platform
Raikes, N. (2012). Making the most of our assessment data: Cambridge Assessment's Information Services Platform. Research Matters: A Cambridge Assessment publication, 13, 38-40.
As new technologies penetrate every part of educational assessment, data is being collected as never before. Traditionally, there were two methods of producing statistical information within Cambridge Assessment. Routine statistical information came from reports built into bespoke examination processing systems. Non-routine analysis and reports were produced by small teams of statistical experts, typically working within research units and using statistical software packages on personal computers. With increasing demand for flexible, high-volume statistical reporting, a new solution was required; one which combined the resilience and scalability of a server-based infrastructure with the flexibility of having statistical experts in charge of creating the statistical content. The Information Services Platform (ISP) is Cambridge Assessment’s solution for these requirements. It provides our statistical experts with access to operational assessment data and tools to automate and schedule analysis and report, and to publish the resulting content on an Intranet Portal for use by colleagues across the organisation. In this paper, I discuss further the thinking behind the ISP and give practical examples of use.
Download
-
Statistical Reports
The Research Division (2012). Statistical reports. Research Matters: A Cambridge Assessment publication, 13, 40.
The ongoing Statistics Reports Series provides statistical summaries of various aspects of the English examination system, such as trends in pupil uptake and attainment, qualifications choice, subject combinations and subject provision at school. This article contains a summary of the most recent additions to this series.
Download
-
Research News
The Research Division (2012). Research News. Research Matters: A Cambridge Assessment publication, 13, 41-42.
A summary of recent conferences and seminars, and research articles published since the last issue of Research Matters.
Download