-
Research Matters 19 - Foreword
Oates, T. (2015). Foreword. Research Matters: A Cambridge Assessment publication, 19, 1.
As this issue of Research Matters goes to press, two important matters remain highly controversial issues in qualifications policy in England – the move in Science from assessment of practical work, and the ‘de-coupling’ of AS and A level examinations – to a position where AS level will continue to exist, but performance in the AS level will not contribute to the grade in the corresponding A level. The approach in Science need not reduce the amount of engaging practical work – but it may so do. The de-coupling of AS level need not reduce the numbers using AS to refine their choices – but it may so do. The proof of the pudding will be in the eating.
Download
-
Research Matters 19 - Editorial
Green, S. (2015). Editorial. Research Matters: A Cambridge Assessment publication, 19, 1.
Following the recent launch of Cambridge Mathematics, most of the articles in this 10th anniversary issue focus on Mathematics in a range of contexts. Details of Cambridge Mathematics can be found at www.cambridgemaths.org.
Download
-
A level History: Which factors motivate teachers’ unit and topic choices?
Child, S., Darlington, E. and Gill, T. (2015). A level History: Which factors motivate teachers’ unit and topic choices? Research Matters: A Cambridge Assessment publication, 19, 2-6.
The flexibility inherent in A level History qualifications means that teachers have to negotiate competing factors that may influence topic, unit or qualification choices. The present article aimed to use questionnaire data derived from heads of History departments to analyse the motivations underpinning the unit and topic choices for an A level History course. A second aim was to analyse whether the Heads of Department from different school types had different influences underlying their choices.
The two most common motivating factors underlying teachers’ choices of units and topics were found to be teacher expertise and perceived student engagement. Fisher’s Exact analyses revealed that these motivations were deemed significantly more important by state school teachers, compared to independent school teachers, in guiding their topic selections (both p < .05). There were also statistically significant differences between school types in terms of how their Heads of Department rated the importance of the curriculum support offered via different resources.
These findings are discussed with reference to the recent qualifications reform in the UK, and the role of the teacher in determining topic choice and delivery of history.
Download
-
Context led Science courses: A review
Wilson, F., Evans, S. and Old, S. (2015) Context led Science courses: A review. Research Matters: A Cambridge Assessment publication, 19, 7-13.
Internationally, there is growing concern about secondary Science education. In many developed countries uptake of Science subjects has been falling, leading to fears that there will be a shortage of people with the scientific skills and knowledge needed in the twenty-first century. Science curricula are often considered to suffer from an overload of content, leading to the perception that Science subjects are among the most difficult. Furthermore, students have difficulties connecting the isolated facts which they are taught, and do not develop coherent mental schema. Content is often presented in an abstract manner that is remote from students’ everyday experiences, so that many students do not understand why they should learn the materials which they are studying, and frequently fail to do so. As a result, students have difficulty applying scientific concepts in a context beyond the one in which they were taught that concept. In this article we examine one approach to Science education: context led Science courses, which have been developed as a result of these concerns.
Download
-
Assessing active citizenship: An international perspective
Carroll, P., Child S. and Darlington, E. (2015). Assessing active citizenship: An international perspective. Research Matters: A Cambridge Assessment publication, 19, 14-19.
This article analysed four education systems across five countries in terms of their approach to the assessment of ‘active’ citizenship. The four countries chosen were: England and Wales, Australia, Singapore and the USA (specifically Washington and Florida). ‘Active’ citizenship was defined as an amalgamation of knowledge (political literacy) and action (civic duty). As such, an assessment which tests both these constructs would be needed to provide a valid measure of active citizenship. For each education system, an overview of the placement of citizenship education is provided, alongside a critical analysis of the assessment of ‘active’ citizenship from the perspective of achieving test validity.
Download
-
An investigation into the numbers and characteristics of candidates with incomplete entries at AS/A level
Vidal Rodeiro, C.L. (2015). An investigation into the numbers and characteristics of candidates with incomplete entries at AS/A level. Research Matters: A Cambridge Assessment publication, 19, 19-25.
AS and A levels are undergoing a comprehensive reform. In particular, from 2015, the AS level will be de-coupled from the A level and will become a standalone qualification, rather than being part of an A level. This move has raised concerns, as without a direct link to A levels, the new AS levels could reduce participation in subjects such as Mathematics or languages and may not be as beneficial.
This research aimed to understand the numbers and types of students who start but do not complete their AS and A levels. Its outcomes will add to the debate surrounding the AS/A level reform and could help to anticipate changes in the uptake of the new AS levels. Furthermore, the subjects taken in the sixth form have implications for university recruitment and, therefore, a good understanding of how students refine their subject choice at the end of the first year of advanced study is crucial.
Download
-
The moderation of coursework and controlled assessment: A summary
Gill, T. (2015). The moderation of coursework and controlled assessment: A summary. Research Matters: A Cambridge Assessment publication, 19, 26-31.
To ensure consistency and accuracy of marking, awarding bodies carry out moderation of GCSE and A level internally assessed work (e.g., coursework or controlled assessment). Training and instructions are provided by the awarding body to the internal assessors in each centre, including training in task-setting, marking and internal standardisation. Awarding bodies are required to modify centres’ marks where necessary to bring judgements into line with the required standard.
Samples are taken of (internally standardised) candidates’ work. A moderator re-marks the sampled work, and if there is a difference between the centre’s and moderator’s marks that is larger than a certain amount then marks should be adjusted. Should it be necessary to adjust a centre’s marks then the magnitude of the adjustments is determined by a regression analysis, based on the relationship between the marks given by the centre and those of the moderator in the sample.
This article summarises the processes undertaken by the Oxford, Cambridge and RSA (OCR) exam board to moderate and, if necessary, adjust the marks of centre-marked coursework and controlled assessments. Some brief data analysis is also presented to give an idea of the extent of moderation and how much difference it makes to candidates’ marks.
Download
-
Reflections on a framework for validation – Five years on
Shaw, S. and Crisp, V. (2015). Reflections on a framework for validation – Five years on. Research Matters: A Cambridge Assessment publication, 19, 31-37.
In essence, validation is simple. The basic questions which underlie any validation exercise are: what is being claimed about the test, and are the claims warranted (given all of the evidence). What could be more straightforward? Unfortunately, despite a century of theorising validity, it is still quite unclear exactly how much and what kind of evidence or analysis is required in order to establish a claim to validity. Despite Kane’s attempts to simplify validation by developing a methodology to support validation practice, one which is grounded in argumentation (e.g., Kane, 1992), and the “simple, accessible direction for practitioners” (Goldstein & Behuniak, 2011, p.36) provided by the Standards (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education [AERA, APA, & NCME], 2014), good validation studies still prove surprisingly challenging to implement.
In response, a framework for evidencing assessment validity in large-scale, high-stakes examinations and a set of methods for gathering validity evidence was developed in 2008/2009. The framework includes a number of validation questions to be answered by the collection of appropriate evidence and by related analyses. Both framework and methods were piloted and refined. Systematic implementation of the validation framework followed which employs two parallel validation strategies:
1. an experimental validation strategy which entails full post-hoc validation studies undertaken solely by research staff
2. an operational validation strategy which entails the gathering and synthesis of validation evidence currently generated routinely within operational processes.
Five years on, a number of issues have emerged which prompted a review of the validation framework and several conceptual and textual changes to the language of the framework. These changes strengthen the theoretical structure underpinning the framework.
This paper presents the revised framework, and reflects on the original scope of the framework and how this has changed. We also consider the suitability and meaningfulness of the language employed by the framework.
Download
-
Text Mining: An introduction to theory and some applications
Zanini, N and Dhawan, V. (2015). Text Mining: An introduction to theory and some applications. Research Matters: A Cambridge Assessment publication, 19, 38-44.
Recent technological advances have led to the availability of new types of observations and measurements that were previously not available and that have fuelled the 'Big Data' trend (Dhawan & Zanini, 2014). Along with standard structured forms of data (containing mainly numbers), modern databases include new forms of unstructured data comprising words, images, sounds and videos which require new techniques to be exploited and interpreted. This article focusses on Text Mining (TM): a set of statistical and computer science techniques specifically developed to analyse text data. It aims to give a theoretical introduction to TM and to provide some examples of its applications.
Download
-
Research News
Barden, K. (2015). Research News. Research Matters: A Cambridge Assessment publication, 19, 45-46.
A summary of recent conferences and seminars, and research articles published since the last issue of Research Matters.
Download
-
Statistical reports
Gill, T. (2015). Statistical Reports. Research Matters: A Cambridge Assessment publication, 19, 47.
The ongoing Statistics Reports Series provides statistical summaries of various aspects of the English examination system, such as trends in pupil uptake and attainment, qualifications choice, subject combinations and subject provision at school. This article contains a summary of the most recent additions to this series.
Download