Cambridge Assessment Research https://www.cambridgeassessment.org.uk:443/our-research/rss/index.aspx Browse the wealth of information offered from an organisation with a century and a half experience of assessment values a broad long term view of research that underpins and informs our thoughts, processes and products. en-gb Research Matters issue 37 - Research News <p>Bowett, L. (2024). Research News. <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37,</em> 84-86.</p> <p>A summary of recent conferences, reports, blogs and research articles published since the last issue of Research Matters.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-research-news.pdf">link Thu, 14 Mar 2024 09:25:48 GMT A Short History of the Centre for Evaluation and Monitoring (CEM) <p>Jellis, C. (2024). A short history of the Centre for Evaluation and Monitoring (CEM). <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 74-83. <a href="https://doi.org/10.17863/CAM.106033" target="_blank">https://doi.org/10.17863/CAM.106033</a></p> <p>The Centre for Evaluation and Monitoring (CEM), based in the North of England, recently celebrated its 40th birthday. Arising from an evaluation project at Newcastle University, and a subsequent move to Durham University, it rapidly grew in scope and influence, developing a series of highly regarded school assessments. For a relatively small organisation, its influence was seen across the world, resulting in outreach centres in New Zealand, Australia, and Hong Kong. Since being acquired by Cambridge University Press &amp; Assessment in 2019, it has established itself in a unique role within the wider Cambridge organisation due mainly to its development of computer adaptive assessments for use in schools. This article documents the rise of CEM, from its early successes to its adoption of new ideas in educational assessment and supporting technology until the present day. However, CEM’s development was not without its controversies, and these too make fascinating reading when set against the background of 40 years of ever changing educational policies.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-a-short-history-of-the-centre-for-evaluation-and-monitoring-cem.pdf">link Thu, 14 Mar 2024 09:09:24 GMT Exploring speededness in pre-reform GCSEs (2009 to 2016) <p>Walland, E. (2024). Exploring speededness in pre-reform GCSEs (2009 to 2016). <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 57-73. <a href="https://doi.org/10.17863/CAM.106035" target="_blank">https://doi.org/10.17863/CAM.106035</a></p> <p>GCSE examinations (taken by students aged 16 years in England) are not intended to be speeded (i.e. to be partly a test of how quickly students can answer questions). However, there has been little research exploring this. The aim of this research was to explore the speededness of past GCSE written examinations, using only the data from scored responses to items from a sample of 340 GCSE components. Speededness was calculated as the average (mean) percentage marks lost from the longest string of unanswered items at the end of each student’s examination paper. The potential impact of student ability on examination completion patterns was taken into account. The data suggested that most GCSEs analysed were unlikely to have been speeded. This method of exploring the speededness of exams using only scored responses has potential (although there are limitations), and it can flag potentially problematic components for further investigation.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-exploring-speededness-in-pre-reform-gcses-2009-2016.pdf">link Thu, 14 Mar 2024 09:03:49 GMT How do approaches to curriculum mapping affect comparability claims? An analysis of mathematics curriculum content across two educational jurisdictions <p>Rushton, N., Majewska, D., &amp; Shaw, S. (2024). How do approaches to curriculum mapping affect comparability claims? An analysis of mathematics curriculum content across two educational jurisdictions. <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 40-56. <a href="https://doi.org/10.17863/CAM.106032" target="_blank">https://doi.org/10.17863/CAM.106032</a></p> <p>Curriculum mapping is a comparability method that facilitates comparisons of content within multiple settings (usually multiple jurisdictions or specifications) and enables claims to be made about those curriculums/jurisdictions. Although curriculum maps have been published, there is little academic literature about the process of constructing and using them. Our study extends the literature by considering the different types of comparisons that can be made from curriculum maps: content coverage, placement, depth, and breadth. We also consider how these comparisons are affected by structural differences in the curricula or using a sub-set of the content.</p> <p>We use our mapping of mathematics in the US Common Core State Standards (CCSS) and the national curriculum in England to explore this. The CCSS for mathematical practice are common to all grades; we mapped these standards against the content for individual years in the national curriculum. The CCSS for mathematical content are set out by grade; we mapped a subset of this content to the national curriculum.</p> <p>Our mapping shows that it is possible to map curricula and make meaningful comparisons despite structural differences and content limitations. However, this affected the types of comparisons that we could carry out and the claims that we could make.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-how-do-approaches-to-curriculum-mapping-affect-comparability-claims.pdf">link Thu, 14 Mar 2024 08:56:40 GMT Does ChatGPT make the grade? <p>Brady, J., Kuvalja, M., Rodrigues, A., &amp; Hughes, S. (2024). Does ChatGPT make the grade? <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 24-39. <a href="https://doi.org/10.17863/CAM.106034" target="_blank">https://doi.org/10.17863/CAM.106034</a></p> <p>This study explores undergraduate students’ use of ChatGPT when writing essays. Three students were tasked with writing two essays each for a coursework component for a Cambridge qualification facilitated by access to ChatGPT. After writing the essays, they participated in semi-structured interviews about their experiences of using the technology. Researchers compared the transcript of the chatlog between the students and ChatGPT with the submitted essays. Analysis showed that the students relied on ChatGPT outputs to different extents, although they followed a similar process of engagement. The students shared their misgivings and points of appreciation for the technology.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-does-chatgpt-make-the-grade.pdf">link Thu, 14 Mar 2024 08:47:26 GMT Extended Reality (XR) in mathematics assessment: A pedagogical vision <p>Li, X. (2024). Extended Reality (XR) in mathematics assessment: A pedagogical vision. <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 6-23. <a href="https://doi.org/10.17863/CAM.106031" target="_blank">https://doi.org/10.17863/CAM.106031</a></p> <p>Extended reality (XR) – encompassing virtual reality (VR), augmented reality (AR), and mixed reality (MR) – emerges as a potential transformative tool in educational realms. This article explores the potential of XR in facilitating mathematics assessments; it proposes a list of mathematical topics that could be effectively mediated by XR’s immersive and interactive features. Additionally, it discusses some major challenges which could be barriers to the widespread adoption of XR in educational contexts and sets out a research agenda for further investigation.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-extended-reality-xr-in-mathematics-assessment-a-pedagogical-vision.pdf">link Mon, 11 Mar 2024 11:50:05 GMT Research Matters 37 - Foreword <p>Oates, T. (2024). Foreword. <em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 4.</p> <p>The explosion of activity around large language models since the release of ChatGPT in November 2022 has pushed some very important assessment issues aside. It is vital not to forget them. The principles and criteria for high quality assessment - whether low or high stakes in character - have been carefully accumulated through experience and research. It's extremely important to remember that innovation in assessment should rest on solid foundations.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-foreword.pdf">link Mon, 11 Mar 2024 11:30:59 GMT Research Matters 37 - Editorial <p>Bramley, T. (2024). Editorial.<em>Research Matters: A Cambridge University Press &amp; Assessment publication, 37</em>, 5.</p> <p>Our first article describes the technologies collectively known as "extended reality" and considers opportunities and challenges for using them in teaching and assessing mathematics. The second reports on a study where three undergraduates were asked to use ChatGPT to assist with writing essays and then interviewed about their approach. Our third article considers the difficult issues that arise when comparing curriculum documents with the aim of making claims about comparability of different curricula. Our fourth article explores the extent to which data (specifically whether a response was missing or not) can support inferences about whether students were under time pressure in paper-based GCSE examination components, and whether exams in some subjects were more "speeded" than others. Our final article presents a historical overview of the Centre for Evaluation and Monitoring (CEM), acquired by Cambridge in 2019 but now celebrating more than 40 years since its creation.</p> https://www.cambridgeassessment.org.uk/Images/research-matters-37-editorial.pdf">link Mon, 11 Mar 2024 11:17:38 GMT Research Matters 37 : Spring 2024 <ul class="rm-issues"> <li><strong>Foreword</strong> <span>Tim Oates</span></li> <li><strong>Editorial</strong> <span>Tom Bramley</span></li> <li><strong>Extended Reality (XR) in mathematics assessment: A pedagogical vision</strong><span>Xinyue Li</span></li> <li><strong>Does ChatGPT make the grade?</strong><span>Jude Brady, Martina Kuvalja, Alison Rodrigues, Sarah Hughes</span></li> <li><strong>How do approaches to curriculum mapping affect comparability claims? An analysis of mathematics curriculum content across two educational jurisdictions</strong><span>Nicky Rushton, Dominika Majewska, Stuart Shaw</span></li> <li><strong>Exploring speededness in pre-reform GCSEs (2009 to 2016)</strong><span>Emma Walland</span></li> <li><strong>A Short History of the Centre for Evaluation and Monitoring (CEM)</strong><span>Chris Jellis</span></li> <li><strong>Research News</strong><span>Lisa Bowett</span></li> </ul> https://www.cambridgeassessment.org.uk/Images/research-matters-37-spring-2024.pdf">link Mon, 11 Mar 2024 11:08:49 GMT Development challenges in challenging contexts: A 3-stage curriculum framework design approach for Education in Emergencies. <p class="xmsonormal" style="margin:0cm;background:white"><span style="font-size: 11pt; font-family: Arial, sans-serif; border: 1pt none windowtext; padding: 0cm;">Johnson, M., Fitzsimons, S., &amp; Coleman, V. (2024, March). Development challenges in challenging contexts: A 3-stage curriculum framework design approach for Education in Emergencies [Paper presentation]. Comparative and International Education Society, Miami, Florida, USA. https://link.springer.com/article/10.1007/s11125-022-09601-0</span></p> https://www.cambridgeassessment.org.uk/Images/709963-development-challenges-in-challenging-contexts-a-3-stage-curriculum-framework-design-approach-for-education-in-emergencies..pdf">link Tue, 05 Mar 2024 13:41:21 GMT Comparing curricula from different regions: a common practice revamped by using MAXQDA. <p class="xmsonormal" style="margin:0cm;background:white"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(28, 29, 30); border: 1pt none windowtext; padding: 0cm; background-image: initial; background-position: initial; background-size: initial; background-repeat: initial; background-attachment: initial; background-origin: initial; background-clip: initial;">Greatorex, J., &amp; Ireland, J. (2024, 28 Feb–1 Mar). Comparing curricula from different regions: a common practice revamped by using MAXQDA [Poster presentation]. MAXQDA International Conference, Berlin, Germany. https://www.maxqda.com/wp/wp-content/uploads/sites/2/JGJI_Indigenous-Knowledge.pdf</span></p> https://www.cambridgeassessment.org.uk/Images/709964-comparing-curricula-from-different-regions-a-common-practice-revamped-by-using-maxqda..pdf">link Thu, 29 Feb 2024 13:51:31 GMT Seeing what they think: Using MAXQDA to explore how concept maps work. <p class="xmsonormal" style="margin:0cm;background:white"><span style="font-size: 11pt; font-family: Arial, sans-serif; border: 1pt none windowtext; padding: 0cm;">Johnson, M. (2024, February). Seeing what they think: Using MAXQDA to explore how concept maps work [Poster presentation]. MAXQDA International Conference, Berlin, Germany. </span><span style="font-family: Arial, sans-serif; font-size: 11pt;">https://www.maxqda.com/wp/wp-content/uploads/sites/2/Martin-Johnson_Seeing-what-they-think.pdf</span></p> https://www.cambridgeassessment.org.uk/Images/709962-seeing-what-they-think-using-maxqda-to-explore-how-concept-maps-work..pdf">link Thu, 29 Feb 2024 13:27:54 GMT Progression from GCSE to A level, 2020–2022 Williamson, J., &amp; Vidal Rodeiro, C.L. (2024). Progression from GCSE to A level, 2020–2022. Statistics Report Series No. 139 https://www.cambridgeassessment.org.uk/Images/707601-progression-from-gcse-to-a-level-2020-2022.pdf">link Thu, 01 Feb 2024 12:00:00 GMT Progression from GCSE to A level, 2020–2022 - Data Williamson, J., &amp; Vidal Rodeiro, C.L. (2024). Progression from GCSE to A level, 2020–2022. Statistics Report Series No. 139 - Data https://www.cambridgeassessment.org.uk/Images/707602-progression-from-gcse-to-a-level-2020-2022-data.xlsx">link Thu, 01 Feb 2024 12:00:00 GMT Cognitive Diagnostic Models and how they can be useful Williamson, J. 2023. <em>Cognitive Diagnostic Models and how they can be useful</em>. Cambridge University Press &amp; Assessment. https://www.cambridgeassessment.org.uk/Images/701443-cognitive-diagnostic-models-and-how-they-can-be-useful.pdf">link Thu, 07 Dec 2023 13:49:09 GMT Multiple marking using the Levels-only method for A level English Literature Ireland, J., &amp; de Groot, E. (2023, November 1-4). <em>Multiple marking using the Levels-only method for A level English Literature</em> [Paper presentation]. Annual conference of the Association for Educational Assessment – Europe (AEA-Europe), Malta. <a href="http://2023.aea-europe.net/">https://2023.aea-europe.net/</a> https://www.cambridgeassessment.org.uk/Images/699398-multiple-marking-using-the-levels-only-method-for-a-level-english-literature.pdf">link Tue, 21 Nov 2023 12:27:13 GMT Uptake of GCE A level subjects 2022 Lim, C.H.J. &amp; Gill, T. (2023). Uptake of GCE A level subjects 2022. Statistics Report Series No. 138 https://www.cambridgeassessment.org.uk/Images/705290-uptake-of-gce-a-level-subjects-2022.pdf">link Fri, 17 Nov 2023 17:35:00 GMT Uptake of GCE A level subjects 2022 - Data Lim, C.H.J. &amp; Gill, T. (2023). Uptake of GCE A level subjects 2022. Statistics Report Series No. 138 - Data https://www.cambridgeassessment.org.uk/Images/705291-uptake-of-gce-a-level-subjects-2022-data.xlsx">link Fri, 17 Nov 2023 17:32:41 GMT Provision of GCSE subjects 2022 Lim, C.H.J. &amp; Gill, T. (2023). Provision of GCSE subjects 2022. Statistics Report Series No. 137 https://www.cambridgeassessment.org.uk/Images/705286-provision-of-gcse-subjects-2022.pdf">link Fri, 17 Nov 2023 17:28:12 GMT Provision of GCSE subjects 2022 - Data Lim, C.H.J. &amp; Gill, T. (2023). Provision of GCSE subjects 2022. Statistics Report Series No. 137 - Data https://www.cambridgeassessment.org.uk/Images/705287-provision-of-gcse-subjects-2022-data.xlsx">link Fri, 17 Nov 2023 17:24:59 GMT