Sylvia Vitello

Sylvia Vitello

Sylvia Vitello

Here at Cambridge University Press and Assessment, I conduct research into various areas of education. I focus primarily on exploring questions about qualifications and assessments, especially ones that are used by secondary school learners. I use a range of quantitative and qualitative methods such as systematic literature reviews, document analysis, interviews, questionnaires etc – it all depends on the nature of the research question.

I came to Cambridge University Press and Assessment from a psychology background, having completed a BSc in Psychology and a PhD in Experimental Psychology, both from University College London. My PhD focused on the cognitive neuroscience of language processing.

When I’m not working, I enjoy spending as much time as possible with family and friends.

Publications

2024

Moderation of non-exam assessments: a novel approach using comparative judgement.

Chambers, L., Vitello, S., & Vidal Rodeiro, C. (2024). Moderation of non-exam assessments: a novel approach using comparative judgement. Assessment in Education: Principles, Policy & Practice.

2023

An example of redeveloping checklists to support assessors who check draft exam papers for errors

Vitello, S., Crisp, V., & Ireland, J. (2023). An example of redeveloping checklists to support assessors who check draft exam papers for errors Research Matters: A Cambridge University Press & Assessment publication, 36, 46-58. https://doi.org/10.17863/CAM.101744

Assessment materials must be checked for errors before they are presented to candidates. Any errors have the potential to reduce validity. For example, in the most extreme cases, an error may turn an otherwise well-designed exam question into one that is impossible to answer. In Cambridge University Press & Assessment, assessment materials are checked by multiple assessment specialists across different stages during assessment development. While human checkers are critical to this process, we must acknowledge that there is ample research showing the shortcomings of being human (e.g., we have cognitive biases, and memory and attentional limitations). It is important to provide assessment checkers with tools that help overcome or mitigate these limitations.

This article is about one type of checking tool – checklists. We describe a research-informed, collaborative project to support assessors in performing their checks of exam papers. This project focused on redesigning the instructional, training and task materials provided to assessors. A key part of this was to design checklists for assessors to use when performing their checks. In this article, we focus primarily on the approach that we took for these checklists in order to draw readers’ attention to the complexity that is involved in designing them and to provide a practical example of how research can be used strategically to inform key design decisions.

Research Matters 36: Autumn 2023
  • Foreword Tim Oates
  • Editorial Tom Bramley
  • The prevalence and relevance of Natural History assessments in the school curriculum, 1858–2000: a study of the Assessment ArchivesGillian Cooke
  • The impact of GCSE maths reform on progression to mathematics post-16Carmen Vidal Rodeiro, Joanna Williamson
  • An example of redeveloping checklists to support assessors who check draft exam papers for errorsSylvia Vitello, Victoria Crisp, Jo Ireland
  • An analysis of the relationship between Secondary Checkpoint and IGCSE resultsTim Gill
  • Synchronous hybrid teaching: how easy is it for schools to implement?Filio Constantinou
  • Research NewsLisa Bowett

2022

What do we know about the evidence sources teachers used to determine 2021 Teacher Assessed Grades?
Vitello, S. & Leech, T. (2022). What do we know about the evidence sources teachers used to determine 2021 Teacher Assessed Grades? Cambridge University Press & Assessment.
What impacts success in proofreading? A literature review of proofreading on screen vs on paper.
Vitello, S. (2022). What impacts success in proofreading? A literature review of proofreading on screen vs on paper. Cambridge University Press & Assessment.
What impacts success in proofreading? A literature review of text feature effects.
Mouthaan, M., & Vitello, S. (2022). What impacts success in proofreading? A literature review of text feature effects. Cambridge University Press & Assessment.
Reflections on teacher assessment after the 2021 Teacher Assessed Grades process in England.
Vitello, S. & Leech, T. (2022, 10 November). Reflections on teacher assessment after the 2021 Teacher Assessed Grades process in England. Open paper presentation, 23rd Annual Meeting of the Association for Educational Assessment – Europe, Dublin, Ireland.

2021

Progression to post-16 education: the role of Vocational Qualifications
Vidal Rodeiro, C.L and Vitello, S. (2021, September 13 - 16). Progression to post-16 education: the role of Vocational Qualifications. [Paper presentation]. Annual conference of the British Educational Research Association, online. Watch the video
What is competence? A shared interpretation of competence to support teaching, learning and assessment
Vitello, S., Greatorex, J., & Shaw, S. 2021. What is competence? A shared interpretation of competence to support teaching, learning and assessment. Cambridge University Press & Assessment Research Report. Cambridge, UK: Cambridge University Press & Assessment
What do we mean by question paper error? An analysis of criteria and working definitions

Rushton, N., Vitello, S., Suto, I. (2021). What do we mean by question paper error? An analysis of criteria and working definitions. Research Matters: A Cambridge University Press & Assessment publication, 32, 67-81.

It is important to define what an error in a question paper is so that there is a common understanding and to avoid people’s own conceptions impacting upon the way in which they write or check question papers. We carried out an interview study to investigate our colleagues’ definitions of error. We found that there is no single accepted definition of a question paper error. There were three interacting aspects that participants considered when deciding whether a problem was an error: the manifestation of the error, the (potential) impact upon candidates, and the stage at which it was discovered.

Research Matters 32: Autumn 2021
  • Foreword Tim Oates
  • Editorial Tom Bramley
  • Learning during lockdown: How socially interactive were secondary school students in England? Joanna Williamson, Irenka Suto, John Little, Chris Jellis, Matthew Carroll
  • How well do we understand wellbeing? Teachers’ experiences in an extraordinary educational era Chris Jellis, Joanna Williamson, Irenka Suto
  • What do we mean by question paper error? An analysis of criteria and working definitions Nicky Rushton, Sylvia Vitello, Irenka Suto
  • Item response theory, computer adaptive testing and the risk of self-deception Tom Benton
  • Research News Anouk Peigne
Progression to post-16 education in England: the role of vocational qualifications

Vidal Rodeiro, C.L. and Vitello, S. (2021). Progression to post-16 education in England: the role of vocational qualifications. Research Papers in Education (ahead of print). 

2020

A way of using taxonomies to demonstrate that applied qualifications and curricula cover multiple domains of knowledge

Suto, I., Greatorex, J., Vitello, S., and Child, S. (2020). A way of using taxonomies to demonstrate that applied qualifications and curricula cover multiple domains of knowledge. Research Matters: A Cambridge Assessment publication, 30, 26-34.

Educational taxonomies are classification schemes which provide the terminology that educationalists need to describe and work with different areas of knowledge. It is good practice to use taxonomies to formulate and review curricula, learning objectives, and associated assessments. Demonstrating sufficient coverage of each of an adequate range of knowledge domains is critical for authenticity, for assessment reliability, and for transparency surrounding what students are learning.

In this study we explored whether any educational taxonomies that were designed for general educational contexts (sometimes called ‘academic’ contexts) could be utilised in applied educational contexts (often called ‘vocationally-related’ in England) . To do this, we identified nine published taxonomies with sufficient potential, and selected and combined the most appropriate. This process led us to develop a new model of demand.  We then applied the selected taxonomies experimentally to existing curricula in a range of applied subjects which are taught at secondary and tertiary level in England. We also used the selected taxonomies to develop a tool for writing educational objectives. This article ends with suggestions for applying the selected taxonomies in other areas of assessment.

Research Matters 30: Autumn 2020
  • Foreword Tim Oates, CBE
  • Editorial Tom Bramley
  • A New Cambridge Assessment Archive Collection Exploring Cambridge English Exams in Germany and England in JPLO Gillian Cooke
  • Perspectives on curriculum design: comparing the spiral and the network models Jo Ireland, Melissa Mouthaan
  • Context matters—Adaptation guidance for developing a local curriculum from an international curriculum framework Sinead Fitszimons, Victoria Coleman, Jackie Greatorex, Hiba Salem, Martin Johnson
  • Setting and reviewing questions on-screen: issues and challenges Victoria Crisp, Stuart Shaw
  • A way of using taxonomies to demonstrate that applied qualifcations and curricula cover multiple domains of knowledge Irenka Suto, Jackie Greatorex, Sylvia Vitello, Simon Child
  • Research News Anouk Peigne
Vocational Qualifications at Key Stage 4 and Key Stage 5: who takes them and how they fit into students' programmes of study

Vidal Rodeiro, C. L., & Vitello, S. (2020). Vocational Qualifications at Key Stage 4 and Key Stage 5: who takes them and how they fit into students' programmes of study. Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.

2019

Moderation of non-exam assessments: a novel approach using comparative judgement
Chambers, L., Vitello, S. and Vidal Rodeiro, C.L. (2019). Moderation of non-exam assessments: a novel approach using comparative judgement. Presented at the 20th annual AEA-Europe conference, Lisbon, Portugal, 13-16 November 2019.
How do vocational qualifications fit into students' programmes of study following government reforms to 14-18 education?
Vitello, S. and Vidal Rodeiro, C.L. (2019). How do vocational qualifications fit into students' programmes of study following government reforms to 14-18 education? Presented at the Journal of Vocational Education and Training International Conference, University of Oxford, UK, 28-30 June 2019.
The effect of adaptivity on the reliability coefficient in adaptive comparative judgement.

Bramley, T. and Vitello, S. (2019). The effect of adaptivity on the reliability coefficient in adaptive comparative judgement.  Assessment in Education: Principles, Policy and Practice, 26(1), 43-58.

2018

AS level trends 2018
Williamson, J. and Vitello, S. (2018). Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.
Reformed A level results: Do candidates who take the AS level achieve better grades?
Williamson, J. and Vitello, S. (2018). Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.
Effects of AS/A level reforms in England: extra insights from the National Pupil Database
Vitello, S. and Williamson, J. (2018). Effects of AS/A level reforms in England: extra insights from the National Pupil Database. Presented at the 44th conference of the International Association for Educational Assessment, Oxford, UK, 9-14 September 2018.
Faster, higher, stronger? A comparison of content coverage and teacher pedagogy in Technical Awards and GCSEs in sport-related subjects
Child, S. and Vitello, S. (2018). Faster, higher, stronger? A comparison of content coverage and teacher pedagogy in Technical Awards and GCSEs in sport-related subjects. Presented at the annual conference of the British Educational Research Association, Northumbria University, Newcastle, UK, 11-13 September 2018.
Vocational qualifications for 14-16 year olds: Exploration of knowledge, skills and teacher perceptions
Vitello. S. and Child, S. (2018). Vocational qualifications for 14-16 year olds: Exploration of knowledge, skills and teacher perceptions. Presented at the 5th International Conference on Employer Engagement in Education and Training, London, UK, 5-6 July 2018.
AS level trends 2017

Vitello, S. and Williamson, J. (2018). Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.

Which tier? Effects of linear assessment and student characteristics on GCSE entry decisions
Vitello, S. and Crawford, C. (2018). Which tier? Effects of linear assessment and student characteristics on GCSE entry decisions. British Educational Research Journal, 44(1), 94-118.

2017

The effect of adaptivity on the reliability coefficient in comparative judgement
Vitello, S. and Bramley, T. (2017). Presented at the annual conference of the Association for Educational Assessment - Europe, Prague, 9-11 November 2017.
Internal versus external assessment in vocational qualifications: a commentary on the government’s reforms in England
Vitello, S. and Williamson, J. (2017). Internal versus external assessment in vocational qualifications: a commentary on the government’s reforms in England.  London Review of Education, 15(3), 536-548.
Foundation or higher tier? Effects of moving from a modular to linear system of GCSE assessment
Vitello, S. and Crawford, C. (2017). Presented at the annual conference of the British Educational Research Association, University of Sussex, Brighton, UK, 5-7 September 2017.

2016

Research Matters Special Issue 4: Aspects of Writing 1980-2014
  • Variations in aspects of writing in 16+ English examinations between 1980 and 2014 Gill Elliott, Sylvia Green, Filio Constantinou, Sylvia Vitello, Lucy Chambers, Nicky Rushton, Jo Ireland, Jessica Bowyer, David Beauchamp
Employers' views on assessment design in vocational qualifications: a preliminary study
Vitello, S., Carroll, P., Greatorex, J. and Ireland, J. (2016). Paper presented at the European Conference on Educational Research (ECER), Dublin, Ireland, 23-26 August 2016.

2015

Gender differences in GCSE
Bramley, T., Vidal Rodeiro, C.L. and Vitello, S. (2015) Cambridge Assessment Research Report. Cambridge, UK: Cambridge Assessment.
Linking instructional verbs from assessment criteria to mode of assessment
Greatorex, J., Ireland, J., Carroll, P. and Vitello. S. (2015) Paper presented at the Journal for Vocational Educational and Training (JVET) conference, Oxford, 3-5 July 2015.

Research Matters

Research Matters 32 promo image

Research Matters is our free biannual publication which allows us to share our assessment research, in a range of fields, with the wider assessment community.