Computer adaptive testing (CAT) is a form of assessment that adjusts question difficulty to match each test taker’s ability, creating more accurate and personalised evaluations.
In this blog, Christopher Hubbard, Assessment Group Manager at Cambridge English, explores the key principles and benefits of computer adaptive testing (CAT) and how his upcoming workshop series will help those further understand this adaptive approach.
What is computer adaptive testing?
In brief, computer adaptive tests use a selection algorithm to tailor test content to test taker ability. The content selection algorithm is designed to adjust the level of item difficulty each examinee receives, based on the correctness of the item responses given earlier in the test. The test administration algorithm seeks to administer items that are well-matched to the examinee’s ability.
How do computer adaptive tests work?
A computer adaptive test uses the interaction between a content selection algorithm and a pool of available content to deliver test events to test takers. The content selection algorithm is designed to adjust the level of item difficulty each test taker receives, based on the correctness of the item responses given earlier in the test. The overall aim is to administer items that are well-matched to an individual test taker’s ability.
What are the advantages of computer adaptive testing?
Computer adaptive tests generally allow the testing of a wide range of abilities from the same pool of content. They also often allow test setters to reach similar levels of reliability of result reporting across ability levels, using fewer items per test taker than would be usual in a fixed format test.
When should you use computer adaptive testing?
For me, the question is more about understanding the testing context, and the required outcomes to be reported; the general ‘what, who and why we are testing’ in any assessment context.
From here, by understanding more about the practicalities and options of computer adaptive tests, it is then easier to decide if computer adaptive tests provide the ‘how’ of the given context.
Is artificial intelligence (AI) impacting the practice of computer adaptive testing?
To some degree computer adaptive tests could be described as being based on a basic AI model given the levels of complexity that can be built into the selection algorithm.
In addition, there are also ongoing AI developments and research projects that can impact and enhance the scope and effectiveness of future computer adaptive testing.
Examples of this include:
- Developments in the automarking of short and extended response, in both text and speech which support the expansion of available task types and so enrich the construct possibilities. AI can also introduce repeatable objective scoring to the process.
- Expanding IRT (Item Response Theory) models to include aspects such as e.g. response times, and interaction patterns.
Why should assessment practitioners attend your upcoming workshop series?
Across the three weeks we’ll be working through the basics of what a computer adaptive test is, and the features available to the test administrator that allows an adaptive test to be configured to suit different testing contexts.
We’ll also talk about what practical considerations need to be discussed and their impacts in order to implement an adaptive test.
Whether you are already using adaptive testing, considering if it suits your testing context, or simply curious about what adaptive tests are, this workshop series aims to offer something valuable for everyone."
Ready to deepen your understanding of computer adaptive testing? Join Christopher's upcoming workshop series on CAT.
The Assessment Network is part of Cambridge University Press & Assessment, providing professional development for impactful assessment practices.