You may have been hearing the buzz about LOFT 3.0. What it is, and how to create test blueprints. We’ve even given a recent webinar on the subject. Talk is great, but can e-assessment walk the talk? Has LOFT 3.0 actually been used in “real life” with an organisation, or is it all pipe dreams and fantasy future tech?
You’ll be pleased to know that Linear-On-the-Fly Tests are alive and well! In this blog, we’ll be using Cirrus’ latest case study as a real-life example of how LOFT 3.0 is being used. AKAD University in Stuttgart, Germany, uses LOFT 3.0 to deliver round-the-clock “Anytime Exams” with LOFT 3.0.
Who is AKAD University?
In today’s fast-paced world, technology is constantly evolving and reshaping how we live and work. This is especially true in the realm of education, where institutions are continually seeking new and innovative ways to deliver knowledge and skills to students. One such institution is AKAD, a state-recognised private university for applied sciences in Germany.
Founded in 1959, AKAD has always been at the forefront of pushing the boundaries of how education works in the country. To further this goal, AKAD recently partnered with Cirrus, a leading provider of e-assessment technology. Together, they set out to revolutionise how online exams are conducted, introducing a new level of flexibility and convenience for students.
The result of this partnership is a cutting-edge assessment platform that offers randomised “Anytime Assessments” using Cirrus’ test Blueprint function, allowing students to take exams at any time from anywhere in the world.
Let’s explore AKAD’s challenge and how they use the test blueprint function to create LOFT exams.
A peculiar challenge
AKAD University is a modern institution that understands the challenges of balancing work, family, and education in today’s fast-paced world. They recognised the need for a flexible online learning experience that can accommodate individuals with busy schedules or those who live far away from a physical campus.
To provide students with a comprehensive exam preparation experience, AKAD utilises a combination of approaches. First, they use the same pool of assignments for practice sessions and actual exams, ensuring students are well-prepared for the real thing. Second, they offer small practice sessions and simulated exams that provide valuable feedback on areas of strength and weakness. AKAD’s focus on formative exams and constructive feedback not only helps students to understand what they did right or wrong but also why, which can be a crucial factor in promoting deeper learning.
However, complying with German law presented a significant challenge for AKAD when it came to exams. AKAD needed an extensive collection of questions for both formative practice and summative exams that were randomised and equally challenging in terms of subject area and length.
To summarise it best: AKAD faced a significant challenge in creating randomised summative exams of identical difficulty for its 11,500 students, to be taken at different times, with personalised feedback and learning guidance during formative exams.
A practical guide to test blueprinting and LOFT 3.0
Develop a large item bank
Developing a large, well-tagged item bank is the first step to developing a robust test blueprint. By creating a comprehensive database of exam items, instructors can save time and effort while delivering virtually unlimited, on-demand exams to their candidates – the bigger the item bank, the less chance of repeating questions.
Building an item bank involves categorising questions by topic, difficulty level, and other relevant factors. This allows for easy organisation and retrieval of exam questions, as well as the ability to create randomised exams that draw from a pool of questions. Additionally, the use of technology can enhance security measures by allowing for randomised exam order and preventing cheating through proctoring tools. By implementing a well-designed item bank, instructors can streamline their exam creation process and provide students with a secure and randomised testing experience.
AKAD, for example, developed a pool of over 70,000 well-tagged items from which the blueprint can draw to create a virtually unlimited number of random tests.
Create a test blueprint
By using a test blueprint, instructors can ensure that the exam questions are distributed evenly among the topics and difficulty levels. This helps create a well-balanced exam that accurately measures the candidate’s knowledge and provides a fair assessment.
At AKAD, they used the topic feature in Cirrus’ test blueprint technology to define item difficulty, thus creating a blueprint with built-in credit points and difficulty levels.
To determine the difficulty level of each exam item, a time calculation has been assigned to each question that equals credit points; the more time required for an item, the more credit points a candidate earns, and then using the topics function, the higher the minutes, the more difficult an item.
For example, 1 minute to answer a question equals 1 credit point, and these questions would therefore also be classified as ‘easy’ questions. 5 points for a question that averages a 5-minute response time would be considered a ‘medium difficulty’ item, and so on. Taxonomies are also factored into the equation of how difficult an item is.
The result of this unique combination of AKAD’s methodology and Cirrus test blueprint technology is an equation for difficulty rating: taxonomy + minutes to answer = difficulty and credit points. This solution ensures that exams are fair and legally defensible while also providing students with the flexibility to take their tests from anywhere in the world.
This is how AKAD has chosen to use the topic functionality, but in Cirrus it’s quite a flexible feature, allowing users to tag items however suits their organisation. In addition, Cirrus will commence work in 2023 on an algorithm that will make it even easier to deliver randomised, psychometrically balanced exams in a Eurostars-funded project.
Select the LOFT function
After the large item bank is created, organised and tagged according to the test blueprint, Cirrus can then randomly select the appropriate number of questions from each category in the blueprint to create a unique exam for each student. This means that AKAD’s students can sit their exams when and where it suits them. The randomised exam order, combined with the random selection of items, ensures that each student receives a unique exam experience, thereby reducing the risk of cheating and improving the overall integrity of the assessment.
Conclusion
AKAD’s partnership with Cirrus has revolutionised education by providing a scalable and manageable solution for online exam delivery and automation. With impressive statistics, including a 50% reduction in fail-rates and a 93% adoption rate for online exams, AKAD’s approach makes education more accessible, personalised, and scalable. The introduction of practice options and exam simulations has contributed to this success, reducing exam anxiety and further lowering failure rates. Overall, AKAD’s innovative approach is transforming education and paving the way for the future.
Keen to learn more about LOFT 3.0? Check out our other resources on the topic:
