In a world where technology is evolving at break-neck speed, two game-changing companies came together to push the boundaries even further. As soon as AKAD met Cirrus, they put their heads together and embarked on a journey to use Cirrus’ blueprint technology in a visionary way.
AKAD required an e-assessment platform for formative (practice) and summative exams that were both randomised and equally challenging in terms of the subject area and length for its 11,500+ students. In addition, they required the possibility to deliver exams on-demand, and provide personalised feedback and learning guidance during formative exams.
In their quest to provide Germany’s most flexible education system, AKAD and Cirrus joined forces to provide randomised “Anytime Assessments” using Cirrus’ Blueprint function.
AKAD is currently seeing the following significant wins:
- 93% of AKAD students now take exams online by choice with Cirrus. As per German Law, the option to take them in-person is still a possibility.
- AKAD’s fail-rate has decreased by 50% across the board, without affecting average grades.
- AKAD uses these grades to maintain consistent exam difficulty.
- Randomised, on-the-fly “Anytime Exams” can be taken by students whenever is convenient for them.
- The introduction of practice options and exam simulations has lowered exam anxiety and contributed to the decrease in failure rates.
- AKAD won the e-Learning Award 2022 in the ‘Assessment’ category for their innovative exams using Proctorio and Cirrus.
- 8.6/10 student satisfaction rate overall.
Background
Founded in 1959, AKAD has always found innovative ways to push the boundaries of how education works in Germany. In an increasingly hectic world where balancing work, family, and education is challenging, AKAD recognised the need to provide remote education and testing and is the first state-recognised private university for applied sciences designed for working people.
Challenge
AKAD University aims to provide its students with a flexible, individualised distance learning and examination experience. The institution has over 90 degree programs and strives to cater to a vast demographic, including individuals with full-time employment, families, those that reside overseas, or have physical disabilities that restrict them from attending a conventional university. AKAD’s study program is 100% online, has no semesters and can be started at any time, and exams are completely flexible – the student decides when to take them.
AKAD’s looming challenge was to provide its 12,000+ students with exceptional online exam preparation and the flexibility to take exams from home at their convenience. Despite AKAD’s students predominantly learning online, they sought to leverage technology to create a more connected experience than what traditional lecture halls can offer.
To prepare students for their exams, AKAD utilises the same pool of assignments for the actual exams, along with small practice sessions and simulated exams. These formative exams provide students with constructive feedback not only on what they did right or wrong, but also why. Furthermore, learning recommendations are given, highlighting areas they should focus on to gain a more in-depth understanding of the learning material.
In compliance with German law, AKAD required an extensive collection of questions for formative practice and summative exams that were both randomised and equally challenging in terms of subject area and length. So a tool had to be found to solve these requirements.
To summarise, AKAD faced a significant challenge in creating randomised exams of identical difficulty for its 11,500 students, to be taken at different times, with personalised feedback and learning guidance during formative exams. Despite the challenge, AKAD was confident it would find a platform to overcome these difficulties.
“Without the assessment software from Cirrus, we would not have been able to achieve our goals in such a short time. Especially for exams, it is important to use highly secure technology. Our students expect perfect service and university degrees with integrity.s in South Africa, with over 50,000 students.”
Norman Grötzbach, CTO of AKAD and Director of Digital Services at Galileo Global Education.
The hunt for a solution: AKAD’s needs
AKAD recognised their challenges and knew they had some hurdles to overcome in finding the right e-assessment platform.
First, they scouted the market for a standard solution to meet their needs. The most important feature was a matrix/blueprint solution that could be used to define how the content of the same taxonomy level could be randomised from defined subject areas. But AKAD could not find a software solution that met this requirement in Germany or the international market.
They then considered developing the technology themselves. Development was already in full swing when AKAD became aware of Cirrus’ solution. It didn’t take long for AKAD and Cirrus to realise that they could be more than just a supplier and client. AKAD and Cirrus operate as partners, continuously collaborating on the Blueprint technology and optimising it to meet all of AKAD’s needs.
AKAD identified that several of the functions they required were already present in the Cirrus solution, and any missing requirements were developed collaboratively between AKAD and the Cirrus team. This cooperative approach enabled AKAD to improve the already impressive Cirrus software solution and customise it to meet their specific needs. Key features included quality management processes, secure exam views, feedback functions, and exam archiving.
Solution
AKAD trials Cirrus’ solution in a first-of-its-kind pilot
AKAD initiated the project with five distinct exam types in fields such as Translation, Economics, Mathematics, Engineering, and Marketing. The primary focus was to devise tasks that could be automatically evaluated, provide qualitative feedback, test fundamental knowledge, and correspond to Bloom’s highest taxonomy levels. Throughout this process, AKAD provided close supervision and support to their exam authors. In conjunction, AKAD and Cirrus developed the Blueprint model, grading system, and authorisation matrix for admins and authors.
After the first five exams, an evaluation period followed, providing numerous lessons learned. AKAD fine-tuned the difficulty levels, answering times, and repetition cycles, discovering that a two-hour exam duration can be reduced for tasks that require less writing. The range of exams expanded gradually, especially in mathematics, where exams were adapted to meet online exam requirements.
Presently, AKAD offers more than 500 online exams with over 70,000 tasks. Remarkably, 92% of students take their exams online, and students use the preparation resources over 500,000 times annually.
“Our students expect perfect service and university degrees with integrity. In Cirrus, we have found a reliable partner with whom we can offer our examinations at the highest level around the clock, every day of the year.”
Norman Grötzbach, CTO of AKAD and Director of Digital Services at Galileo Global Education.
Cirrus’ blueprint technology
AKAD arrived at the decision to choose Cirrus because of their Blueprint technology, which allows users to automatically create balanced assessments based on a set of criteria they select. Cirrus Blueprint has several ways to select and classify items, such as taxonomies, learning objectives, and topics. In the Cirrus platform, topics are highly flexible and customisable. AKAD could use topics to define item difficulty, thus creating a blueprint with difficulty levels and credit points built into it.
At AKAD, exams are crucial as they determine the credit points that students need to accumulate to complete a module successfully. Exam credit point values range from 45 to 240, with a specific minimum number of points required to pass the module. A time calculation has been determined for each item that equals points; the more time required for an item, the more credit points a candidate earns. For example, 1 minute to answer a question equals 1 credit point, and these questions would therefore also be classified as ‘easy’ questions. 5 points for a question that averages a 5-minute response time would be considered a ‘medium difficulty’ item, and so on. Taxonomies are also factored into the equation of how difficult an item is. Basically, AKAD, combined with Cirrus blueprint technology, has determined an equation for difficulty rating: taxonomy + minutes to answer = difficulty and credit points. In addition, Cirrus will commence work in 2023 on an algorithm that will make it even easier to deliver randomised, psychometrically balanced exams in a Eurostars-funded project.
AKAD has leveraged Cirrus technology in an unparalleled way by utilising blueprint technology for creating fair and legally defensible exams automatically. This technology enables AKAD to generate completely randomised exams on demand, giving students the flexibility to take their tests from anywhere around the world.
Implementation
As always, implementing significant changes to teaching methods will always come with resistance, from sceptics who cling to traditional practices and a “we’ve always done it this way” attitude to legal frameworks and data protection and the monumental task of getting students and teachers fully on board.
AKAD recognised these obstacles and proactively engaged students and teachers throughout the implementation process, gathering targeted feedback through extensive surveys and establishing exchange groups to address specific issues.
AKAD succeeded – particularly with mathematics exams – in developing tasks that effectively utilised the Cirrus platform, ensuring exams retained their quality and ambition when mapped online. These successes and the benefits of online exam preparation convinced even the staunchest of sceptics and procrastinators of the value of AKAD’s approach.
AKAD has developed an effective process for optimising online exams, including a quality assurance program. The authors are guided through the first 30 exams to ensure questions, taxonomy levels, and feedback are on point. Before the exams go live, a formal and technical quality check is performed, and three months after launch, the exams are reviewed again. Various factors are taken into account, such as grade, failure rate, and student feedback, to identify areas for improvement. Students can also provide feedback on individual questions. This showcases the strong partnership between AKAD and Cirrus, who share a common vision for online assessments.
The student satisfaction ratings speak volumes, with students giving AKAD’s online exams an impressive 8.6 out of 10. Furthermore, with a rating of 9.9, students report that they can study more flexibly thanks to this innovative exam format. AKAD’s forward-thinking approach has contributed significantly to the learning experience and opened doors for more accessible and personalised education.
Results
Delivering assessments the scalable way
AKAD’s innovative approach to online exam delivery and automation through its partnership with Cirrus has paved the way for scaling and growth. With automatic grading and parallel process automation, exams begin immediately, markers are alerted, and grading is streamlined, making the solution scalable and manageable with minimal staff. AKAD’s 70,000+ tasks across 500+ exams and continuously increasing student enrollment is largely due to the success of their online exams and exam prep, helping transform education to be more accessible, personalised, and scalable.
AKAD has reaped numerous benefits.
- 93% of AKAD students now take exams online by choice with Cirrus, as per German Law, the option to take them in person is a possibility.
- AKAD’s fail-rate has decreased by 50% without affecting average grades. Some subjects have a 30% fail-rate while others have a 10% fail-rate, and AKAD was able to cut all across the board by 50%.
- AKAD uses these grades to maintain consistent exam difficulty.
- The introduction of practice options and exam simulations has lowered exam anxiety and contributed to the decrease in failure rates.
- AKAD won the e-Learning Award 2022 in ‘Assessment’ category for their innovative exams using Proctorio and Cirrus.
- 8.6/10 student satisfaction rate overall.
“With Cirrus as our partner, we can continuously improve our online exam offerings and are without a doubt trendsetters in the student and learning experience!“
Norman Grötzbach, CTO of AKAD and Director of Digital Services at Galileo Global Education.
