better promote test-taker integrity and continuously monitor and improve the quality of your item bank. We’ve also made multiple choice question scoring much more fl
What’s new in the Cirrus platform in March 2023? We’ve been working hard to help you better promote test-taker integrity, make your multiple choice questions more accurate in testing knowledge and continuously monitor and improve the quality of your item bank. Read on to find out what’s new!
Improved plagiarism detection
In Q4 2022, we introduced the Turnitin integration, giving you even more options to promote test-taker integrity. To recap, Turnitin offers fully automated plagiarism detection: It ensures the originality of written work by checking written exam answers against Turnitin’s database of scholarly publications and web content. Significantly, Turnitin has also been working on expanding the tool with AI-detection capabilities that will help detect test-takers use of an AI tool like ChatGPT.
We introduced this integration last year with support for essay questions and have upgraded it in the past month: Turnitin is now available for Office 365 questions as well. For more information on how it works and how to set it up, please refer to the relevant Knowledge Base article. Note that this integration needs to be enabled by the Customer Service Team for your organisation. If you’re interested in taking advantage of the enhanced plagiarism in Cirrus, please contact us.
More accurate MCQ scoring
In addition, we’ve added an improvement that makes the Multiple Choice question type much more nuanced and accurate in assessing test-takers’ knowledge: You can now set a custom score for each answer alternative where before, you could only mark one alternative as the right answer and award points for that answer.
The possibility of assigning different scores to each alternative, allows you to differentiate between different levels of knowledge or partial understanding of the topic. Furthermore, different scores per alternative can also help to identify misconceptions or common errors in understanding a topic. By analysing the scores for each option, you can gain insight into the areas where the respondents may be struggling and provide targeted feedback or adjust the teaching approach accordingly. It greatly expands the potential usage and accuracy of the MC question type and is available for all our users.
Continuously monitor and improve your item bank
Despite best intentions and extensive quality control, it is always possible for questions with issues to slip through the net. Issues may range from minor typos to major issues like the question being unsolvable with the provided information. In both cases, you’ll wish to fix the issue as soon as possible. Our new Candidate Feedback option will help you continuously improve your test items.
For these situations, we’ve now added the option for test-takers to leave feedback on specific questions during exams. The option is available to all Cirrus users, but please note that it must be enabled by your admin first.
When enabled, candidates can choose to give feedback, select one of the options and add additional free-text descriptions:
- Not solvable question
- Spelling/typo
- Feedback/Hint is Wrong
- Display problem
To minimise your administrative burden, all feedback is collected in the Content Management section. Here, you can share feedback with item creators and/or change the status when the issue is under scrutiny, resolved, etc.
For more information about the feature, please refer to our Knowledge Base:
What’s next for Cirrus?
In these past few months, we’ve expanded Cirrus’ plagiarism detection capabilities, increased the flexibility of Multiple Choice Question scoring and made it easier for you to continuously improve your item bank.
So what’s next? We’ve got ambitious goals for the coming months! Future changes to Cirrus range from major improvements to library management, psychometric data and more.
Improved library management
It may not sound like the most exciting of improvements, but we believe our new table component will bring a huge improvement to your day-to-day usage of Cirrus. Whether you have hundreds or thousands of items, the library revamp will make it easier to organise and use items, while the new filters will give you better control and allow you to filter on a granular level.
You’ll be able to edit, label or assign items much more efficiently. Last but not least, the new labelling workflow will make it easier to find and select items for an exam, analyse test results and save time in the long run.
Ensure the reliability and validity of your exams
Psychometric data gives you invaluable insights into the quality of your items and helps ensure the reliability and validity of your exams. In upcoming upgrades, we will will make several quality of life improvements to the Statistics tab:
- You will be able to set Angoff values, a widely accepted and reliable method for setting cut scores on high-stakes exams. Read more about Angof in our recent blog post.
- You will also be able to set the difficulty thresholds, which will automatically be displayed in your collections. This way you can easily keep track of the difficulty level of your items.
And more…
Additionally, we’re implementing a variety of improvements to:
- Save time on exam creation, e.g. Assessment Templates that can be applied to new assessments).
- Enjoy more flexibility in creating questions – such as Question Timers on individual questions and the possibility for test-takers to add a Rationale in file response questions.
Stay tuned for our next Platform Update blog where we will keep you updated about recent upgrades and reveal other upcoming features (including the upcoming Marking coordinator dashboard, which will help you manage your marking workflow by giving insight into markers’ progress).
