Welcome to the spooky October edition of Hidden Gems! Don’t be frightened, this gem isn’t tricky and it’s a real treat – seed items!
Did you know that not every question in an exam actually counts towards the final score? In the world of assessment, some questions are simply there to test the test. These are called “seeded questions,” and they’re like a secret ingredient in delivering better exams. Let’s break it down with an example:
Imagine your candidates are sitting an exam with 10 items, but two of them (let’s say questions 3 and 7) are special. These seeded items don’t count towards your candidates’ final score—they’re there so you can see how your test-takers handle them. Are they too easy, too hard, or perhaps a bit unclear? Once the data is collected, you can decide whether to tweak the question, toss it, or make it a permanent part of future tests. It’s an easy way to figure out if your items are fair and balanced, and the candidates are none the wiser.
Why Seed Questions?
Seed items allow you to test new ideas, formats, and levels of difficulty without impacting performance. It’s like test-driving an item in a real-world setting before it officially hits the road.
The Perks for Your Organisation:
Fairer tests
By using seed items, we ensure that when a question does count, it’s been thoroughly tested for fairness, accuracy, and clarity, creating a more balanced exam experience for all candidates.
Continuous improvement
We’re constantly collecting performance data on items so that we can improve our question pool over time. This allows us to fine-tune both current and future exams, ensuring that they meet the high standards we set.
Less risk for candidates
Since seeded items don’t affect your candidates’ scores, they give us the flexibility to innovate, test new item types, and try different difficulty levels—without adding extra stress for test-takers.
Data-driven insights
Seeded items provide valuable psychometric data. We analyse this data to ensure our exams not only meet industry standards but exceed them. This data-driven approach helps us offer assessments that are rigorous, but never unfair, giving both organisations and candidates more confidence in the process.
How Seeded Questions Work in Cirrus:
Step 1: Create a seed item: When creating an item in Cirrus, by default, it is set to ‘off’, the toggle bar will be greyed out. To use it as a seed item, simply toggle ‘on’, the bar will now be green. This means the question will be included in the exam but won’t count toward the candidate’s score.
Step 2: Deliver the exam: Seeded questions are mixed with regular questions during the exam. Candidates won’t know which questions are seeds—they look and function exactly like any other.
Step 3: Collect psychometric data: After the exam, Cirrus tracks how candidates perform on all questions – including the seeded questions – collecting key metrics like accuracy, response time, and how well the question differentiates between high- and low-performing candidates.
Step 4: Calculate the P-Value: Cirrus calculates the p-value for each item, representing the percentage of candidates who answered correctly. A p-value between 0.25 to 0.9 indicates the item is acceptable to use.
You can find all items’ p-values in the ‘Statistics’ tab:
Step 5: Test before scoring: Seeded items allow organisations to gather data and ensure the item is fair, clear, and appropriately difficult before it’s officially used to calculate scores in high-stakes assessments.
Step 6: Review and finalise: Based on the psychometric data gathered, you can decide whether to refine the seeded item, discard it, or promote it to a fully graded question in future exams.
In short, seed items make exams more reliable and help you stay ahead of the curve, creating better assessments for everyone. This feature is just one of the many tools Cirrus offers to ensure that our platform delivers exams that are not only fair but continuously evolving to meet the needs of modern education and professional certification!
