Can You Evidence That?

Share via:

In assessment, the spotlight can land on us at any time, and scrutiny no longer comes only from regulators.

It can arrive via social media, traditional media or a candidate willing to challenge a decision publicly, and when questions are asked, they rarely focus on whether our intentions were good.

They focus on whether we can evidence what we did, and why.

Recent, highly publicised reviews of assessment practices at respected institutions reinforce a simple truth. Scrutiny now arrives faster and with greater visibility, than ever before.

In that environment defensibility is not optional, it is foundational. 

The risk that we face is not bad intent. It is complacency.  We can all assume that because processes have worked up until now, they will survive future assessment cycles. We can all be guilty of thinking “it can’t happen here”. 

For many of us, the core of our assessment processes has changed less than we might assume. Layers have been added. Systems have been introduced. Reporting has improved. But underneath, manual workarounds remain. Spreadsheets proliferate. Temporary fixes have gradually become permanent features.

And if scrutiny comes, “we’ve always done it this way” will not be considered as a defensible answer. We will be asked to evidence our decisions. 

What “Regulatory-Ready” Actually Looks Like

Regulatory expectations for assessment programmes are surprisingly consistent across frameworks, even if they’re not always spelled out in a single document. Common themes include: 

  • Clarity of purpose and construct: A clear definition of what is being assessed, and why. Ambiguity at this stage creates problems later in the cycle.
  • Auditability: Every significant decision, from design through to marking and result release, can be quickly and easily reconstructed and explained. 
  • Documented governance: Clear roles, responsibilities, escalation routes, and conflict management, actively monitored, not assumed.
  • Robust processes: Processes that are fit for purpose and reviewed regularly.  
  • Transparent standard-setting: Clear evidence of how pass marks, grade boundaries and rubrics are established, reviewed, and adjusted where needed.
  • Continuous reliability and validity monitoring: Not a one-off report, but an iterative cycle where data informs action.
  • Sustainable quality assurance:  Well documented and robust systems that are strong enough to withstand workload pressure and the turnover of key staff.   

These are not aspirational extras. They are the foundations that allow an organisation to respond calmly and confidently when challenged 

Where Things Start to Drift

In practice, most of us sit somewhere between compliant and fully defensible. 

Policies exist, but they do not always reflect what happens in practice. 

Decisions are made with sound, professional judgement, but the rationale is inconsistently recorded and difficult to find.  

Governance minutes may highlight concerns, but actions are delayed under workload pressures. 

Reliability data is produced, but not always used strategically. 

Quality assurance becomes reactive rather than a continuous safeguard.

None of this reflects incompetence. It reflects busy, committed teams working under constraint.

But when scrutiny intensifies, gaps that felt manageable internally can appear significant to external parties. 

And when documentation has to be located or recreated under pressure, the situation rarely improves.

Defensibility is not tested on quiet days. It is tested when the spotlight turns unexpectedly.

Building Confident Defensibility   

Defensibility is not something we simply have or we do not have. It is something that we build and strengthen over time. 

So where do we start?

The first step is clarity: an honest view of where our current assessment programme aligns with expectations, and where it relies too heavily on memory, informal understanding or inherited process. 

That means asking some uncomfortable, but essential, questions:

  • Are our processes truly fit for purpose?  When were they last reviewed?  How are concerns raised and dealt with internally? 
  • Is our construct clearly defined, and consistently translated into assessment design?
  • Are we confident that our governance and QA processes are clearly documented and robust? Could they survive the sudden loss of a key member of the team?
  • Can we evidence our standard-setting decisions quickly and clearly?  
  • How do we track reliability and validity over time, and what do we do when the data signals concern?

If any of these questions feel challenging, that is often where the most valuable insight lies. Because defensibility is not about preparing for crisis. It is about reducing the likelihood of one. 

Clarity Is Hard from the Inside

Asking these questions is an important first step. But answering them clearly and with evidence is not always straightforward from within the day-to-day reality of delivery.

Processes evolve, pressures build, and ways of working adapt over time. What feels clear internally can look very different when viewed from the outside. Gaps that are manageable in practice can become harder to explain under scrutiny.

For some organisations, that clarity comes from stepping back and taking an independent view of their assessment processes, to see what stands up well and where further confidence is needed.

If you would like a clearer, evidence-based view of how your assessment processes stand up to scrutiny, our assessment process check is a simple way to get started.

Share via:
Topics
Picture of Robert Burns
Robert Burns
Robbie works with assessment organisations on the practical realities of delivering high-stakes assessments, with a particular focus on operational resilience, assessment integrity and candidate experience in digital delivery models.
Would you like to receive Cirrus news directly in your inbox?
More posts in Better Assessments
Better Assessments

Choosing the Right Proctoring Model for Your Assessment

Everyone wants to know the best way to proctor an exam. There isn’t one. Every qualification carries its own purpose, stakes, candidates and constraints, so the useful question is not which model is best, but which model fits, and whether you can explain why you chose it.

Read More »
Better Assessments

How Secure Is Secure Enough?

There is no universal definition of secure enough. The answer depends on what the task exposes, what is at stake if the result is wrong, and the scale you are working at. The third article in The Proctoring Question sets out how to match the control to the risk, and what it costs when you get it wrong.

Read More »

Know exactly where your assessment operation stands

Your free personalised report in 4 minutes

Answer 12 questions across strategy, delivery, design and data and get a clear, personalised breakdown of where your assessment operation is strong and where to focus next.