You’re on the hunt for an e-assessment platform. The world’s gone digital, and the pandemic didn’t just nudge but rather shoved us towards online solutions seemingly overnight. Maybe you’re using clunky, outdated legacy software or perhaps you’re ready for an alternative to the mountains of paper exams and expensive test centres that are a logistical nightmare. In our previous blog post, we covered the process of creating an exam platform shortlist.
The next step, navigating the space between “I need a new platform” and “Let’s take a demo” can be overwhelming and daunting. And demos can feel like one big sales pitch, leaving you to sift through a lot of new information, wondering what’s important and what’s just fluff.
Fear not! In this blog, we’re taking you through the world of e-assessment demos, minus the confusion and the sales jargon. This checklist will take you through the three critical steps of the demo: the pre-demo preparation, the demo itself, and the post-demo debrief, and guide you through all the questions you should be asking to get the most out of your demo.
Let’s dive in!
Preparing for a demo with a new exam platform vendor is important in ensuring that the chosen platform aligns with your organisation’s needs. Some platforms specialise in multiple choice exams only and may lack the critical features to give high-stakes medical exams. Others may only serve a specific industry. This guide will help you navigate the pre-demo preparation process effectively to ensure you can head into the demo armed with the right information.
Understand Your Current Situation
- Reflect on the limitations and problems with your current exam system or paper-based delivery.
- Determine what issues your new platform must address to improve your current organisation (efficiency, user experience, security, scalability, etc…determine what is important for YOU)
Set Clear Requirements
- Focus on the three most critical requirements for your organisation. These could be specific features, aspects of vendor reliability, customer support, or technological capabilities. Recognise that no platform will have every feature available in the market, so it’s really about aligning with top requirements that suit your needs.
- It’s about playing “chess” and thinking several moves ahead. Consider both your immediate needs and potential future expansions.
Decide the Mode of Exam Delivery
- Decide whether you need on-location, remote, or a combination of paper-based and online exam delivery methods. Laws vary country to country about offering in-person exams, so check with local laws before going fully digital.
- Understand how this choice impacts the security and integrity of your exams.
Determine Integration Capabilities
- If integration with existing systems (like LMS or student information systems) is necessary, outline specific requirements.
- Consider data flow, compatibility, and ease of integration.
Engage the Right People
- Identify the Decision-Making Unit (DMU) for acquiring the new exam platform. Involve them from the start, including during the demo.
- Include roles like Assessment Manager, Functional Administrator, lecturers, and IT staff, especially if integration is a key concern.
- This diverse involvement ensures that the selected platform meets the needs of all users and stakeholders.
Pre-Demo Meeting for Larger Organizations
- If your organisation is large, consider a pre-demo meeting with the vendor to discuss your main challenges. This helps the vendor tailor the demonstration to your specific needs.
Establish a Timeline
- Determine a timeline for your goal to be fully operational with the new platform. Communicate this to the vendor to align expectations and planning.
Ask to Record It
- Ask the vendor in advance if you can record the demo for future reference. Especially if the DMU cannot be there (we strongly recommend that they are present)
The key to a successful pre-demo preparation is to clearly understand your needs, involve the right people, communicate effectively with the vendor, and manage expectations realistically.
Tips from our experts:
💡Keep documents handy – Keep a record of your requirements, questions, and notes for easy reference during the demo.
💡 Schedule properly, and come prepared – Prepare a clear agenda for the demo, highlighting the areas you want the vendor to focus on. E-assessment platforms offer free, tailor-made demos and they are happy to cater their presentation to you, but this often involves time and work from the vendor. Prepping this information beforehand ensures you get the perfect demo, and the vendor knows exactly what you need.
💡 Keep space for questions – Allocate time during the demo for questions and discussions. Before a demo, don’t be afraid to ask for a 10 minute breather after the demo where you can mute your mic and discuss questions with your team before returning to the demo session.
During the Demo
Attending a demo for an e-assessment platform is essential for making an informed choice. Prepare by assembling a team of key stakeholders with specific questions and concerns. On the day of, hold the demo in a distraction-free environment with a good internet connection and plenty of coffee.
Start with a Brief Introduction:
- Begin by summarising your organisation’s requirements and goals, as outlined in your pre-demo prep.
- Confirm the agenda for the demo, ensuring all key areas will be covered.
Evaluate User Interface and Experience
- Focus on the simplicity and intuitiveness of the platform’s interface.
- Assess the ease of navigation, both for exam creators and takers.
- Check for customisation options to match your organisation’s branding or specific needs.
Assess Key Functionalities
- Question types and flexibility: ensure the platform supports various question types (e.g., multiple choice, essays, short answers) and multimedia content if needed.
- Monitoring and anti-cheating measures: explore the platform’s capabilities in real-time monitoring, plagiarism detection, and other security features.
- Data analysis and reporting: Look into how the platform handles data analytics, grading, and reporting. These features are crucial for tracking performance and making informed decisions.
Assess Integration and Compatibility
- “Can you demonstrate how your platform integrates with our systems?” Seamless integration is necessary for operational efficiency.
- “What are the technical requirements for your platform?” Ensuring compatibility with existing infrastructure avoids additional investments.
Ensure Good Support and Training
- What kind of training and onboarding do you provide? Adequate training ensures your team can make the most of the platform from day one.
- How is customer support handled? Look for vendors that offer comprehensive support, including accessible technical assistance and various support channels (like email, phone, knowledge base, and live chat).
- Can you customise the support to our specific needs? Customised support can especially benefit organisations with unique requirements or limited technical expertise.
Explore Customisation and Scalability
- “How customisable is your platform to specific needs?” Customisability ensures the platform meets your unique requirements.
- “How well does your platform scale?” A scalable solution is necessary to accommodate growth or changes in user numbers.
Discuss Pricing and Cost Structure
- What are the long-term costs? This includes the initial purchase price and recurring fees, such as maintenance, support, or additional features.
- Are there any hidden fees? Be wary of costs that are not immediately apparent, like charges for additional users, data storage, or updates.
- Can you provide a detailed breakdown of your pricing structure? A clear and detailed pricing structure helps avoid unexpected costs down the line. E-assessment platforms can charge per test, per user, etc., so you must know what suits your organisation best.
Security, Privacy, and Compliance
- “What security protocols are in place to protect sensitive data?” Ensuring strong security measures is vital for data protection.
- “How does your platform comply with regulations like GDPR?” Compliance is necessary to avoid legal issues and ensure data privacy.
- “Could you explain your policies regarding data ownership and privacy?” Understanding these policies is crucial for safeguarding your organisation’s and users’ data.
Performance and Reliability
- “How does your platform ensure reliability during high-stakes exams?” Feeling confident that the platform won’t buckle under pressure during critical tests is important.
- “Could you share your platform’s uptime statistics and how you handle technical issues or downtimes?” Get a feel for how often things might go sideways and what the plan is when they do.
Discuss Implementation and Transition
- Talk about the implementation process, timelines, and support offered during this phase.
- Understand the steps involved in transitioning from your current system to the new platform.
Future Upgrades and Roadmap
- Enquire about the vendor’s roadmap for future updates and features.
- Understand how the platform adapts to changing technologies and educational needs.
Ask for Case Studies or References
- Can you share case studies or references from similar institutions? Case studies demonstrate the vendor’s ability to meet the needs of organisations like yours, while references allow you to hear directly from their customers about their experiences.
- What is your experience in our specific sector (education, corporate training, etc.)? This question helps gauge their understanding of and experience in addressing your field’s unique challenges and requirements.
Take Detailed Notes
- Assign someone to take detailed notes or, if permitted, record the session for later review.
- Pay special attention to how the platform addresses your top three requirements.
Have a Q&A Session:
- Have a Q&A session post-demo to address any immediate questions or clarifications.
- Discuss any concerns or specific scenarios that weren’t covered during the demo.
- Agree on the next steps post-demo, including any follow-up meetings or additional information required.
- Set a timeline for these follow-up actions.
And your first demo is done! At this point, you should have a good idea of whether or not this e-assessment platform makes the next round of cuts.
The ‘during the demo’ phase is a great opportunity to actively engage with the e-assessment platform, allowing you to assess its capabilities, usability, and alignment with your organisation’s specific needs. This phase is not just about observing the platform’s features but also about understanding how it can enhance your assessment processes, integrate with your existing systems, and adapt to your future requirements. Active participation, critical evaluation, and thorough questioning during the demo are key to gaining a comprehensive understanding of the platform.
Tips from our experts:
💡 Be interactive – Participate in the demo by exploring the platform hands-on. This could involve simulating the creation of an exam or navigating the platform as a test-taker. Interactive engagement helps understand the user experience from both the administrator’s and the student’s perspectives, providing a clearer picture of the platform’s ease of use and functionality.
💡 Clarify, clarify, clarify – I know we’ve said this a lot, but use your demo as an opportunity to ask questions and seek clarification. Prepare a list of questions beforehand and encourage stakeholders from different areas of your organisation to inquire about aspects relevant to their roles. This ensures that all concerns are addressed and you comprehensively understand the platform’s capabilities and limitations.
The demo has wrapped with the e-assessment vendor – what’s next? It’s important to dive into a structured evaluation. It’s all about figuring out if what you’ve seen aligns with what your organisation, and if the platform covers all of your bases.
Debrief and Consolidate Feedback:
- Organise a debrief session with all stakeholders who attended the demo.
- Encourage an open discussion to gather diverse perspectives and insights.
- Consolidate feedback on the platform’s features, usability, integration capabilities, support, and overall impression.
Review Recorded Demo (If Applicable)
- If the demo was recorded, review the recording to revisit key points and clarify any uncertainties.
- Pay special attention to how the platform addresses your top three requirements and the additional benefits of digitisation.
Evaluate Against Initial Requirements:
- Refer back to your initial checklist of requirements and challenges.
- Assess how well the platform meets these criteria, including the must-have features and integration needs.
- Consider both the immediate functionalities and long-term scalability.
Discuss Implementation and Transition Plan
- Reflect on the discussed implementation process. Is it feasible and well-aligned with your timeline?
- Consider the resources and support offered for transitioning to the new platform.
- Evaluate the potential impact on your current operations during the transition period.
- Weigh the costs, including any additional charges, against the platform’s benefits and improvements.
- Consider the return on investment, particularly regarding efficiency, security enhancements, and user satisfaction.
Solicit Additional Information or Clarification
- Reach out to the vendor for further clarification on any unclear points after the demo.
- Request additional documentation or case studies if needed.
Compare with Other Options
- If you are considering multiple vendors, compare each platform against the same set of criteria.
- Use feedback from the demos to rank the platforms based on how well they meet your needs.
Engage in a Trial Period (If Possible):
- If the vendor offers a trial period, take advantage of it to test the platform in a real-world scenario.
- Use the trial to validate the platform’s performance and alignment with your expectations.
Plan for a Follow-Up Meeting
The post-demo phase is a period for reflection, analysis, and collective decision-making. Thorough evaluation, considering both immediate and long-term needs, and engaging all relevant stakeholders during this phase are crucial for selecting a platform that not only meets your current requirements but also aligns with future growth and technological advancements.
Three Key Tips:
💡Leverage collective insight – Encourage open discussion and gather feedback from all stakeholders involved in the demo. Different perspectives, especially from those who will directly use or manage the platform, can provide invaluable insights that might not be immediately apparent. This collective insight helps in making a well-rounded decision.
💡Focus on practical application – If a trial period is available, utilize it to test the platform in a real-world environment. Hands-on experience can reveal practical strengths and limitations that might not be evident in a controlled demo setting. This approach helps to validate the vendor’s claims and ensures the platform truly meets your operational needs.
💡Put the e-assessment vendor on speed dial – This is a big step, and good vendors can answer any or all your questions regarding their platform. Keep the lines of communication open with the vendor after the demo. Follow-up questions, requests for additional information, or clarifications are common and necessary. Responsive communication from the vendor at this stage can indicate the level of support and partnership you can expect in the future.
💡Don’t dismiss a vendor straight away if they’re missing a key feature you’re after. Instead, ask about their willingness to develop it specifically for you. The best vendors often show flexibility and may offer to create bespoke features tailored to your exact needs. On the other hand, some vendors might be more rigid. It’s a good idea to inquire about their approach to customisation.
And there you have it! You’ve navigated the pre-demo prep, actively participated in the demo, and now you’re through the post-demo evaluation. It’s a lot to take in, but remember, choosing the right e-assessment platform is a big decision, and it’s about finding that perfect fit for your organisation. By engaging thoroughly at each stage, asking the right questions, and involving the right people, you’re setting yourself up for a smart, well-informed choice. Whether it’s about getting down to the nitty-gritty of features, understanding the cost implications, or just feeling out if the vendor is willing to walk that extra mile with you – it’s all part of the journey. So take your time, weigh your options, and soon enough, you’ll be on your way to transforming your assessment process with the ideal platform. Happy hunting!
Haven’t created your shortlist yet? Check out our previous blog on the 4 essential steps to guide your selection. Coming up soon: Practical tips to help you make the final decision and choose the best online exam platform for your requirements.