My account

New online exam system testing feedback

26 July 2023

In June over 300 candidates from over 30 countries helped test our new online exam system. 

Testing a system under exam conditions and gathering candidate feedback is an essential part of any system development and provides us with critical information to help further improve our design, operational processes, and communications. 

We would like to thank everyone who participated in these test exams and shared feedback with us. 

Candidates who took part in the test were able to experience sitting an exam with remote invigilation for the first time and provided feedback on using the new system. 

We want to share key insights and learning from our exam system tests: 

Connecting to the exam system

  • The remote invigilation onboarding process, which involves verifying candidates’ identity and exam environment, took on average, 10 to 20 minutes to complete. 
  • Providing photographic identification and taking a ‘selfie’ was easy to do, however taking a 360° room scan was more difficult. 

Testing has highlighted the importance of candidates being able to practice the onboarding process before sitting their exam. We will therefore make sure all candidates can practice onboarding before a live exam and have access to detailed instructions on how to complete the process.

Using the exam system

  • Essay-style and objective style questions are clear and easy to use.  
  • It is possible to add complex calculations onscreen and there is a good range of mathematical functions.
  • Spreadsheet questions were tricky to use and did not contain all standard Microsoft functions. 
  • A small number of participants found the system slow to load questions, especially those that contained complex content, such as spreadsheets. 
  • 90% of participants liked the exam count down timer. 

Candidates will need to practice using the in-system tools well in advance of a live exam. To support candidates, we will be releasing familiarisation exams and producing system guides and other resources. We are continuing work to identify how best to deliver exams which require large amounts of complex mathematical calculations or the use of spreadsheets before we move them to the new exam system in September 2024.

Exam support services and remote invigilation

  • Participants were unclear as to which of the support chats they should use for particular issues.  Further clarity will be needed before exam day. 
  • Most participants chose to email the support team for assistance rather than call.  This led to slower issue resolution times. 

We will be providing clarity on when candidates should be using onscreen technical chat. We will also develop and share clear guidance on exam rules and regulations before go-live. 

Preparing the new system

We are continuing our analysis of candidates’ test feedback and working with our supplier to further refine, test and improve the new online exam system and our operational processes. In August, we will assess of our readiness to launch the new system and will share the outcome with candidates and employers at the earliest possible opportunity.

We will continue to update you on the introduction of our new exam system and share guidance and resources on our exam developments web page.