Response to the Independent Review of the Assessment Processes of the RCoA

We welcome the findings of an independent review of all examinations delivered by the Royal College of Anaesthetists, conducted by Professor John McLachlan and published in full today.

We will address the recommendations, many of which align closely with those of our recent internal review of the FRCA, working closely with the Faculties of Pain and Intensive Care Medicine. We are committed to delivering fair, robust and high-quality examinations that are aligned with GMC standards and best contemporary practice and that uphold and improve standards of patient care. The reviews will inform our developmental roadmap, and we will implement changes with care over the coming months and years, taking into account the needs of candidates and consulting with stakeholders.

We understand that the FRCA exams are high stakes assessments that have a significant impact on candidates’ personal and professional lives. For that reason, it was essential for the review to capture the experiences and opinions of our members and other stakeholders, particularly those of current and past candidates. Their input in the review process has been invaluable, as has that of our examiners. We are grateful to everyone who has participated in the review process and thank them for their openness and honesty. We also extend our thanks to Professor McLachlan for undertaking the review.  

While it is likely that changes will be made to the FRCA exam format in future, the current FRCA examinations remain valid, reliable, fit for purpose and are approved by the GMC. We will ensure exam candidates are given at least 12 months’ notice of any significant changes to the FRCA exams in order to facilitate good preparation and support. Therefore, anaesthetists in training and prospective candidates should not change the way they prepare for the current sets of exams until formally notified of any changes.

Scope of the independent review

We commissioned the review to provide independent assurance and insights into the development and delivery of our assessment processes, alongside our regular internal reviews of the FRCA. The review considered all aspects of the FRCA, FFICM and FFPMRCA exams, including an error in the delivery of the September 2021 FRCA Final Written Exam and an unexpectedly low pass rate for the October 2021 FFICM Objective Structured Clinical Examination (OSCE).   

We have published a summary of our proposed timetable for implementation of the changes resulting from the review, and we highlight the main themes relevant to the FRCA examinations here. We will keep our members and the GMC informed of our progress through regular updates on our website and member communications.

Response to the September 2021 FRCA Final Written Exam  

The review concluded that the error in the delivery of the September 2021 FRCA Final Written Exam was derived in large part from the stresses of converting written assessments to online versions due to the pandemic, and that such errors are unlikely to occur again. It also found that actions taken by the College to mitigate the effects were as reasonable as possible in the circumstances.

We understand that this does not lessen the impact of the errors that were made, and we apologise again for the difficulties and stress caused to candidates. We are fully committed to ensuring errors do not reoccur. In line with Professor McLachlan’s recommendations, we have taken action to invest in additional capacity in our exams team and made improvements to our pre-delivery processes for written exams. We will make further improvements to the FRCA in terms of resourcing and our platforms, processes and support for candidates, as outlined below.  

Our programme of improvement 

We are committed to pursuing excellence in all that we do, including the governance and delivery of the FRCA and Faculty examinations. We fully endorse the recommendations from Professor McLachlan in relation to examiner culture, including ensuring our codes of conduct are a key part of examiner training and continuing our efforts to increase the diversity of examiner pools.

In considering the recommendations from both the independent review and our internal review, we will: 

• Invest additional resources in our examinations function to implement changes arising from the review and strengthen our capacity to deliver exams. We have established an Examinations Development and Assurance Group (EDAG) to lead the development, quality and alignment of exam processes across the FRCA and Faculty exams, where appropriate. This group will have an independent chair and comprise examiners from all exams delivered by the College, members of the exams team, lay and trainee representatives and a specialist educational assessment expert.
• Give candidates and anaesthetists in training a greater role in our assessment processes, including more representation on College committees engaged in assessment. These representatives will also play a central role in supporting communication with candidates. 
• Review the purpose of our assessments in accordance with the changes made in the curriculum and the changing nature of clinical practice in our specialties. We will consult with members and stakeholders to define this.  
• Implement a unified approach to the use of, and training in, standard-setting across the FRCA exams and all assessments delivered by the College. 
• Undertake new research to inform our assessment design process. We are committed to investigating the impact of gender, ethnicity and educational background on exam performance. We also aim to undertake validity research to compare exam performance with performance in the workplace as estimated by trainers and supervisors with performance in RCoA assessments.  

Your feedback

We understand that members may have questions about the independent review, and the action we are taking as a result. We welcome your feedback and encourage you to attend our Let’s Talk member engagement online event on Wednesday 8 March at 19.00.

• Register to attend Let’s Talk

Members can also share their views with us using our online form. This feedback will be shared with our Examinations Development and Assurance Group. 

• Share your feedback using our online form

Dr Fiona Donald
President, Royal College of Anaesthetists