Examination Complaints and Appeals Policy

Multiple Choice Question (MCQ) examinations

All MCQ examinations are automatically scored in the platform, TestReach, which reduces the risk of a calculation error compared to manual scoring. The allocation and addition of scores are checked before a standard setting meeting is held. This meeting determines the passing mark using the Angoff standard setting method and checks the performance of examination questions using question psychometrics and any candidate feedback which has been received.

Constructed Response Question (CRQ) examination (Final FRCA only)

Live raters score the CRQ component of the Final FRCA examination. Examination scripts are downloaded as PDFs from the platform, TestReach, soon after the examination is delivered. The scripts are allocated to examiners who mark them using an online question marking scheme. Before the marking process begins, all examiners attend a meeting that determines the final marking scheme. Examiners form 12 groups with each group reviewing one question from the paper. They do this by first marking a sample of the scripts and comparing the candidate response to the set marking scheme. The marking scheme is discussed and edited accordingly. Only once the marking scheme has been agreed upon does marking begin. Examiners mark one question each therefore each exam paper has a total of 12 separate markers. This process ensures the accuracy and consistency of marking is of a high standard and reduces bias.

The leads for each question group sample the marking submitted.  If there are discrepancies in how the marking scheme has been applied, feedback is sent to the examiner, who will review the marked scripts. Second marking may also take place if required.  A robust moderation meeting is then held which scrutinises examiner grading and question performance before the release of results. 

Each question is marked out of 20 with the pass mark for each question determined by the examiners collectively before the examination using the Angoff standard setting method. The pass marks for the 12 questions are summed to give a total mark for the whole paper. 

Objective Structured Clinical Examination (OSCE) and Structured Oral Examination (SOE)

The OSCE and SOE components of RCoA examinations are scored on iPads using an online marking scheme via the delivery platform, Practique. Colour coding on the system enables examiners to identify areas yet to be marked. On submitting the marks, the system flags up any unmarked questions. An administrator checks the accuracy of the live submission of marks throughout the day. If a mark was missed during this process, the Standard Setting Manager notifies the examination floor, and the examiner reviews the relevant question and resubmits the marks.  Any anomalies in the marking are reviewed by the Standard Setting Manger and discussed at the post-exam moderation board.  Two examiners independently mark each part of the SOE and provide feedback to support the marks awarded, which can be requested by candidates on receipt of their examination result.