Implementation of Directly observed procedural skills (DOPS) as a tool for formative assessment among anesthesia residents- perception of faculties and students”

Background

Skill development is very important part of anesthesia training. During conventional anesthesia post graduate training, the procedures are usually done under faculty’s supervision and real time guidance is provided. However, focus is on technical aspects of the procedures, and other aspects like preparation, communication with the patient, identifying difficulty and timely seeking help are usually overlooked. Assessment drives learning. Traditional assessment methods do not judge students’ skills.

In post graduate training, students become focused on knowledge gain so as to pass in summative assessment and get the post graduate degree.  The skills and confidence of the students can be enhanced if formative assessments and feedback are done regularly during their training. Workplace based assessment (WPBA) tools like DOPS, mini-CEX, case-based discussions can very well help the students to understand their fallacies and enhance their skills, as constructive feedback is integral part of such assessments.

Purpose/ Aim:

With this project, we would like to start WPBA in the form of DOPS in regular practice, along with its entry in particular student’s logbook.

Objectives:

·      To plan and implement DOPS as a routine formative assessment

·      To assess improvement in post graduate students’ skills

·      To get qualitative feedback regarding feasibility of DOPS from faculties and students by focused group discussions

·      Change in practice from routine assessments to formative assessments

 Methodology

Study design- mixed method observational study

Study participants and setting- The project will be conducted in department of anesthesia among the residents and faculties, after ethical approval from institutional ethics committee. Faculties who wish to participate and anesthesia residents in their first and second year of training will be included, after taking written and informed consent. The residents who have other peripheral postings will be excluded.

Study procedure- The residents and faculties will be trained about DOPS before commencement of the project, by sharing literature, videos, and power point presentation by a faculty from MEU of our institute. One day workshop will be organized at the MEU center for methods of WPBA, inviting a prominent faculty from this field. The faculties will also be trained on how to give constructive feedback to the students.

Study tool: DOPS checklists for anesthesia skills are available online from Australian and New Zealand College of Anaesthetists (ANZCA). They will be modified as per the procedures if needed, and will be validated by subject experts as well as member of MEU. After modification, final checklists will be prepared. DOPS checklist includes 11 points, each point is given score from 1 to 5, very satisfied, Satisfied, neither satisfied nor dissatisfied, Dissatisfied, and Very dissatisfied.

We plan to conduct DOPS assessments monthly during resident’s OT postings, on elective patients. The students will be assessed for skills as per their level of training. As for example, first year residents will be assessed for spinal anesthesia and supraglottic airway insertion. Second year residents will be assessed for peripheral nerve blocks and laryngoscopy and intubation. The faculty assessing the student will score the performance from 1 to 5 on each of the 11 points of DOPS checklist. At the end, faculties will also give score as per global assessment scores, as per their confidence on students to leave the procedures on themselves without supervision. The students will be given constructive feedback post assessment. Mainly, the points on which student performed well, and area where there is a scope of improvement. Time taken for DOPS and feedback will be recorded. Each such interaction will be entered in their logbooks. We plan for minimum 3 interactions per student, per procedure.

At the end of this project, focused group discussion will be planned for faculties as well students, separately, to know their feedback on DOPS. Discussion will be conducted by a faculty who is not directly involved in this project, and with whom faculties as well students can share their opinions without hesitancy. Opinions regarding feasibility, problems, student and faculty interest, their approach, communication, satisfaction, etc. will be discussed. The discussion will be audio recorded after their consents, and will be transcribed. Transcripts will be reviewed by two members involved in this project. Potential problems and scope of improvements will be identified, if any. Themes of the qualitative feedback will be identified and analysed accordingly.

Quantitative feedback will also be taken from faculties and students regarding usefulness of such assessments. Feedback from the faculties regarding feasibility, limitations, time constrain, etc will also be taken