Faculty Development to Improve Mini-CEX Ratings

Not Applicable

Completed

• Conditions: Faculty Development
• Interventions: Behavioral: Rater training workshop

• Registration Number: NCT00667940
• Lead Sponsor: Mayo Clinic

Brief Summary

We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.

Detailed Description

Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy.

Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability.

Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy.

Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields.

Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.

Basic Information
|
Recruitment & Eligibility
|
Study & Design
|
Trial Locations

Study Timeline

• Study Start: 2006-03
• Primary Completion: 2006-06
• Study Completion: 2007-06
• Status Verified: 2008-04
• Study First Submitted: 2008-04-23
• Study First Submitted QC: 2008-04-25
• Last Update Submitted: 2008-04-25
• Study First Posted: 2008-04-28
• Last Update Posted: 2008-04-28

Recruitment & Eligibility

• Status: COMPLETED
• Sex: All
• Target Recruitment: 52

Inclusion Criteria

• Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota

Read More

Exclusion Criteria

• None

Read More

Study & Design

• Study Type: INTERVENTIONAL
• Study Design: PARALLEL

Arm && Interventions

Group	Intervention	Description
Workshop	Rater training workshop	Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion

Primary Outcome Measures

Name	Time	Method
Interrater reliability (videotaped resident-patient encounters)	April-June 2006

Secondary Outcome Measures

Name	Time	Method
Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores	April-June 2006
Interrater reliability (actual resident-patient encounters)	March 2005 (historical) to June 2007

Trial Locations

Locations (1)

Mayo Clinic; 🇺🇸 Rochester, Minnesota, United States