AI Transcription and Reporting in General Practice: a Longitudinal Study
- Conditions
- Workload
- Registration Number
- NCT06691724
- Lead Sponsor
- Erasmus Medical Center
- Brief Summary
General practitioners (GPs) in the Netherlands are under unsustainable pressure. Recent surveys show that 68% of general practitioners find the workload too high and 18% find their work extremely or very stressful. The pressure on GPs significantly harms patient care, as reduced physician well-being can negatively impact patient experiences, treatment adherence, patient-provider communication, healthcare costs, care quality, and patient safety.
A key contributor to the stress is the increasing time commitment associated with clinical documentation. The documentation process has evolved into a time-intensive task, which is a significant obstacle to efficient patient care. Large language models (LLMs) are promising artificial intelligence (AI) solutions to reduce the documentation in general practice. In this project, the investigators aim to study an AI-based transcription and reporting tool in general practice.
- Detailed Description
Introduction Over the years the introduction of the electronic health record (EHR) and escalating demands for documentation have led to a mounting burden on GPs. Clinical documentation is a major barrier to efficient patient care as last year more than half the GPs spend more than 20% of their time on administrative duties. This burden leads to reduced job satisfaction and well-being, increased rates of burnout, and employee attrition. The documentation burden also negatively influences patient-provider communication. It leads to GPs making less eye contact, having a more closed body posture, and conveying less information to their patients. The documentation in the EHR however also has positive effects, such as reduced cognitive load and improvements in patient safety and care. Reducing the provider-computer interaction during the consultation may improve patient-provider interaction and provider well-being while retaining the benefits of EHRs.
In fact, studies showed that employing medical assistants for documentation during consultations leads to an increase in face-to-face time and improves patient satisfaction. Moreover, speech-to-text technologies for dictation after the consultation alleviated the documentation burden by increasing documentation speed, patient experience, and provider satisfaction. These interventions may lower burnout and attrition rates, strengthening the well-being of the providers as well as that of the healthcare sector.
The rapid advancement of large language models (LLMs) has opened new avenues to reduce documentation burden. LLMs, such as ChatGPT, are artificial intelligence (AI) models that can interpret and generate text. These models can transcribe and summarize a consultation with the GP in real-time removing the need for medical assistants or dictation after the consultation. However, if the LLM makes mistakes, this may lead to increased administrative workload. The investigators aim to assess the effect of a AI-based transcription and reporting tool in general practice.
Objectives Our primary objective is to assess the effect of a transcription and reporting tool on time spent on clinical documentation in general practice.
Secondary objectives are to assess the effect of a transcription and reporting tool in general practice on
* Total consultation time
* GP experience
* Patient experience
* Documentation length and quality
To assess the usage rates of the tool To assess the acceptability of use of the tool
Design This is a longitudinal before-after study. For each GP the investigators will observe two days of consultations without the tool. This will be done two weeks before implementation of the tool (baseline period). The investigators will also observe two days of consultations with the tool. This occurs two weeks after implementation of the tool (intervention period). Consultations from the intervention period will be compared with the baseline period to assess effectiveness of tool on the various outcomes.
Sample size A sample size simulation showed the investigators would need 30 observations to have enough power for the primary outcome. To improve the generalizability of our research and increase the sample for the qualitative outcomes, the investigators aim to observe in total four days of consultations for ten GPs from at least five different practices. Practices will be sampled purposively for differences in patient population and practice organization.
Outcomes The investigators will measure time outcomes through continuous observation. An observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.
Patient experience of the consultation will be measured with a validated patient experience questionnaire (PEQ) for general practice. Patient attitude on the use of the tool during the consultation will be expanded upon with a semi-structured interview based on prior research. The experiences of the GP with and impact of the tool on clinical duties will be measured with semi-structured interviews based on prior research. The acceptability and use of the tool by GPs will be assessed according to the unified theory of acceptance and use of technology (UTAUT).
Usage rates of the tool will be measured by assessing the proportion of consultations in which the tool is used. Documentation volume will be measured by determining the length of documentation in words. The documentation's information quantity will be measured by the number of relevant clinical variables in the note.
Recruitment & Eligibility
- Status
- ACTIVE_NOT_RECRUITING
- Sex
- All
- Target Recruitment
- 800
Not provided
Not provided
Study & Design
- Study Type
- OBSERVATIONAL
- Study Design
- Not specified
- Primary Outcome Measures
Name Time Method Time spent on documentation Measured during the consultation (baseline and intervention) This outcome refers to the time spent on documentation for a clinical consultation. The investigators will measure time outcomes through continuous observation. An external observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.
- Secondary Outcome Measures
Name Time Method Usage rates Measured directly after the consultation Usage rates of the tool will be measured by assessing the proportion of consultations in which the tool is used.
Documentation volume Measured directly after the consultation Documentation volume will be measured by determining the length of documentation in words.
Total consultation time Measured during the consultation (baseline and intervention) This outcome refers to the total time spent on the clinical consultation. The investigators will measure time outcomes through continuous observation. An external observer will monitor the time spent on various tasks during a consultation, including taking the medical history, conducting the physical examination, explaining the diagnosis or treatment plan, consulting a colleague, clinical documentation, and administrative duties like prescribing or referring.
GP experience with the tool Measured within one week after the two-day intervention period The experiences of the GP and impact of the tool on clinical duties will be measured with semi-structured interviews based on prior research with the following topic guide:
* What do you think about Juvoly QuickConsult?
* Does the system help you? How? Workload? Focus? Job satisfaction?
* Does the system help the patient? How? More time to explain or ask questions?
* Does the system hinder the you? How? Distraction?
* Does the system hinder the patient? Distraction? Don't want to talk about certain issues/share certain information? Less room for asking questions?Patient experience with the consultation Measured within 1 week after the consultation Patient experience of the consultation and communication with the GP will be measured with a patient experience questionnaire (PEQ) for general practice (doi: 10.1093/fampra/18.4.410).
The PEQ contains five scales:
* Outcomes of the visit (4-20, higher is better)
* Communication experiences (4-20, higher is better)
* Communication barriers (4-20, higher is worse)
* Experience with the auxiliary staff (2-10, higher is better)
* Emotions (4-28, higher is better)Patient experience with the tool Measured within 1 week after the consultation Patient experience and attitude with the tool will be assessed using a semi-structured interview based on prior research with the following topic guide:
* How did you experience the consultation?
* What do you think about the system the GP used to write down the consultation?
* Does the system help the GP? How? More explanation time?
* Does the system help you? How? More elaboration time?
* Does the system hinder the GP? How? Distraction for the GP?
* Does the system hinder you? Distraction? Don't want to talk about certain issues/share certain information? Less room for asking questions?Documentation information density Measured directly after the consultation The documentation's information quantity will be measured by the number of relevant clinical variables in the note. They will be categorized according to their type:
* Signs and related variables such as site, onset, character, or time course
* Contextual factors such as self-care, risk behavior or family history
* Symptoms
* (Differential) diagnosis
* PlanGP acceptability and use of the tool Measured 1 week before the baseline period, 1 week before the start of the intervention period, and 1 week after intervention period The acceptability and use of the tool by GPs will be assessed according to the unified theory of acceptance and use of technology (UTAUT). The questionnaire has five domains.
Performance expectancy: 4-20 points, higher is better Effort expectancy: 4-20 points, higher is better Social influence: 4-20 points, higher is better Facilitating conditions: 4-20 points, higher is better Behavioral intention: 3-15 points, no value judgement
Trial Locations
- Locations (1)
Erasmus MC
🇳🇱Rotterdam, South Holland, Netherlands