Accuracy and Effect of Prescribed Generative AI Patient Education Module Within Breast Oncology Care
Trial Snapshot
- Phase
- Not Applicable
- Status
- Recruiting
- Enrollment
- 35
- Locations
- 1
- Primary Endpoint
- Mean score of Provider Assessment of AI patient Education Tool Survey (Provider)
Study Overview
Brief Summary
This study evaluates the safety, accuracy, and impact of an artificial intelligence (AI) tool designed to support patient education in breast cancer care for breast oncology patients under selected physicians care within the University of California, San Francisco breast cancer center and affiliate sites.
Detailed Description
PRIMARY OBJECTIVES:
I. Lead-in period: To evaluate the safety of the OpenEvidence CareConnect a Retrieval-Augmented Generation (RAG) Large Language Model (LLM) tool during the initial lead-in phase in patients with metastatic breast cancer.
II. To evaluate the clinical accuracy and appropriateness of the OpenEvidence CareConnect RAG LLM tool in delivering general health and breast cancer-specific information.
SECONDARY OBJECTIVES:
I. To assess patient perceptions of the AI tool's helpfulness, alignment with other information sources, and its impact on communication with their oncology care team.
Study Design
- Study Type
- Interventional
- Allocation
- Randomized
- Intervention Model
- Crossover
- Primary Purpose
- Health Services Research
- Masking
- None
Eligibility Criteria
- Ages
- 18 Years to — (Adult, Older Adult)
- Sex
- All
- Accepts Healthy Volunteers
- No
Inclusion Criteria
- •Females or males ages 18 and over.
- •Patients must have either:
- •Stage IV breast cancer facing a treatment change and are expected to have follow-up visits at least once every three months, or
- •Stage I-III breast cancer diagnosed within the past 6 months.
- •Basic computer literacy and regular internet access at home.
- •Able to understand study procedures and to comply with them for the entire length of the study.
- •Ability of individual or legal guardian/representative to understand a written informed consent document, and the willingness to sign it.
Exclusion Criteria
- •Contraindication to any study-related procedure or assessment.
- •Cognitive impairment that would interfere with tool usage or survey completion
Arms & Interventions
Safety Lead-in Phase
Participants will receive access to OpenEvidence CareConnect a Retrieval-Augmented Generation (RAG) Large Language Model (LLM) tool that provides general health and breast cancer-specific information for 3 months.
Intervention: Survey (Provider) (Other)
Interventional (Immediate Access)
Participants will be given immediate access to OpenEvidence CareConnect for up to 3 months and complete surveys regarding use of tool. Participants are then followed up for an additional 3 months.
Intervention: Survey (Provider) (Other)
Control (Delayed Access)
Participants will be given access to OpenEvidence CareConnect for 3 months 90 days after enrollment
Intervention: Survey (Participant) (Other)
Safety Lead-in Phase
Participants will receive access to OpenEvidence CareConnect a Retrieval-Augmented Generation (RAG) Large Language Model (LLM) tool that provides general health and breast cancer-specific information for 3 months.
Intervention: Survey (Participant) (Other)
Safety Lead-in Phase
Participants will receive access to OpenEvidence CareConnect a Retrieval-Augmented Generation (RAG) Large Language Model (LLM) tool that provides general health and breast cancer-specific information for 3 months.
Intervention: OpenEvidence CareConnect (Other)
Interventional (Immediate Access)
Participants will be given immediate access to OpenEvidence CareConnect for up to 3 months and complete surveys regarding use of tool. Participants are then followed up for an additional 3 months.
Intervention: Survey (Participant) (Other)
Interventional (Immediate Access)
Participants will be given immediate access to OpenEvidence CareConnect for up to 3 months and complete surveys regarding use of tool. Participants are then followed up for an additional 3 months.
Intervention: OpenEvidence CareConnect (Other)
Control (Delayed Access)
Participants will be given access to OpenEvidence CareConnect for 3 months 90 days after enrollment
Intervention: OpenEvidence CareConnect (Other)
Control (Delayed Access)
Participants will be given access to OpenEvidence CareConnect for 3 months 90 days after enrollment
Intervention: Survey (Provider) (Other)
Outcomes
Primary Outcomes
Mean score of Provider Assessment of AI patient Education Tool Survey (Provider)
Time Frame: Up to 180 days
AI patient Education Tool Survey evaluates clinical accuracy and appropriateness of the OpenEvidence CareConnect tool based on provider responses using a 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree).
Mean score of Assessment of Answer Accuracy and Content Appropriateness (Provider)
Time Frame: Uup to 180 days
Provider will evaluate participant questions, answer accuracy and content provided within the OpenEvidence using 3-point scales.
Mean score of Safety and Risk Assessment (Lead-in and Provider)
Time Frame: Up to 90 months
Safety and Risk Assessment evaluates safety concerns based on provider responses using a 5-point Likert scale.
Secondary Outcomes
- Mean score of Medical Information Experience Survey (Participant)(up to 180 days)
- Mean score of AI Tool Experience survey (Participant)(Up to 180 days)
- Mean Change of Score of Patient AI Tool Experience Survey (Participant)(up to 180 days)
- Mean Change of Score of Medical Information Experience Survey (Participant)(Up to 180 days)