Artificial Intelligence as a Decision Making Tool in Neurology
- Conditions
- Clinical Decision-makingMedical ReportingNeurological Diseases or ConditionsEmergency Department Visit
- Registration Number
- NCT06902675
- Lead Sponsor
- Rambam Health Care Campus
- Brief Summary
Background: The establishment of neuroinformatics as a distinct field has enabled the integration of computational biology and informatics to improve neurological research. This interdisciplinary approach enhances the capacity to integrate diverse datasets, unravel complex neural networks, and develop computational models that can improve clinical management. The investigators aim to evaluate whether an artificial-intelligence-based tool is effective in non-English-speaking regions.
Hypothesis: Integrating a language model-based clinical assistance system within the neurology ward will significantly enhance the efficiency and accuracy of patient care by leveraging neuroinformatics principles. The investigators hypothesize that combining natural language processing and data analytics will improve diagnostic and treatment processes.
- Detailed Description
Research Design:
A pre-post-intervention design will be used, measuring outcomes before and after the implementation of a neuroinformatics-driven clinical assistance system. Changes in diagnostic accuracy, treatment decisions, and workflow efficiency will be quantified.
Workflow:
1. A neurologist evaluates each patient, and the patient signs informed consent.
2. A medical student manually uploads de-identified clinical information to a secure interface.
3. Data are analyzed by a large language model (LLM) system through a hospital-approved application.
4. A senior physician must approve any decision based on the LLM's recommendation.
5. For half of the prospectively enrolled participants, the LLM's recommendation is presented to the resident.
Retrospective Data:
1. The investigators will extract data from 10,000 patients who were evaluated by a neurologist in the Emergency Department.
2. Clinical information (without patient identifiers) will be uploaded to a secure, hospital-approved LLM with a security key.
3. The model's output will be compared with actual clinical decisions and patient outcomes (e.g., mortality, discharge status).
Recruitment & Eligibility
- Status
- ACTIVE_NOT_RECRUITING
- Sex
- All
- Target Recruitment
- 1000
- Informed consent
- Unable to give informed consent
Study & Design
- Study Type
- OBSERVATIONAL
- Study Design
- Not specified
- Primary Outcome Measures
Name Time Method Proportion of correct diagnoses generated by the LLM compared to the final attending physician diagnosis. 3 years * Description: Measures the percentage of times the LLM's recommended diagnosis matches the final attending physician's diagnosis.
* Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%)
* How Assessed: Each LLM recommendation is compared to the final documented diagnosis in the medical record. The number of correct diagnoses is divided by the total number of cases.Accuracy of Next-Step Recommendations Up to 3 years. Proportion of correct or appropriate next-step management decisions recommended by the LLM.
Description: Proportion of correct next-step management decisions recommended by the LLM, compared to either the attending physician's final plan or standard-of-care guidelines.
Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%) How Assessed: For each encounter, the LLM's recommended "next step" is recorded and deemed correct if it aligns with the attending plan or guideline.
- Secondary Outcome Measures
Name Time Method Cost of Running the LLM Up to 3 years Average daily cost (in USD) to operate the large language model.
Description:
This outcome will measure the direct computational and licensing costs incurred by running the LLM in the clinical setting.
Unit of Measure: US Dollars (per day). How It Is Assessed: The total daily spend for the LLM (e.g., cloud-compute fees, licensing fees) will be recorded and divided by the total number of patient encounters that day to derive an average cost.Staff Compliance With AI Up to 3 years This outcome will assess how often (compliance) and how willingly (acceptance) physicians, residents, and nurses use the LLM in eligible clinical scenarios.
Description: Determines how often physicians, residents, and nurses use the LLM for eligible patient encounters.
Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%) How Assessed: Tracks the number of encounters in which the LLM is actually used, divided by the total number of eligible encounters. Higher percentages indicate greater compliance.Transparency/Explainability 3 years Description: Evaluates how clearly the LLM's reasoning is communicated to clinical staff.
Unit of Measure: Score on a 5-point Likert scale (1 = not at all clear, 5 = extremely clear) How Assessed: After each LLM-guided decision, the resident or attending physician rates the clarity of the model's explanation. A mean score is reported. Higher scores indicate better clarity.Quality of LLM-Generated Clinical Reports Up to 3 years Accuracy and Completeness Score of the LLM-Generated Clinical Report This outcome evaluates how accurately and comprehensively the LLM summarizes a patient's clinical encounter and recommended plan in a structured report (e.g., discharge instructions, progress notes).
Unit(s) of Measure: Quality Score based on a modified SOAP (Subjective, Objective, Assessment, Plan) Note Rating Scale, ranging from 0 to 10, where higher scores indicate better documentation quality. The score reflects the inclusion of required elements and correctness of clinical details.
Assessment Method: Each LLM-generated report is independently evaluated by a panel of attending physicians or trained clinical staff. Using a standardized checklist, reviewers assess completeness (e.g., inclusion of symptoms, physical findings, diagnosis, and plan) and accuracy (e.g., correct medications, identifiers, and diagnoses), then assign a score based on the predefined rubric.Staff Acceptance of AI Up to 3 years Description: Evaluates how willingly staff integrate the LLM into their clinical workflow.
Unit of Measure: 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) How Assessed: After each LLM-assisted encounter, staff complete a brief survey about perceived helpfulness and willingness to reuse the LLM. Higher scores indicate greater acceptance.
Related Research Topics
Explore scientific publications, clinical data analysis, treatment approaches, and expert-compiled information related to the mechanisms and outcomes of this trial. Click any topic for comprehensive research insights.
Trial Locations
- Locations (1)
Rambam healthcare campus
🇮🇱Haifa, Israel