Preventing Medication Dispensing Errors in Pharmacy Practice with Interpretable Machine Intelligence
- Conditions
- Machine Intelligence in the Pharmacy
- Interventions
- Behavioral: No MI HelpBehavioral: Scenario #1Behavioral: Scenario #2
- Registration Number
- NCT06245044
- Lead Sponsor
- University of Michigan
- Brief Summary
Pharmacists currently perform an independent double-check to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. This research is being conducted to examine the effectiveness of the timing of machine intelligence (MI) advice on to determine if it results in lower task time, increased accuracy, and increased trust in the MI.
- Detailed Description
Pharmacists currently perform an independent double-check currently to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. Instead, pharmacists rely solely on reference images of the medication which they can compare to the prescription vial contents. Previous research has shown that decision support systems can effectively improve healthcare delivery efficiency and accuracy, while preventing adverse drug events. However, little is known about how MI technologies impact pharmacists' work performance and cognitive demand.
To facilitate the long-term symbiotic relationship between the pharmacists and the MI system, proper trust needs to be established. While trust has been identified as the central factor for effective human-machine teaming, issues arise when humans place unjustified trust in automated technologies do not place enough trust in them. Over trust in automation can lead to complacency and automation bias. For instance, the pharmacists may rely on the MI system to the extent that they blindly accept any recommendation by the system. Under trust can result in pharmacist disuse and potential abandonment of the MI system.
Furthermore, little is known about the timing of the MI advice on pharmacists' work performance. For example, showing the MI's advice while the pharmacist is performing the medication verification task may yield different results than showing the MI's advice after the pharmacist made their decision.
The study investigators have developed a MI system for medication images classification. The objective of this study is to examine the effectiveness of the timing of MI advice to determine if it results in lower task time, increased accuracy, and increased trust in the MI.
Recruitment & Eligibility
- Status
- COMPLETED
- Sex
- All
- Target Recruitment
- 69
- Licensed pharmacist in the United States
- Age 18 years and older at screening
- PC/Laptop with Microsoft Windows 10 or Mac (Macbook, iMac) with MacOS with Google Chrome, Edge, Opera, Safari, or Firefox web browser installed on the device
- Screen resolution of 1024x968 pixels or more
- A laptop integrated webcam or USB webcam is also required for the eye tracking purpose.
- Participated in Wave 1 or Wave 2
- Eyeglasses
- Uncorrected cataracts, intraocular implants, glaucoma, or permanently dilated pupil
- Require a screen reader/magnifier or other assistive technology to use the computer
- Eye movement or alignment abnormalities (lazy eye, strabismus, nystagmus)
Study & Design
- Study Type
- INTERVENTIONAL
- Study Design
- CROSSOVER
- Arm && Interventions
Group Intervention Description Scenario #2 Scenario #2 MI help will be displayed concurrently with the filled and reference images. No MI Help Scenario #1 No MI help will be presented during the verification tasks Scenario #1 Scenario #2 MI help will be presented in the form of a pop-up message the participant's decision differs from the MI's determination. Scenario #2 Scenario #1 MI help will be displayed concurrently with the filled and reference images. Scenario #2 No MI Help MI help will be displayed concurrently with the filled and reference images. No MI Help No MI Help No MI help will be presented during the verification tasks No MI Help Scenario #2 No MI help will be presented during the verification tasks Scenario #1 No MI Help MI help will be presented in the form of a pop-up message the participant's decision differs from the MI's determination. Scenario #1 Scenario #1 MI help will be presented in the form of a pop-up message the participant's decision differs from the MI's determination.
- Primary Outcome Measures
Name Time Method Reaction time 1 day - Single study visit Difference in task time measured by the number of seconds from starting the task to accepting or rejecting a medication image
Trust 1 day - Single study visit Score on the Muir \& Moray's 100-point trust scale where higher scores indicated greater levels of trust.
Decision accuracy 1 day - Single study visit Difference in detection rate measured by number of medication verification errors
Trust change 1 day - Single study visit Difference in trust as measured by visual analog scale will be calculated based on AI advice accuracy. Participants will indicate their level of trust in the AI advice after every trial on a scale from 1-100, with higher scores indicating greater levels of trust.
- Secondary Outcome Measures
Name Time Method Usability 1 day - Single study visit Usability will be assessed using the System Usability Scale (SUS). SUS scores range from 0 to 100 with a higher scores indicating greater trust.
Cognitive effort 1 day - Single study visit Difference in cognitive effort measured by duration of fixation
Workload 1 day - Single study visit Workload will be measured by NASA Task Load Index
Trial Locations
- Locations (1)
University of Michigan
🇺🇸Ann Arbor, Michigan, United States