Usability Evaluation of Gen AI-based Nutrition Chatbot for Pregnant Women: A Pilot Quasi-experimental Study
Trial Snapshot
- Phase
- Not Applicable
- Status
- Not yet recruiting
- Enrollment
- 100
- Locations
- 1
- Primary Endpoint
- System Usability Scale (SUS)
Study Overview
Brief Summary
Background: Pregnancy imposes significant physical demands, with complications like gestational diabetes (GDM) and pre-eclampsia posing serious risks. Nutrition is crucial for mitigation, but accessing reliable guidance remains challenging. This study evaluates the feasibility of an AI chatbot providing nutritional guidance for managing these conditions.
Methods: In a quasi-experimental design, 100 pregnant women will self-select into either the intervention group (n=50, using an AI chatbot) or control group (n=50, receiving standard care). The primary outcome is usability measured by the System Usability Scale (SUS) at 12 weeks, with an expected mean difference of ≥13 points. Secondary outcomes include technology acceptance (Technology Acceptance Model), user engagement, information accuracy, and changes in dietary knowledge/behaviors. Quantitative data will be analyzed using intention-to-treat and t-tests. Semi-structured interviews with 20 participants will explore user experiences through thematic analysis.
Expected Results: The AI chatbot is anticipated to demonstrate superior usability and high user acceptance (TAM >5.0/7), with improvements in dietary knowledge and behavior. Qualitative findings will provide insights into benefits, barriers, and engagement factors.
Conclusion: This study will establish an evidence base on AI chatbot feasibility and acceptance for prenatal nutrition, informing tool optimization and future large-scale trials.
Detailed Description
Objectives: This study primarily aims to evaluate the usability of a nutrition AI chatbot for pregnant women by comparing System Usability Scale (SUS) scores between intervention and control groups. Secondary objectives include assessing technology acceptance (Technology Acceptance Model), engagement patterns, information quality (accuracy, comprehensibility, consistency), and changes in nutritional knowledge.
Design: A quasi-experimental design with two parallel groups (n=50 each) will be employed. Using self-selection, participants will choose to enroll in the intervention group (access to an AI chatbot plus routine care) or the control group (access to a standardized WeChat information service plus routine care). Routine care for all participants includes standard prenatal clinic visits and printed nutritional materials.
The WeChat service for the control group will be operated by a trained research assistant using a pre-defined script during two scheduled windows daily, providing information quoted from the official nutritional leaflets. This isolates the mode of information delivery (AI versus human-facilitated messaging) as the primary variable.
Participants: Inclusion criteria: pregnant women aged ≥18 years, able to consent, owning a smartphone with internet access. Exclusion criteria: enrollment in other nutrition interventions or severe mental health conditions impairing technology use. A purposive subsample of 20 participants (10 per group) will complete qualitative interviews.
Sample Size: Based on an expected mean SUS score of 78 (SD=12) in the intervention group and 65 (SD=15) in the control group (Cohen's d=0.95), 23 participants per group are required for 90% power at alpha=0.05. Accounting for 50% attrition, 50 participants per group will be recruited. Propensity score matching will be applied to reduce selection bias using variables including age, gestational age, parity, education, and baseline technology use.
Study Design
- Study Type
- Interventional
- Allocation
- Non Randomized
- Intervention Model
- Parallel
- Primary Purpose
- Supportive Care
- Masking
- None
Eligibility Criteria
- Ages
- 18 Years to — (Adult, Older Adult)
- Sex
- Female
- Accepts Healthy Volunteers
- No
Inclusion Criteria
- •Pregnant women aged 18 years or older
- •Able to provide informed consent in the study language
- •Own a smartphone with internet access and the WeChat application
Exclusion Criteria
- •Current enrollment in other nutrition intervention studies
- •Severe mental health conditions that may impair technology use or ability to provide informed consent
Arms & Interventions
The intervention group
The intervention group will access the the nutrition AI chatbot.
Intervention: a culturally tailored nutrition AI chatbot for pregnant women (Behavioral)
The control group
The control group will receive routine care along with access to a standardized WeChat information service. To ensure a fair comparison, the WeChat service for the control group will be operated by a trained research assistant using a pre-defined script and protocol. The assistant will respond to enquiries during two pre-scheduled windows per day (e.g., 10:00-12:00 and 14:00-16:00) by providing information directly quoted or paraphrased from the official nutritional leaflets.
Outcomes
Primary Outcomes
System Usability Scale (SUS)
Time Frame: 12weeks
Usability will be assessed using the System Usability Scale (SUS), a 10-item questionnaire with five-point Likert responses. SUS yields a total score ranging from 0 to 100, with higher scores indicating better perceived usability. Scores will be compared between groups at 12 weeks.
Secondary Outcomes
- Mean Score on the Technology Acceptance Model (TAM) Questionnaire(12 weeks)
- Proportion of Participants Achieving Adequate Engagement Adherence(12 weeks)
- Mean Number of Platform Logins per Week(12 weeks)
- Mean Number of Queries Submitted per Participant(12 weeks)
- Proportion of Chatbot Responses Rated as Accurate by Clinical Expert Panel(12 weeks)
Investigators
Dr Bronya LUK Hi Kwan
Assistant Professor
Hong Kong Metropolitan University