Evaluation of a Chatbot With CALMO (Chatbot-Assisted Learning Model) to Enhance Clinical Learning in Nursing Students

Not Applicable

Not yet recruiting

• Conditions: Clinical Competence

• Registration Number: NCT06780228
• Lead Sponsor: Caritas Institute of Higher Education

Brief Summary

The present proposed study will design and develop a chatbot to support graduating nursing students' clinical learning. The efficacy of this chatbot technology will be evaluated, using a mixed methods quasi-experimental design. the proposed study hypothesizes that the chatbot-assisted clinical education has the potential to promote clinical performance by improving nursing knowledge, general self-efficacy, and self-reflection and insight.

Detailed Description

The present proposed study will design and develop a chatbot to support graduating nursing students' clinical learning. The objectives of the proposed study are: (1) Design and develop a chatbot prototype that can simulate conversations, provide clinical information, and offer guidance relevant to various clinical scenarios; (2) Evaluate the impact of the chatbot intervention on nursing students' nursing knowledge, general self-efficacy, and self-reflection and insight; (3) Explore students' perceptions, satisfaction, and acceptance of the chatbot as an educational tool.The efficacy of this chatbot technology will be evaluated, using a mixed methods quasi-experimental design. the proposed study hypothesizes that the chatbot-assisted clinical education has the potential to promote clinical performance by improving nursing knowledge, general self-efficacy, and self-reflection and insight. The proposed study will be conducted in two phases. Phase 1: Design and develop a chatbot prototype which can provide evidence-based clinical information, engaging in interactive dialogue, and offering learning resources tailored to different clinical scenarios. The chatbot will be accessible through a web-based platform or mobile application. Phase 2: Conduct a mixed-methods quasi-experimental design and focus group interviews to evaluate the impact of the chatbot intervention on clinical learning. A convenience sample of 174 nursing students enrolled in a nursing program from two institutions will be recruited and assigned to either the experimental group, which will receive chatbot-supported clinical learning, or the control group, which will receive traditional clinical education without chatbot integration. Quantitative data will be collected from the two groups at three-time points: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2) to assess students' nursing knowledge, general self-efficacy, and self-reflection and insight. Qualitative data will be collected through focus group interviews to explore students' perceptions, satisfaction, and experiences with chatbot-assisted learning. Quantitative data will be analyzed using MANOVA to compare the outcomes between and within groups. Thematic analysis will be conducted for qualitative data to identify emerging themes and patterns related to student perceptions and experiences using chatbot. Participants in the Experimental Group (EG) will receive an access code for the self-developed web-based chatbot platform called CALMO, a gender-neutral name as an acronym for Chatbot-Assisted Learning Model, which is planned to be developed during the first phase of this proposed study. The intervention will be available throughout the 12-week study.Participants in the Control Group (CG) will receive their usual clinical education support during the intervention period.A project management team will be formed to evaluate the intervention and ensure the quality and consistency of study intervention via regular face-to-face or online meetings. The fidelity of the study will be ensured by developing mechanisms for tracking participant logins, session completion rates, and engagement with intervention materials. Regular checks of the chatbot functionality will be performed by the platform developer with provision of regular service reports. Communication channels will be established for participants to provide feedback, report technical issues, or seek support from the research team during the intervention. Data will be analyzed using a mixed-methods approach, combining both quantitative and qualitative data. The statistical analysis using Statistical Package for the Social Sciences (SPSS), Version 26. The baseline characteristics of the participants in the two arms will be compared to identify any clinically meaningful differences using Chi-square test (for nominal data), Mann-Witney U test (for ordinal or non-normality data) or t-test (for normality data). Any variables considered to be significantly different will be further evaluated (e.g., included as covariates in the statistical analyses) and adjusted to account for their potential confounding effects on the outcomes. To test the hypothesis and to cater for the clustering effect with repeated measures of the outcome variables, a multivariate approach, such as a Generalized Estimating Equation (GEE), with adjustment of the baseline outcome variables to test for between-group differences will be employed. If students drop out between baseline and final data collection, intention to treat analysis will be applied. Both the Time x Group effect will be tested using a two-sided type I error rate of 5% for all tests of significance. Thematic analysis will be conducted for qualitative data to identify emerging themes and patterns related to students' perceptions and experiences.

Study Timeline

• Status Verified: 2025-01
• Study Start: 2025-01-11
• Study First Submitted: 2025-01-11
• Study First Submitted QC: 2025-01-15
• Last Update Submitted: 2025-01-15
• Study First Posted: 2025-01-17
• Last Update Posted: 2025-01-17
• Primary Completion: 2026-09-30
• Study Completion: 2026-12-30

Recruitment & Eligibility

• Status: NOT_YET_RECRUITING
• Sex: All
• Target Recruitment: 174

Inclusion Criteria

• Baccalaureate nursing students participating in their final clinical placement
• have sufficient knowledge of English language
• own a smartphone with internet access
• possess a valid Hong Kong phone number
• possess a valid email address

Exclusion Criteria

• those who have been involved in the pilot test of the proposed chatbot intervention

Study & Design

• Study Type: INTERVENTIONAL
• Study Design: PARALLEL

Primary Outcome Measures

Name	Time	Method
nursing knowledge	before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).	multiple-choice questionnaire (MCQ) consisting of 50 items will be administered to evaluate students' nursing knowledge. the score ranges from 0 to 50. the higher the score, the better nursing knowledge.
General self-efficacy	before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).	A 10-item General Self-Efficacy Scale to assess a general sense of perceived self-efficacy to manage hassles. It is scored with possible responses, 1=not at all true; 2=hardly true; 3=moderately true; and 4=exactly true, yielding a total score between 10 and 40. The higher the score, the higher the optimistic to cope with daily hassles.

Secondary Outcome Measures

Name	Time	Method
Self-reflection and insight	: before-intervention at baseline (T0), 4 weeks after baseline (week 4, T1) and after intervention (week 12; T2).	A 12-item Self-Reflection and Insight Scale to assess the need for self-reflection and engagement in self-reflection. The scale is scored from 6 (highly matched) to 1 (least matched), yielding a total score between 12 and 72. Higher scores indicated greater self-reflection and insight.
Qualitative Comments on the Perceptions of Chatbot-assisted technology	at the end of the 12-week study	it is a narrative feedback from the participants using focus group interviews to explore users' experiences with the chatbot during their clinical learning. A set of semi-structured interview questions encompassing questions to describe the overall experience, identify useful features, and assess the impact on understanding and retention of clinical concepts. Usability and user-friendliness of the chatbot interface will also be evaluated. For example: Can you describe your experience using the chatbot for clinical learning? What specific features or functionalities did you find useful or engaging? How do you think the chatbot has contributed to your understanding and retention of clinical concepts? Are there any limitations or challenges you encountered while using the chatbot? Did you perceive any differences in your learning outcomes when using the chatbot compared to other learning methods or resources? If so, what were those differences?