Usability Evaluation of EpiGyaan: An AI-Powered Educational Chatbot for Learning Epidemiology

Title: Usability Evaluation of ’EpiGyaan’: An AI-Powered Educational Chatbot for Learning Epidemiology.

Introduction

Epidemiology is a field of study devoted to the study of distribution and determinants of diseases or health events in a population[1]. Epidemiology is the core subject of study in Public Health as well as Community Medicine specialties[2]. Epidemiology being based on mathematical principles is often difficult to understand and grasp. AI Chatbots which enable self-paced learning are known to help learning[3]. Large language Models trained on large data have the ability to provide authentic, updated and curated information to the learner[4,5].

EpiGyaan is an AI-powered educational chatbot developed using Python, Streamlit, and OpenAI LLM to assist students of epidemiology, faculty and health professionals in learning epidemiological concepts. This innovative digital solution integrates PubMed references, R programming code snippets, and LaTeX-rendered epidemiological formulas.

The Chatbot has been developed by the investigator of this study and is proposed to be copyrighted (The python source code and app interface). The proposed study will evaluate the usability, functionality, and user satisfaction of the EpiGyaan chatbot.

Objectives

1.     To assess the ease of navigation and user interface of the EpiGyaan app.

2.     To evaluate clarity of instructions provided by the chatbot.

3.     To measure overall user satisfaction with the app experience.

4.     To identify areas for improvement in the user interface and app functionality.

Materials and Methods

Study Design: Cross-sectional usability evaluation

Study Site: Goa Medical College, Goa

Study Population: postgraduate residents and faculty in Community Medicine at Goa Medical College.

Study Duration: 6 months following approval from the Institutional Ethics Committee

Sampling

Sample Size: 30 participants, convenience sampling. (as required for a usability evaluation study)

Eligibility Criteria

Inclusion Criteria: Postgraduate students, or faculty in Community Medicine. Voluntary participation.

Exclusion Criteria: Unwillingness to provide feedback.

Data Collection Methods

Participants will interact with the EpiGyaan chatbot, complete predefined tasks, and subsequently complete a structured usability evaluation survey.

Usability Evaluation Matrix

Ease of Navigation (Likert Scale 1-5)

Instructions Clarity (Likert Scale 1-5)

Feature Utility (Likert Scale 1-5)

Technical Performance (user-reported issues)

Overall User Satisfaction (Likert Scale 1-5, plus open-ended feedback)

Data Management and Statistical Analysis

Descriptive statistics will summarize quantitative responses (mean, standard deviation). Qualitative feedback will be analyzed using thematic analysis methods.

Ethical Considerations

Written informed consent will be obtained from all participants. Participation is voluntary, anonymous, and confidentiality is assured. Ethical clearance will be obtained from the Institutional Ethics Committee, Goa Medical College.

Conflict of Interest

Dr. Frederick Vaz declares membership in the Institutional Ethics Committee, Goa Medical College.

Funding

The study is self-funded with no external financial support.

Statement of Confidentiality and data privacy: All data collected during the study will be kept strictly confidential and stored in password secured computers.

References:

1.         Celentano DD, Szklo M, Gordis L. Gordis epidemiology. 2019.

2.         National Medical Commision. Guidelines for Competency Based Postgraduate Training Programme for MD in Community Medicine. [Internet]. 2018 [cited 2025 Aug 3]. Available from: https://www.nmc.org.in/information-desk/for-colleges/pg-curricula-2/

3.         Han JW, Park J, Lee H. Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study. BMC Med Educ. 2022 Dec 1;22(1):830.

4.         Chan KY, Yuen TH, Co M. Using ChatGPT for medical education: the technical perspective. BMC Med Educ. 2025 Feb 7;25(1):201.

5.         Non LR. All aboard the ChatGPT steamroller: Top 10 ways to make artificial intelligence work for healthcare professionals. Antimicrob Steward Healthc Epidemiol. 2023;3(1):e243.