Impact Evaluation of Maternal Health Information Messaging in India

The Johns Hopkins Global mHealth Initiative, Oxford Policy Management, and University of Cape Town proposes supporting BBC Media Action and the Ministry of Health and Family Welfare (MOHFW) to conduct an external evaluation of Mobile Academy (MA) and Kilkari. In the below text, we outline 6 proposed activities which comprise the evaluation, which are planned from mid 2018 to April 2020.

These studies will help inform program modification and scale-up plans, and will help better understand the effects of maternal mobile messaging in India and globally.

1. Secondary analysis of cross-site trends in key performance indicators (Duration: throughout study):

Mobile Academy (MA): We will use system generated data to estimate the proportion of ASHAs who have completed MA training across 12 states. We will additionally aim to understand linkages between ASHA's exposure to the MA training content and performance on MA knowledge assessments.

Kilkari: We estimate that Kilkari users will have differing levels of exposure to the program. For example, some women may not be present when the Kilkari calls come and thus miss out on key content. Analyses of system generated data will aim to measure exposure to Kilkari messages by content area (e.g. nutrition, family planning, etc) across 12 states where implementation has occurred to date. As part of this analysis we'll seek to understand reasons why calls are not received, including phones being out of network, changes in SIM cards, etc. 2. Summative evaluation of Kilkari (Period: April 2018 to December 2019) Objective: To determine the effectiveness of mobile maternal health information messages on key health outcomes in 4 districts of Madhya Pradesh (MP).