Evaluating the programmatic and economic impact of digital health solutions for immunization (eIR and eLMIS) in LMICs

A consortium of researchers in Guinea, Honduras, Rwanda and Tanzania led by the SDA Bocconi School of Management and MM Global Health Consulting (MMGH Consulting) have conducted evaluations of the programmatic and economic impact of using electronic immunization registries (eIR) and electronic logistics management information systems (eLMIS) for immunization.*

With the increasing digitalization of health systems in low- and middle-income countries (LMICs), there is growing interest from governments, donors, and implementing partners to introduce and scale-up digital systems to support more effective and efficient immunization service delivery. Upon the request by the Bill and Melinda Gates Foundation (BMGF), WHO, and Gavi, the Vaccine Alliance, a multi-country evaluation was performed with the overall aim of generating robust actionable evidence to enable future decisions on the introduction and scale-up of eIR and eLMIS in LMICs.

This evaluation was conducted in four LMICs, Guinea, Honduras, Rwanda and Tanzania, by local Principal Investigators (PIs) and their research institutions which planned, conducted and managed the fieldwork. The initial data cleaning and analyses was performed in close collaboration with the Center for Research on Health and Social Care Management (CERGAS) at SDA Bocconi School of Management, Bocconi University and MMGH Consulting. Additional joint analysis guided the development and finalization of report for each country, as well as this cross-country report. Local IPs and research institutions included: Africa Health Consulting in Guinea, senior consultants Edith Rodriguez and Luis Castillo in Honduras, the Centre for Impact Innovation and Capacity Building for Health Information and Nutrition (CIIC-HIN) in Rwanda and the National Institute for Medical Research – Mbeya Medical Research Center (NIMR-MMRC) in Tanzania.

While current evidence suggests that these electronic tools can contribute to improved data quality and use, and ultimately positively impact the functioning of immunization programs, many were not rolled out at scale. Similarly, while evaluations to date have explored critical factors influencing the success of their implementation, they have offered little insights into their costing and affordability. Where innovations around digitalization have failed, it was often because the specific country context, user requirements and/or issues related to interoperability with existing health management information systems (HMIS) were overlooked. Importantly, no impact has been observed from technological interventions alone. Other parallel activities, including capacity-building of health workers and change management for the sustained use of new technologies, were deemed essential (WHO, 2021).

As such, this evaluation further explored those critical areas and aimed to generate evidence on the practical experience of implementing these systems and on their effectiveness, affordability and sustainability. Four specific country reports exploring the challenges and opportunities around developing and implementing these digital systems, the associated costs and the programmatic and economic impact in each unique context have been developed. A cross-country report synthesizes the main insights from four diverse contexts, draws common lessons learned and posits recommendations for future investments, both domestic and external, in digital systems to improve immunization service delivery.