Estimators Used in Multisite Healthcare Costing Studies in Low- and Middle-Income Countries: A Systematic Review and Simulation Study
Emma Clarke-Deelder, MPhil, Anna Vassall, PhD, Nicolas A. Menzies, PhD
Background: In low- and middle-income countries, multisite costing studies are increasingly used to estimate healthcare program costs. These studies have employed a variety of estimators to summarize sample data and make inferences about overall program costs.
Objective: We conducted a systematic review and simulation study to describe these estimation methods and quantify their performance in terms of expected bias and variance.
Methods: We reviewed the published literature through January 2017 to identify multisite costing studies conducted in low- and middle-income countries and extracted data on analytic approaches. To assess estimator performance under realistic conditions, we conducted a simulation study based on 20 empirical cost data sets.
Results: The most commonly used estimators were the volume-weighted mean and the simple mean, despite theoretical reasons to expect bias in the simple mean. When we tested various estimators in realistic study scenarios, the simple mean exhibited an upward bias ranging from 12% to 113% of the true cost across a range of study sample sizes and data sets. The volume-weighted mean exhibited minimal bias and substantially lower root mean squared error. Further gains were possible using estimators that incorporated auxiliary information on delivery volumes.
Conclusions: The choice of summary estimator in multisite costing studies can significantly influence study findings and, therefore, the economic analyses they inform. Use of the simple mean to summarize the results of multisite costing studies should be considered inappropriate. Our study demonstrates that several alternative better-performing methods are available.
A new systematic review and simulation study discussed cost estimators. Simple mean has an upward bias from 12% to 113%. To reduce bias, use volume-weighted mean, calibration, or regression estimators.
The EPIC project is proud to announce the Sample Design Optimizer (SDO) instrument. This instrument aims to assist research teams develop cost-effective and efficient sample design to maximize expected precision within a data collection budget constraint.
A new working paper provides a practical guide on immunization costing primary data collection and analysis. We would like to invite the Immunization Economics community to explore the working paper and provide feedback
Analyses estimating program costs collected on a subsample of facilities are subject to statistical uncertainty, often quantified via standard errors and/or 95% confidence intervals. This recent publication elaborates methods to reduce statistical uncertainty.
In the October 2017 Newsletter Issue, Dr. Brenzel thanks and welcomes the members of the Immunization Economics Community of Practice. Read her welcome letter here.
Using a unique pooled dataset of 316 health facilities in six countries, EPIC project analyses describe differences in unit cost estimates assocated with service volume, delivery methods, immunization coverage, staff training, seniority, and site location. Access datasets and paper here.
Health Policy & Planning podcast discusses the importance in understanding detailed cost structure, its relationship with efficiency, and implications for health policy and future planning in immunization programs. Listen to podcast here.
Health Policy and Planning blogpost examines how detailed understanding of the cost structure of immunization programs can inform policy. Read more here.