Abstract
Background: Calls for evidence-informed public health policy, with implicit promises of greater program effectiveness,
have intensified recently. The methods to produce such policies are not self-evident, requiring a conciliation of values
and norms between policy-makers and evidence producers. In particular, the translation of uncertainty from empirical
research findings, particularly issues of statistical variability and generalizability, is a persistent challenge because of the
incremental nature of research and the iterative cycle of advancing knowledge and implementation. This paper aims to
assess how the concept of uncertainty is considered and acknowledged in World Health Organization (WHO) policy
recommendations and guidelines.
Methods: We selected four WHO policy statements published between 2008-2013 regarding maternal and child nutrient
supplementation, infant feeding, heat action plans, and malaria control to represent topics with a spectrum of available
evidence bases. Each of these four statements was analyzed using a novel framework to assess the treatment of statistical
variability and generalizability.
Results: WHO currently provides substantial guidance on addressing statistical variability through GRADE (Grading of
Recommendations Assessment, Development, and Evaluation) ratings for precision and consistency in their guideline
documents. Accordingly, our analysis showed that policy-informing questions were addressed by systematic reviews
and representations of statistical variability (eg, with numeric confidence intervals). In contrast, the presentation of
contextual or “background” evidence regarding etiology or disease burden showed little consideration for this variability.
Moreover, generalizability or “indirectness” was uniformly neglected, with little explicit consideration of study settings
or subgroups.
Conclusion: In this paper, we found that non-uniform treatment of statistical variability and generalizability factors that
may contribute to uncertainty regarding recommendations were neglected, including the state of evidence informing
background questions (prevalence, mechanisms, or burden or distributions of health problems) and little assessment of
generalizability, alternate interventions, and additional outcomes not captured by systematic review. These other factors
often form a basis for providing policy recommendations, particularly in the absence of a strong evidence base for
intervention effects. Consequently, they should also be subject to stringent and systematic evaluation criteria. We suggest
that more effort is needed to systematically acknowledge (1) when evidence is missing, conflicting, or equivocal, (2) what
normative considerations were also employed, and (3) how additional evidence may be accrued