How Do We Evaluate Health in All Policies? Comment on “Developing a Framework for a Program Theory-Based Approach to Evaluating Policy Processes and Outcomes: Health in All Policies in South Australia”

Abstract

It is well-established that population health is influenced by a multitude of factors, many of which lie outside the scope of the health sector. In the public health literature it is often assumed that intersectoral engagement with nonhealth sectors will be instrumental in addressing these social determinants of health. Due to the expected desirable outcomes in population health, several countries have introduced Health in All Policies (HiAP). However, whether this systematic, top-down approach to whole-of-government action (which HiAP entails) is efficient in changing government policies remains unclear. A systematic evaluation of HiAP is therefore much needed. Lawless and colleagues present an evaluation framework for HiAP in their article: “Developing a Framework for a Program Theory-Based Approach to Evaluating Policy Processes and Outcomes: Health in All Policies in South Australia.” This work is an important endeavor in addressing this problem (of uncertainty as to whether HiAP is effective) and represents an essential contribution to the HiAP literature. Nonetheless, in the spirit of encouraging ongoing reflection on this topic, we wish to highlight some challenges in the presented framework, which may pose difficulties in operationalization. We find that the evaluation framework faces two main limitations: its unclear causal logic and its level of complexity. We argue that in order to function as a tool for evaluation, the framework should be explicit about the mechanisms of change and enable us to trace whether the assumed causal relations resulted in changes in practice. Developing manageable evaluation frameworks, albeit simplified, may then be an important part of cumulating the theoretical insights aspired in theory-based evaluation. On this basis, we highlight how HiAP processes and healthy public policies respectively involve different mechanisms, and thus argue that different program theories are needed. 

Keywords: Health in All Policies, Theory-Based Evaluation, Intersectoral Policymaking, Policy Process