Evidence-Based Policy
The Methodology of Evidence-Based Policy
There are many methodologies for evidence-based policy but they all share the following characteristics:
-
Tests a theory as to why the policy will be effective and what the impacts of the policy will be if it is successful
-
Includes a counterfactual: what would have occurred if the policy had not been implemented
-
Incorporates some measurement of the impact
-
Examines both direct and indirect effects that occur because of the policy
-
Separates the uncertainties and controls for other influences outside of the policy that may have an effect on the outcome
-
Should be able to be tested and replicated by a third party
The form of methodology used with evidence-based policy fit under the category of a cost-benefit framework and are created to estimate a net payoff if the policy was to be implemented. Because there is a difficulty in quantifying some effects and outcomes of the policy, it is mostly focused broadly on whether or not benefits will outweigh costs, instead of using specific values.
Science for Policymakers
Evidence-Based Policy is public policy informed by rigorously established objective evidence. It is an extension of the idea of evidence-based medicine to all areas of public policy. An important aspect of evidence-based policy is the use of scientifically rigorous studies such as randomized controlled trials to identify programs and practices capable of improving policy-relevant outcomes. Good data, analytical skills and political support, as such, are seen as the important elements
Although evidence-based policy can be traced as far back as the fourteenth century, the Blair Government in the United Kingdom more recently popularized it. The Blair Government said they wanted to end the ideological led-based decision making for policymaking. For example, a UK Government white paper published in 1999 ("Modernizing Government") noted that Government must "produce policies that really deal with problems, that are forward-looking and shaped by evidence rather than a response to short-term pressures; that tackle causes not symptoms"
The term "evidence-based policy" has evolved from "evidence-based medicine,” in which research findings are used as the support for clinical decisions and evidence is gathered by randomized controlled trials (RCTs), which is comparing a treatment group with a placebo group to measure results. In 1993, the Cochrane Collaboration was established in the UK, and works to keep all RCTs up-to-date and provides.” Cochrane reviews" provides primary research in human health and health policy. There was then an increase in research and policy activists pushing for more evidence-based policy-making, which led to the formation of the sister organization to Cochrane Collaboration, the Campbell Collaboration in 1999. The Campbell Collaboration conducts reviews on the best evidence that analyzes the effects of social and educational policies and practices.
The Economic and Social Research Council (ESRC) became involved in the push for more evidence-based policy making with its 1.3 million pound grant to the Evidence Network in 1999. The Evidence Network is a center for evidence-based policy and practice and is similar to both the Campbell and Cochrane Collaboration.
More recently, the Alliance for Useful Evidence has been established, with funding from ESRC, Big Lottery, and Nesta, to champion the use of evidence in social policy and practice. The Alliance is a UK-wide network that promotes the use of high quality evidence to inform decisions on strategy, policy, and practice through advocacy, publishing research, sharing ideas and advice, and holding events and training.
Precedent from Medicine: Rigorous studies – particularly randomized controlled trials– have led to remarkable improvements in human health over the past 50 years.
Well-conducted trials have stunned the medical community by overturning widely-accepted practices, such as hormone replacement therapy for post-menopausal women (shown to increase the risk of stroke and heart disease for many women), and stents to open clogged arteries (shown no better than drugs for most heart patients). Such trials have also provided the conclusive evidence of effectiveness for most of the major medical advances of the past half-century, including vaccines for polio, measles, and hepatitis B; effective treatments for hypertension and high cholesterol; and cancer treatments that have dramatically improved survival rates from leukemia, Hodgkin’s disease, breast cancer, and many other cancers.
Key Lessons of Evidence Based Policy-making
Six key lessons have been developed, which are:
-
Policy processes are complex and rarely linear or logical and simply presenting information to policy-makers and expecting them to act upon it is very unlikely to work. Policy processes are not purely linear as they have various stages that each take varying lengths of time to complete and may in fact be conducted simultaneously. Strategies must be fluid.
-
Policy is often only weakly informed by research-based evidence due to information gaps, secrecy, and the need for speedy responses, political expediency, and the fact that policy makers are rarely scientists.
-
Research-based evidence can contribute to policies that have a dramatic impact on lives. Success stories quoted in the UK's Department for International Development's (DFID) new research strategy include a 22% reduction in neonatal mortality in Ghana as a result of helping women begin breastfeeding within one hour of giving birth, and a 43% reduction in deaths among HIV positive children using a widely available antibiotic.
-
The need for a holistic understanding of the context in which the policy is to be implemented.
-
Policy entrepreneurs need additional skills to influence policy. They need to be political fixers, able to understand the politics and identify the key players. They need to be good storytellers, able to synthesize simple compelling stories from the results of the research. They need to be good networkers to work effectively with all the other stakeholders, and they need to be good engineers, building a program that pulls all of this together.
-
Policy entrepreneurs need clear intent – they need to really want to do it. Turning a researcher into a policy entrepreneur, or a research institute or department into a policy-focused think tank involves a fundamental re-orientation towards policy engagement rather than academic achievement; engaging much more with the policy community; developing a research agenda focusing on policy issues rather than academic interests; acquiring new skills or building multidisciplinary teams; establishing new internal systems and incentives; spending much more on communications; producing a different range of outputs; and working more in partnerships and networks.
These lessons show that the relationship between research, policy, and practice is complex, multi-factorial, non-linear, and highly context specific. What works in one situation may not work in another. Developing effective strategies in complex environments is not straightforward. Simple tools such as cost–benefit analysis, logical frameworks, traditional project management tools and others may not work on their own, as they fail to take into account the existing complexity.
Based on research conducted in six Asian and African countries, the Future Health Systems consortium has identified a set of key strategies for improving uptake of evidence in to policy, including: improving the technical capacity of policy-makers; better packaging of research findings; use of social networks; establishment of fora to assist in linking evidence with policy outcomes.
.
