Evidence-based policy making relies on robust, diverse research insights to inform policy ideas and decision making. But that’s easier said than done, particularly for complex policy challenges with evidence that contains many different actors and factors influencing the outcomes.
This is the first of a two-part blog post where we’ll explain the combination of methods we’ve tested to review and make sense of a complex evidence base with a wider group of people on our programme.
In this first post, we’ll describe the challenges of evidence-based policy making we’ve experienced on the Farming and Countryside Programme and explain the steps we took to prepare for a participatory evidence mapping workshop. In the second post, we’ll explain how we designed, ran and analysed the evidence mapping workshop, combining systems thinking and serious games.
Why is evidence-based policy making challenging?
Evidence-based policy making is an important aspiration referenced in the HM Treasury's Green Book, but it presents practical challenges for policy teams working under pressure which we’ve experienced first-hand on our programme:
- There is often an overwhelming volume of evidence for policy teams to consider in a short space of time
- Policy teams might not be aware of all the evidence sources on a particular subject, which can come from multiple government research professions, disciplines and teams, including but not limited to: user research, social research, behavioral science, and customer insights
- The granular insights that policy teams need for evidence-based policy making are often contained within long research reports and slide decks. Even with access to the right documents, it is labour intensive for policy teams to extract and synthesis insights
- For complex policy areas (like farming and land management), policy teams can struggle to build a collective understanding of the complex, inter-related nature of the insights within the evidence base
These challenges present risks for policy teams. Without a holistic view of the evidence, we can introduce assumptions and bias. For complex policy areas, a lack of methods to collectively make sense of complexity in the evidence base, can lead to siloed policy ideas and interventions which don’t make sense in delivery, for end users or frontline workers.
Equally, for research colleagues, they can experience frustration if policy colleagues aren’t aware of their work which can lead to duplication of effort if new research is commissioned without knowledge of existing evidence. This also risks undermining trust in government if citizens are engaged in research on the same issues multiple times.
How can policy teams gather robust and complex evidence bases and make sense of them?
In response to these challenges faced by our policy teams, we asked ourselves:
- What if policymakers could easily engage with all the evidence a programme holds?
- What if they could make sense of relationships between different pieces of evidence?
- What if they could use that understanding to identify opportunity areas to test a range of complementary government policy ideas and interventions?
To address these challenges and support our policy colleagues in their work, we designed and tested a four-phase participatory methodology to allow our policy team to make sense of their complex and extensive evidence base and come up with a portfolio of policy ideas to test and learn from. We’ll explain the first two stages of this methodology below which cover the evidence gathering activities. In the next blog we’ll explain the evidence mapping workshop itself.
Evidence discovery
Inspired by Policy Lab’s collaborative evidence discovery method, we first hosted an online evidence discovery workshop, inviting any colleagues on our programme who might have research related to our policy challenge.
We used an online whiteboard tool to populate an ‘evidence gallery’ with all the evidence the policy team currently held. In the workshop, we allowed time for everyone to explore and read through the evidence gallery, inviting them to add any evidence or specific insights they thought were missing
This first workshop helped us to quickly grow a more robust and diverse evidence base, drawing on the breadth and diversity of knowledge, experience and research disciplines in our programme.
Insight ‘nuggets’
After the online evidence discovery workshop, we were inspired by Tomar Sharon and Benjamin Gadbaw’s work on the atomic unit of a research insight. They argue that insight ‘nuggets’ (rather than research reports) are the most useful, actionable unit of research. And they’ve shown how research operations can evolve to extract insight ‘nuggets’ that become readily available to organisations to answer a whole range of questions and challenges over time.
We distilled our evidence into an online searchable database of 150 individual insights, tagged with themes, and with referenced sources. We now had a much richer and more accessible evidence base, that we could use to underpin the activities.
To enable a group of people to use these insight ‘nuggets’ in our evidence mapping workshop we printed them out as individual ‘evidence cards’. This physical, tactile format allowed the evidence to be easily and quickly socialised and deliberated among participants in a more nuanced and dynamic way compared to our experiences attempting to run to similar activities online.
In our next blog post we will detail how we worked with professional games designer Matteo Menapace to design our participatory ‘evidence mapping’ workshop, that allowed 36 colleagues with little or no prior knowledge of systems mapping to explore the 150 insights nuggets and build their own maps.
If you would like to find out more about this project, or others undertaken by Defra’s Policy Design Lab, get in touch! Equally we’d love to hear about any similar approaches you’ve been testing for evidence collection and review. Please email policydesignlab@defra.gov.uk.
Join our community
We use this blog to talk about the work of the multidisciplinary policy design community. We share stories about our work, the thinking behind it and what policymaking might look like in the future. If you would like to read more, then please subscribe to this blog. If you work for the UK's government, then you can you join the policy design community. If you don't work for the UK government, then connect with us on social media at Design and Policy Network and subscribe here to be notified about our monthly speaker events to hear from influential design thought leaders and practitioners.
1 comment
Comment by Millie Chan posted on
Amazing work Becky, thoroughly enjoyed the workshop