Policymakers across the UK government say that if we want to provide citizens with meaningful change, then we must get evidence gathering and evaluation right when delivering public policies and services.
But even though policymakers think these parts of the process are the most important, many of the same people say that it often doesn’t happen.
Evidence provides certainty that the policy intent will be delivered
Policymakers say they are often given a solution and asked to implement it, but the solution doesn't always meet the needs of people who will use the policy or service. The ideas are rarely tested with those that will use them before they are implemented, so policymakers cannot be sure they will have the intended effect.
“We often don’t pilot stuff ...you pilot big policies” Policymaker in workshop
If a policymaker doesn't test their idea before delivery, then they are storing up risk that the intervention will fail by not delivering the intended outcome. They may not know that policy or service has failed until years after.
Serious consequences to not evaluating
Evaluation indicates if the policy or service is delivering its intended outcome. It is a feature of all theoretical policy making models and policymakers recognise its value, but it is often not done and policymakers think they should do it more.
Where it is done, evaluation is typically a single event which occurs years (policymakers frequently mentioned 5 years) after the policy or service has been implemented. But it is rare that the policymaker is present 5 years later to receive insight from the evaluation of their work or that it can inform the live policy or service.
“Poor measurement of what projects achieve, reduces accountability and transparency for government and Parliament, and makes it difficult to assess whether the costs of projects are justified. It also means that government is missing an opportunity to learn about what constitutes success.” National Audit Office, 2018
Make small and frequent tests normal
By testing ideas early, policymakers are more likely to mitigate risk early and establish assurance that the policy or service will land well and be adopted by people who will use it, like citizens.
But there is a perverse incentive infrastructure for testing. Policymakers associate a failed test as a personal or professional failure, rather than proving or disproving an uncertain aspect of a policy or service. This might be because policymakers associate testing with ‘piloting’. These tend to be large scale, infrequent and high profile.
“It always gets presented in the press as failure or a U-turn, as opposed to ‘no, this is us testing a service and that bit didn’t work so now we’ll fix it'. That’s part of the process of improving the service" Senior policymaker, large department
This might be addressed by normalising tests to be smaller and more frequent.
A good way to design a solution is to determine the intent, then understand the problem and the needs of the people that will use the policy or service, and then design and test the ideas. Once you are sure that it will work, you can build and deliver the policy or service with confidence.
“Rapid policy prototyping at the early stages of the policy cycle, design approaches can de-risk delivery further down the line” Anna Whicher, Cardiff Metropolitan University, 2020
"Prototypes are tested not only in terms of their technical robustness and effectiveness, but also of their fit with users’ needs" Nick de Leon, Royal College of Arts, 2018
Take action to improve public value
The UK is getting serious about delivering better public value. HM Treasury is working with government departments to ratchet up improvements through its Public Value Framework.
Our research with policymakers in 2020 independently came to very similar conclusions.
Everyone can contribute to improving public value by testing and prototyping your ideas early and regularly. Involve people who will use the policy or service in designing, testing and iterating the solution. As soon as your policy or service is operational, be sure it is having the intended effect by keeping a live data feed on its performance that monitors things like:
- Increased satisfaction with a policy or service
- Increased uptake of a policy or service
- Increased completion rate of a service
- Increased certainty that the policy or service will deliver the intended outcome
- Increased public value
Join our community
You can find out more about our research by downloading our report. We use this blog to talk about the work of the multidisciplinary policy design community. We intend to share more stories about our work, the thinking behind it and what policymaking might look like in the future. If you would like to read more, then please sign-up for updates. Join the conversation by commenting below.
Leave a comment