Better data for better decisions: how to create user-centred national monitoring systems for WASH

on
22 July 2020
Water monitors from Basbedo presenting the data from the water resource monitoring during a village assembly, in the commune of Tenkodogo, in the Centre-East region, Burkina Faso.
WaterAid/ Basile Ouedraogo

In low-income countries, even where water, sanitation and hygiene (WASH) data are collected they are often not used to inform policy decisions. Stuart Kempster discusses the challenges to data-informed policy making in WASH and introduces a guide to help governments and their partners to move towards more user-centred monitoring systems.

Too often, investments in national monitoring systems for WASH fail to have any significant impact on decision making. Our From data to decisions research project showed that to ensure investments in national monitoring drive transformational change in the wider WASH system governments and development partners must build a better understanding of data use by drawing on insights from political economy and behavioural science.

They must take a long-term approach, ensuring monitoring systems are co-designed with the end users of data, and develop processes to incentivise data use and mitigate potential biases, resulting in data-informed planning, budgeting and service delivery.

COVID-19 shows how vital national monitoring systems are to effective policy

The role of data in policy making has perhaps never been more prominent, as governments around the world scramble for evidence to inform their responses to COVID-19. Effective responses can often be seen to pair with strong data systems. For example, the ability of Germany’s public health institute to provide a steady flow of information is said to have been a key factor in the country’s quick response to the pandemic, at both national and local levels.

In places with weaker health information flows, governments have been seeking to build more innovative monitoring approaches to aid their response. For example, the Government of India has partnered with the World Health Organization’s national polio surveillance network to help provide the data they need to tackle COVID-19. The African Union and Africa CDC are supporting states to integrate testing for the virus into existing national surveillance systems.

The pandemic has also shown that national monitoring systems that don’t respond to decision makers’ needs can undermine the response. In the United Kingdom, for example, the ability of local leaders to manage new coronavirus outbreaks has been hampered by gaps in the reporting of infection data for subnational regions. The Mayor of Leicester has cited this gap as a key reason for the city’s delayed policy response to a recent spike in virus cases.

Challenges of national monitoring systems for WASH

The practical challenges of using national monitoring systems as the basis for evidence-informed policy making are not new to those working in the global WASH sector. In 2018, we convened a side session at UNC’s Water and Health conference (PDF) bringing together a wide range of stakeholders to discuss how development agencies can effectively engage in this complex area. Participants raised familiar issues:

  • Collecting data and using it in decision making is inherently political, and this needs to be recognised explicitly.
  • Data collection is too often driven by external factors and processes, culminating in too much data and too many reports. More effort is needed to understand which data are needed by whom, and for what decisions.
  • It is important that sub-national data feeds national processes on an ongoing basis, and similarly that outcomes of national processes are documented and cascaded to sub-national levels where they can then be implemented.
  • There is a particular need to strengthen the links between monitoring programmes and broader sector reform, ensuring data feeds into processes of learning and adaptation.

The story of national WASH monitoring programmes in low-income countries tends to be one of unmet potential; even where data are collected, often they don’t inform decision making. The transformation promised by the greater use of data in planning, budgeting and service delivery is largely not being fulfilled.

Ali Sabo, 51, a water monitor, demonstrating how he uses a rain gauge to monitor rainfall, Dungass, in the department of Dungass, Zinder, Niger, February 2019.
WaterAid/ Basile Ouedraogo
Ali Sabo, a water monitor, demonstrating how he uses a rain gauge to monitor rainfall in Dungass, Zinder, Niger.

What needs to change to improve monitoring?

The World Bank has argued that to overcome these challenges we need to move towards a ‘user-centred data culture (PDF)’. Over the past two years, we have been trying to identify how this can be achieved in the WASH sector. Previous research into sector monitoring (PDF) highlighted that, alongside production of data, it is important to strengthen institutions, processes and incentives. But there has been little discussion of how this can be achieved in practice.

To try to answer this question, we commissioned the Overseas Development Institute to produce a review of evidence-based decision making, drawing on political economy and behavioural science literature.

Combining these insights with evidence on the practicalities of national WASH monitoring programmes, we developed a framework to help actors better understand and strengthen the use of data in policy decisions.

How to strengthen data use in policy making: the data use framework

Step 1: Purpose. What types of decisions are made, and by whom?

Much of the sector monitoring literature refers to ‘a decision’ without specifying the purpose of that decision and who is involved in making it. However, behavioural science shows that how we reason is strongly linked to the purpose of a decision. It is therefore important that the first step of the framework seeks to identify specific users of data and their specific uses – that is, the decisions or activities that data can inform.

Step 2: Context. What are the features of the institutional and political environment in which these decisions are made?

The importance of the context in which decisions are made is stressed by both political economy and behavioural science. From the perspective of the specific users and uses of data, the second step analyses the clarity of institutional arrangements, processes for planning and budgeting, and political priorities within the WASH sector and beyond.

Step 3: Data. What types of data and information are needed by the data users for their purposes?

The third step considers the types of data required to meet the needs of the identified uses and users. It analyses the technical features of data production and how these might interact with contextual features around decision making to either promote or inhibit the use of data.

Step 4: Processes. How do governmental processes support evidence use and/or mitigate potential biases?

The final step focuses on governmental processes. It analyses mechanisms for data reporting, systems for data verification, platforms for analysing and sharing data, and how monitoring processes are funded. From a political economy perspective, these processes are vital in supporting data use, and can play an important role helping to mitigate potential cognitive biases.

Ultimately, actors involved in the design – or redesign – of a national monitoring system need to pay as much attention to data use as to data production. And this must happen from the very outset of any design or redesign process.

From theory to practice

To better understand the practical value of the data use framework, we used it to analyse contemporary examples of sector monitoring programmes in Nicaragua, Sierra Leone and Timor-Leste.

The purpose of these case studies was not to provide evidence of best practice to be replicated elsewhere, but rather to demonstrate the types of insight that can be gained through a better understanding of data use, and to show how these insights could be used to strengthen the design of monitoring programmes. The table below gives an overview of the findings and their implications for sector monitoring programmes.

Step in the framework

Key insight

Implications

Purpose

Decision-making processes are not clear cut and decision makers do not always see themselves as such.

Sector monitoring programmes that understand and work with the grain of these complex processes are more likely to result in greater data use.

WASH monitoring data is used for several purposes – and can sometimes be useful even if not used to make specific decisions.

Sector monitoring programmes will have a greater impact on data use if they consider all possible uses from the outset.

Context

Wider institutional arrangements, such as decentralisation and cross-ministerial coordination, can either promote or inhibit data-informed decision making.

This highlights the importance of aligning sector monitoring programmes with existing institutional arrangements – both vertical and horizontal.

Integrating sectoral monitoring data in core government public financial management functions is a key step in promoting data-informed decision making.

This demonstrates that the need for monitoring programmes to integrate WASH data within core governmental processes, and to understand how these processes play out in practice.

Data

The type of WASH data needed is specific to the decisions being made or the potential uses, but monitoring systems are unlikely to meet all needs.

Sector monitoring programmes need to prioritise the most critical data needs that cut across many groups of stakeholders.

Issues around data collection and processing can have important consequences for data use, and vice versa.

This highlights that neither data production nor data use can be considered in isolation – programmes must aim to strengthen the whole sector monitoring sub-system.

Processes

A reporting culture can discourage data use at the local level, but well-designed processes and ‘data dialogues’ can encourage use at all levels.

To foster a culture of data use, monitoring programmes must feed into processes that produce substantive actions and follow up.

The way WASH monitoring and WASH interventions are designed and funded shapes the effectiveness of data use.

Sector monitoring programmes must develop processes that are collaborative, participatory and take a long-term approach.


The data use planning guide

Obviously, these insights are rather generic, and serve only to highlight the potential value of this approach.

To support stakeholders to build a more critical understanding of the issues that affect the use of WASH monitoring data in their own specific context, we have developed the Data use planning guide. In the guide, we provide step-by-step guidance to support governments and development partners to apply the data use framework to new or existing monitoring programmes.

The planning guide does not provide a roadmap for the entire process of designing a sector monitoring programme, but it can inform strategies to address issues of data use within those programmes. It is intended to be used as the first step in designing a user-centred WASH monitoring system, or when improving or redesigning an existing system to better support the use of data.

Next steps

The From data to decisions synthesis report, policy brief, and planning guide are all available on WASH Matters. We are working to integrate these findings into our programmatic work on sector monitoring, and will use evidence gained from this experience to influence the work of others.

Our From data to decisions research has shown that governments and development partners need to go beyond the production of data to strengthen all the institutions, processes and incentives required to enhance the use of data in decision making. This can be achieved by a better understanding of data use, through an analysis of purpose, context, data and processes.

Stuart Kempster is Senior Policy Analyst – Governance at WaterAid UK.