How do we know if we’re strengthening the system?

A system-wide approach is increasingly recognised as essential to the achievement of universal access to safe water, sanitation and hygiene (WASH). But how do we know what impact we’re having? Clare Battle, Senior Policy Analyst – Governance, shares reflections from our policy work.

A system-wide approach means working together to transform the national, regional and local systems (including institutions, infrastructure and resources) needed for sustained and equitable WASH services. But for donors, governments and implementing partners, moving to such an approach is both practically and politically challenging, nowhere more so than in how progress is measured and communicated. 
Learning from current practice

In 2016, we carried out research into donor approaches to measuring WASH sector progress, and the degree to which these were embracing aspects of systems thinking. Our study found that – at that time – no donor yet had a holistic systems approach that encompassed analysis, design, implementation, monitoring, evaluation and learning.

Over the past three years, interest in systems-orientated approaches has continued to increase. Several donors have released WASH strategies with system strengthening at their centre, and are now developing monitoring approaches to accompany them. But identifying and monitoring the drivers of sector performance in a way that satisfies the need for easily communicable and politically straightforward results remains a challenge.

In this context, we conducted a further study, looking not only at WASH sector thinking and practice, but also drawing in lessons from the health sector, where approaches to assess and monitor system strengthening are longer established. By analysing different approaches and their associated risks, it found that successfully monitoring and incentivising systems change will require us to combine elements from a number of different approaches, and work together to address the risks and misaligned incentives these can create.

Moving forward together

Inspired by this, we used the ‘All Systems Go’ Symposium in March to convene representatives from DGIS, United States Agency for International Development (USAID), the UK Department for International Development (DFID) and Water for People (WfP), to discuss the challenges they face investing in, and monitoring, systems change. These included:

•    Navigating political pressures. For the past few years, many donors have been working to significant numerical access targets. Most are still mandated to talk first and foremost about services delivered, rather than the strength of the service delivery system.
•    Allowing for aggregation. The need to aggregate results across different levels of a country, and across a portfolio as a whole, remains a key constraint for donors when it comes to measuring system strength.
•    Unpacking attribution. USAID is increasingly tracking progress through ‘metrics for self-reliance’ built around capacity and commitment, but these provide a broad, country-level picture, rather than being attributable to the agency itself.
•    Addressing complexity. Monitoring systems change is incredibly hard to do, so finding a practical approach can mean imposing artificial boundaries on the system. 
•    Accommodating flexibility: Donors often struggle to balance flexibility (in which indicators can be identified locally, or emerge during the course of a programme), against the paralysis that can come from tracking too many different things.
•    Budgeting for systems change. Strengthening systems and ensuring sustainability costs more and takes longer. Yet donors remain constrained by short budget and planning cycles.

But it’s not all bad news! 

Speakers also shared some of the responses they are using to push internal conversations forward. These included a central commitment to learning, and strengthening the incentives for field officers to work on things that might not see results until after a programme has ended. And there is clear enthusiasm for a more nuanced approach to results, so development partners can better convey the full impact they are having.

So while the conversation is still evolving, there are promising signs that the sector is moving forwards, and actors are willing to work together to find solutions. WaterAid remains committed to supporting this dialogue. Only by doing so will we be able to establish a clear view of the markers of progress, and effectively channel investments towards WASH sector improvements.

Read and download the policy brief in English and French here >

Clare Battle is Senior Policy Analyst – Governance at WaterAid UK. Follow her at @Clare__B