Building an evidence base for WASH: linking research and practice
"We have a tendency to want to boil complex pieces of research down to simple conclusions... the real world is much more complicated than this." Panelists discuss conclusions from a session at Stockholm Water Week 2018 on linking water, sanitation and hygiene (WASH) research and practice.
The session
Generating robust evidence about the impact of complex interventions is difficult1. During Stockholm Water Week 2018 individuals from academia, government, donors and WASH organisations came together to discuss the challenges faced in real-world environments, as well as examine current approaches to generating (and sharing) evidence to determine how researchers and practitioners can collaborate more effectively.
The need for evidence
Governments and NGOs which implement water, sanitation and hygiene (WASH) programmes, as well as those who fund them, need evidence of the effectiveness, value-for-money and impact of those interventions. This is especially so when there are competing demands for financial resources.
"As a policy maker, I can say that evidence really matters… choices have to be made… donors need to focus on value-for-money and economic aspects."
The complexity of WASH
Each WASH intervention is different in many respects including: the biophysical context; cultural norms; national and local resources; expectations and responses of the population; the institutional context and political economy. Standard methods for synthesizing evidence, such as Cochrane Reviews and meta-analyses, can often mask the underlying diversity behind each individual WASH intervention.
"We should expect heterogeneity… we need to learn to think in a less mechanistic way."
Research
As in all fields of human endeavour, there are gaps in knowledge and evidence in the WASH sector(s). Research studies contribute to reducing that gap, but they must ask the right questions and use methods of investigation which are appropriate to those questions.
"As a policy maker, I am frustrated with research which answers the wrong questions… "
Identifying relevant and useful research questions requires dialogue between the various stakeholders – the questions which are important to a donor over a relatively short time scale may be different to those of importance to Governments which need to turn evidence into policy. Academics and others also have their own perspectives on research priorities.
"We need to be more intelligent in how we ask questions... as researchers we need to be answering the questions which matter to donors... we need to explain better why our questions are relevant."
It is clear that no individual research approach or method is universally superior to the rest; multiple strands of evidence contribute to our understanding. In a complex social-technical intervention such as WASH, context and implementation are highly location- and time-specific. Understanding the true impacts of WASH interventions will require flexibility in approach and diversity of designs.
"We need multiple threads of evidence… relying on a single research design is not safe…"
A randomised controlled trial provides rigorous evidence of the impact of a particular intervention in a particular place at a particular time. However, there is a trade-off between their high degree of internal validity and their very limited external validity. In the absence of parallel process evaluations and other studies of context and political economy, it may be difficult to identify why a study’s findings are as they are.
"... randomised controlled trials are really good at controlling for confounding so they’re really good at being internally valid, but they’re terrible at external validity."
"Any individual research study represents, as it were, a single data point. What matters is the cumulative body of evidence which builds from multiple studies using a variety of investigative approaches."
Short-term and long-term impacts
Even in the short-term, there are many potential benefits associated with improvements to WASH services and practices than extend beyond the traditional definition of public health. For example, the World Bank Water and Sanitation Program’s work on the economics of WASH lists 33 benefits of which only a handful are monetised. In regard to health impacts, while we know that the use of safe sanitation and water services and the practice of good hygiene are essential, we also recognise that achieving short-term impacts on diarrhoea, stunting and other health indicators is challenging.
"It’s hard to see health benefits within a period of 3-5 years... on the other hand I know from the long-term historical studies that you do not have healthy societies unless they have functioning water, sanitation and hygiene."
"There’s an inter-generational pay-off associated with mothers being able to spend more time with their children... we need to take a punt at trying to identify these longer-term impacts."
Furthermore, less tangible benefits should not be ignored in favour of those health and time-saving benefits which may be quantifiable.
"We looked at time savings in Mozambique… we found that the hour-and-a-half to two hours these people were saving… they weren’t chaining themselves to sewing machines… they were hanging out with their frickin’ kids… that’s bad, right?[!]"
Tensions
There are tensions between practitioners who may emphasise the human rights dimensions of WASH interventions and researchers interested in the epidemiological dimensions and health impacts of WASH. This can be a constructive tension if it encourages dialogue between practitioners and academics.
As well as conventional research, more real-world studies – operational research, programme evaluations – are needed. These provide grounded evidence in a way which more controlled experimental research studies (especially those studying efficacy rather than effectiveness) cannot.
"One step we could take is to do more rigorous and routine evaluation of programmes ... there’s so much experience out there, and that needs to filter out ... we would get a lot of learning out of that..."
Learning
In a situation where rights-driven practitioners and health-focused epidemiologists may be 'talking past each other', the key requirement is a commitment to joint learning.
"... and we need to be open to the fact that not all the news is always going to be good news"
Communicating evidence
Finding the right messages and the right language with which to communicate the findings of individual research studies and bodies of evidence is crucial. Communicating research findings and evidence has political and ethical dimensions.
"[the findings of a health impact study] made a storm in the health sector… actually this is a problem… [failing to communicate the nuances of the research] is very troublesome to Government…"
The need for better dialogue
When major pieces of research are planned and implemented, it is essential to involve and fully represent the views and perspectives of all stakeholders (including Government, policy makers, implementers, academia, donors and research consumers) in order to identify the right research questions and appropriate methods, draw the right conclusions from the data, and communicate the findings in responsible ways.
"How often does interaction between the various stakeholders happen before large-scale trials are undertaken ...?"
Recommendations from the panel for the WASH sector
- Research studies examining the outcomes and impacts of WASH interventions should begin with in-depth dialogue between implementing organisations (including Government) and research teams. All stakeholders (Governments, donors, academic institutions, implementing agencies and civil society) need to be involved in the research process.
- An important element of learning about “what works” can arise from project and programme evaluations; we therefore recommend that more evaluations are undertaken, and that their findings should be regularly synthesised to generate sector learning.
- Many recent WASH impact studies have sought to uncover short-term (2-3 year) health impacts. While in many cases it may be relevant to measure short-term outcomes and impacts, some research should also address questions with much longer-term results.
- The entire WASH sector needs to become more intelligent, articulate and nuanced in how it frames research questions, how it expresses the findings of such research, and how it communicates with different audiences.
- The focus of WASH research should move from a pre-occupation with impacts on health (infant diarrhoea and stunting) to questions about how WASH programmes can bring about inclusive and sustained use of services and practice of hygiene behaviours – in other words the intermediate outcomes which are thought to be some of (but not all) the pre-conditions for ultimate health impacts.
These notes were prepared by the authors of this blog from a recording of the session. The statements made here have been reviewed by the panel members and a few other session participants: Sanya Tahmina – Office of Director General of Health Services, Bangladesh; Khairul Islam – WaterAid Bangladesh; Guy Howard – UK Department for International Development; Stephen Luby – Stanford University; Robert Dreibelbis – London School of Hygiene and Tropical Medicine. The quotations are mostly verbatim but in some cases have been paraphrased for clarity.
Footnote:
