You cannot manage what you do not measure; but should you measure what you cannot manage?

6 min read
Thumbnail
Image: WaterAid/Sam Vox

Many countries are developing or reviewing their water sector monitoring systems to see whether they comply with the Sustainable Development Goals. But should we rely on data captured at household level on access to limited, basic or safely managed water as defined by the Joint Monitoring Programme?

Countries have committed to reach Sustainable Development Goal (SDG) 6: providing universal access to their population with safely managed water supply services, with country specific targets. This is a process that governments, as duty bearers, need to manage. Therefore they are also responsible for measuring progress.

The definitions that are proposed by the Joint Monitoring Programme (JMP) for SDG target 6.1 require data to be collected at household level, as it is households that need to indicate whether they get water from an improved source, the time it takes them to get that water and whether that water is available when needed. Only water quality – for now – is not measured at household level, but coming from data at service provider level. There are different instruments that governments can use to obtain data from households – often led by national bureaus of statistics:

  • Censuses. In theory, a census would capture answers to the above questions for every household in the country. It would provide planners at all levels – from district to national level – insight into the percentages of the population that have different levels of access; it would also give insight into where these people live. The drawback of censuses is that they do not happen frequently – typically only once every 10 years. That is too long a horizon for planning and management of water interventions.
  • Household surveys. National bureaus of statistics also regularly – every year or two years – do different household surveys, such as MICS (Multi-Information Cluster Surveys), or DHS (Demographic and Health Surveys). These are not census-based but sample-based. A representative sample of the population is asked a series of questions including on water and sanitation access. Extrapolating the answers gives insight into the percentages of the population with different levels of access. Mostly, those surveys are representative of the level of the country or large subnational units, such as states in federal countries, or provinces. That means differences in access between those larger administrative units can be seen, and used for prioritization between those units. But those units are rarely the ones at which water supply is managed. Moreover, the surveys don't give insight into where the people with different levels of service live within that unit. Surveys would have use in macro-level planning and progress tracking, but not in planning of interventions at the level where they usually happen: the district.

Water sector monitoring systems – which are in development or in review in many countries – could fill the gap. They are meant to provide district governments and service providers with data on water supply services status and quality, so that they can plan, budget and manage infrastructure and the services they provide. A key question being asked in several of the discussions that we have been part of, is whether those sector monitoring systems should include data coming from household surveys.

To answer that, we need to know first what it is that district governments manage. Only then can we answer what they should measure and what what are the right indicators for that. Broadly speaking, district governments and service providers manage three types of activities around water supply:

  • Developing the water infrastructure that provides the services. For that, insight into access and the level of service at household level is too fine-grained. It would suffice to know broadly, in which communities there is a functioning water supply system, and in which ones not – but without necessarily knowing the detail of whom in the village actually access and use the system. In fact, planning for the development of infrastructure based on sample-based data at household level is impossible. For that one needs to know not only the percentage of the population with different levels of access, but also where they live. Therefore, for this type of activity a district ideally uses data on whether there is a water service in a village or not.
  • Managing the existing services and the levels of service these provide. This will require data at the level of the service provider and at the system, though not necessarily the level of service used by households. Ideally, a district has such information for all the systems and providers in its area of jurisdiction to inform interventions in the management of existing services.
  • Ensuring the use of services. Even where services exist, they are not always fully used by household members – for different reasons. Understanding and addressing the reasons for under-use is key to come to targeted management response to improve the use. This would require insight into the level of access at the household, through household surveys. Moreover, it would require sampling at the level of individual communities. One needs to know not only what % of the population uses different water systems, but also where they are located. Only with that insight can efforts be targeted at those populations.

In short, for managing service provision, household data on level of service received is useful, but not essential. Household data is essential for tracking the use of services and defining measures to ensure use. Moreover, such data would serve to validate the services levels provided.

Household surveys require a large effort and resources, not only to collect information, but also to analyse that. Governments therefore need to evaluate the effort of doing household surveys in relation to the additional insight it gives and its potential use. It is therefore not a matter on whether household level data should be collected or not. It is more one of the – temporal and spatial – extent with which it is collected. Specifically, we would recommend the following:

  1. Engage with household data as collected by national bureaus of statistics. These will continue to collect such data at a regular basis and used to track a country's overall progress against SDG 6 and in many countries with some sub-national break-down. We realise that this rarely goes down to the level at which water services are managed. But it is very important data to track progress. We still see that too few sector professionals fully understand these data, let alone engage in their use.
  2. Collect household data as part water sector monitoring systems, but with a relatively low frequency, e.g. every four years or so or when it is available from other sources (e.g. iNGO who collect representable baselines). This is a sufficient time frame to validate data coming from system or provider level information with ones on the actual use.
  3. Prioritize household data in countries or districts where nominal access levels are relatively high, let's say above 85%. In such cases, there are normally very few communities without any improved supplies. In those places, focus is ensuring that 'the missing 15%' also get access to services. This means in many instances addressing intra-village equity, promoting behavioural change or working on affordability of services.

In conclusion, governments, as duty bearers, and service providers need to prioritize measuring data on services provided, so that these can inform the management of services. Management on the basis of household data on the services received, is more difficult. Therefore, we recommend taking caution in the extent of measuring such data as part of national sector monitoring systems.

 

Ellen Greggio is Programme Advisor for Monitoring & Mapping at WaterAid, and tweets as @EllenGreggio. Stef Smits is Senior Programme Officer at IRC and tweets as @SmitsStef.

This blog was first published on the IRC Wash website >