Editor’s Note: This post comes from the Water Point Data Exchange, an initiative aimed at creating a standard for water point data collection to allow for better information sharing across organizations working in WASH. The Exchange invites those working in the sector to provide feedback on their draft standard, available here. The public comment period will be open through March 15.
No challenge in eradicating water poverty is as pressing as improving sustainability by ensuring continuous high-quality drinking water service provision. High failure rates in the WASH sector squander scarce financial resources. Further, broken water systems can cause harm to millions of people in the Global South who have invested their own time and resources and come to depend on these systems. Despite knowledge of this issue for decades and increased attention over the past few years, there is still a severe lack of understanding about the true nature of this challenge. Limited available data on water points has stymied efforts to better understand and improve sustainability. Facing these challenges, the water sector is at a pivotal point in history where data can play a vital role in the eradication of water poverty.
The amount of water point data is increasing as more organizations and governments monitor their water point functionality over time. Unfortunately this progress has been accompanied by increasingly numerous unique methods through which data is collected, stored and shared. As a result, new data is creating new “data islands.” These emerging monitoring efforts will have limited impact on improving programming and sustainability as long as the data remains inaccessible on organizational servers, in PDF reports, and in proprietary monitoring systems. Unless these different data sources can be harmonized, the learnings will be small and piecemeal, with the true potential of this information remaining untapped.
The Water Point Data Exchange (WPDx) plans to harness this momentum by working with the water sector to create a standard for water point data sharing. To learn more about efforts to develop a framework for sharing this data, view the webinar, hosted by Global Water Challenge, on February 5:
Based on this standard, WPDx will create an effective central hub for the standard-compliant data and will engage the WASH sector to make full use of this dynamic hub, both for sharing and utilizing data. Ultimately, this initiative will transform a growing body of disconnected data into improved water services for millions.
To make this initiative work, we need your input! The WPDx Working Group is inviting water access experts from around the world to provide feedback on the draft standard, available at https://collaborase.com/wpdx. This public comment period will close March 15, so please share your feedback soon. It will only take a moment, and no registration is required. Click here to review the draft standard and share your feedback now.
Editor’s Note: This post is authored by Cor Dietvorst, Programme Officer at IRC. In his piece, Cor discusses the monitoring requirements surrounding India’s Swachh Bharat program, which Prime Minister Narendra Modi launched in October 2014 with the aim of ending open defecation in the country by 2019. He compares India’s sanitation monitoring initiative with other large-scale monitoring efforts with which IRC has been involved in Bangladesh and Indonesia. This post originally appeared here on the IRC blog.
According to some media the Indian government has unleashed “toilet police” or “toilet gestapo” into the country.1 In fact, the central government has instructed local officials to take photographs of new toilets to prove that they have not only been constructed but are also being used. If states don’t upload photos by February 2015, the water and sanitation ministry has threatened to withhold funding from a new national sanitation programme.2
Open defecation free by 2019
Prime Minister Narendra Modi launched the Swachh Bharat (Clean India) Mission on 2 October 2014. His aim is to attain a 100 percent open defecation free India by 2019. Since the launch, over half a million household toilets have been constructed.3
By implementing “real time monitoring” the government hopes it can correct past mistakes caused by ineffective monitoring and wasted investment in sanitation. The 2011 census revealed that 43% of government funded toilets were either “missing” or non-functional.4 Now the government wants to show that its investments in sanitation are delivering lasting results.
The Ministry of Drinking Water and Sanitation is appointing around two dozen additional staff including two joint secretaries and four directors to strengthen the implementation and monitoring of the Swachh Bharat Mission. An Expert Committee for innovative sanitation technologies and a national telephone helpline for rural water supply and sanitation are other new initiatives that will support the Mission.5
Local officials charged with monitoring toilet construction and use need to download an app on a mobile device. The app allows them to upload photos as well as the personal data and geo-coordinates of the beneficiaries to a public website. Progress is slow though: as of 14 January 2015, data of less than half a percent (2,383) of the newly constructed toilets has been recorded. Data collected before 2015 does not include toilet use.
How do other countries carry out large-scale monitoring?
Compared to examples of large-scale sanitation monitoring in Bangladesh and Indonesia, the toilet use indicators collected in India -- is the toilet in use, is it clean and is water available -- are rather limited.
The BRAC WASH programme in Bangladesh uses benchmark indicators developed by IRC for questions like: do all household members use toilets, do they use them at all times, and are there provisions for handwashing and pit emptying.6
In Indonesia IRC has helped design a monitoring system for the SHAW (Sanitation, Hygiene and Water) programme, where every three months 20,000 community volunteers visit more than 300,000 households. For SHAW monitoring is not merely an accountability tool as it is in India, but a way to motivate and encourage people to improve their sanitation facilities and hygiene behaviour.7
India's decision to track toilet use as part of its new monitoring initiative is a major step forward. From its neighbours, India can draw valuable lessons on how to monitor sanitation as a sustainable service that benefits all.
2 Letter to Principal Secretary/Secretaries in charge of Rural Sanitation all States and UTs. Ministry of Drinking Water & Sanitation, 05 Dec 2014
3 Unused rural toilets to face public scrutiny, The Hindu, 01 Jan 2015
4 Tiwari, R. The case of the missing toilets. India Today, 02 Oct 2014. See also: Hueso, A. & Bell, B., 2013. An untold story of policy failure : the Total Sanitation Campaign in India. Water policy ; 15 (6), pp.1001–1017. DOI: 10.2166/wp.2013.032. and Hueso, A., 2014. The untold story of India's sanitation failure, Addendum. Community-Led Total Sanitation.org, 11 Mar 2014
5 Nationwide monitoring of use of toilets will be launched from January, 2015, PIB, 31 Dec 2014
6 IRC - Monitoring at scale in BRAC WASH
7 Baetings, E., 2014. How are you and how is your loo?
Have you searched our map to orient yourself to the WASH funding landscape? Do you regularly read this blog? Are you familiar with our featured case studies or funder profiles? If you’ve visited WASHfunders, we’d love your feedback as part of our evaluation of the site.
Launched more than 2 ½ years ago, WASHfunders has been an experiment for our team at the Foundation Center. We hoped WASHfunders would facilitate knowledge-sharing, foster collaboration, and promote transparency related to foundation funding. But the truth is, we know relatively little about how our audience is using the site. Now is an opportune time for us to step back and take stock -- to understand where we can improve and how the site can be a better resource for the sector.
We’re currently working with an independent evaluation team to assess the impact and effectiveness of WASHfunders and want to hear your thoughts on the site.
If you’re interested in participating, the evaluation team will contact you to conduct a short one-time phone interview. If you would like to contribute to this effort, please contact Seema Shah at firstname.lastname@example.org. Thank you for your help!
Editor’s Note: This guest blog post was authored by Ben Seidl, program director at World Water Relief, an NGO launched in 2008 with the goal of bringing sustainable water purification solutions to people in developing nations. In his post, Ben discusses the push for better monitoring and evaluation (M&E) in the WASH sector and the challenges and opportunities that this trend presents for small NGOs. Ben emphasizes the importance of local engagement as the key to both effective M&E and, ultimately, project sustainability.
As the WASH sector continues to expand and strengthen its role in global health, the sector’s trends and objectives have become more data-oriented and results-focused. Mobile, field-level technology has enabled NGOs to undertake data processing and monitoring of water resources in real-time…a practice that was previously only afforded to large municipal utilities and corporations. While this technological leap has ushered in a new era of transparency and reporting, there are some fundamental building blocks of sustainability that are beyond data.
Human capital is still the true driver behind sustainability and M&E in the WASH sector. Local, dedicated stakeholders are the true source of long-term sustainability and accurate, reliable monitoring and evaluation. Without the involvement of these local community stakeholders, the sustainability of any WASH project will undoubtedly wither over time.
As Program Director for World Water Relief in the Dominican Republic and Haiti, my team and I are tasked with building a responsive and flexible monitoring program to ensure that our projects are creating measurable impact and consistent WASH service delivery. World Water Relief is an NGO with limited manpower and resources. Thus, we are faced with the challenge of producing high-quality WASH projects with a high level of feedback and sustainability on a shoestring budget.
Without the funds for advanced technology and data collection, we are tasked with finding alternative ways to ensure that our WASH projects are meeting these three criteria:
I) Beneficiaries’ needs
II) Industry and international standards
III) Donor expectations
To address each of these criteria in a cost-effective way, we need to craft local, low-technology relationship networks to implement and feed our data and sustainability measures. As an organization of less than ten employees, we depend on the passion, dedication, and involvement of the stakeholders in the communities we work in to be the drivers behind our sustainability and M&E initiatives.
One such program we employ in both Haiti and the Dominican Republic is the Youth Water and Hygiene Club. This type of school-based youth programming has been championed by the WASH sector as an intervention capable of providing youth with leadership training, experiential learning, and an in-depth opportunity to learn and practice water, sanitation, and hygiene solutions firsthand. Our Youth Water and Hygiene Club has been both catalyzing for the participating youth and beneficial to the schools and communities they serve. Students are empowered to be active participants in improving and maintaining the World Water Relief WASH infrastructure in their respective schools and communities. This means helping to clean drinking water stations and hand washing stations, chlorinating potable water holding tanks, initiating trash and recycling collection, teaching WASH principles to student peers, and providing direct monitoring and feedback on WASH service delivery.
The second benefit of a school-integrated program like this is that M&E is conducted on a daily basis at each WASH in Schools site. The Youth Water and Hygiene Clubs provide detailed and dedicated reporting on the status of their schools WASH projects. The World Water Relief program mangers in both the DR and Haiti are in daily communication with the club officers and have frequent regional meetings that feature 82 youth from 16 schools. These meetings provide an excellent opportunity for club leaders to learn from each other and for World Water Relief to continue empowering an inter-connected network of dedicated WASH youth.
The ultimate goal of WASH M&E initiatives is to provide insightful field-level information and analysis that drives accurate and timely project oversight. Ideally, WASH implementers are then able to relay these informative reports to donors and stakeholders in order to prove the efficacy of WASH projects around the world. The rapid progression of technology over the past decades has greatly enhanced the sector’s ability to create and share these important results. However, when we think about sustainability and evaluation, we must remember that data and observation can only take us so far. True sustainability still lies in the hands of the local users and stakeholders.
As the WASH sector moves forward in its pursuit of real-time tracking and evaluation of project efficacy, we mustn’t lose sight of the ability and potential of end-user involvement. Data can inform and guide, but the root of sustainability is still built through long-term relationships, strong personal communication, and direct face-to-face participation.
Editor’s Note: This guest post was authored by Malaika Cheney-Coker, learning and influencing advisor of the Water Team at CARE USA. Malaika reflects on CARE’s recent and first-ever global assessment of its water portfolio, Water+Impact Report: Walking the Talk. The report aims to answer questions such as: What happens after CARE leaves? Is CARE creating the right type of change? What differences has CARE made in the lives of women and girls? It offers a comprehensive meta analysis of 51 different project evaluations from around the world. (The views expressed here are those of the author and do not represent those of CARE.)
Here’s one reason not to do a global impact report: you might not like the results. When the Water Team at CARE headquarters decided to do one, we, were enthusiastic about having a global picture for water performance that hadn’t existed before. That global picture was a mixed bag. Through a fairly subjective review of 51 results-focused documents dating from 2006-2010, we came up with scores: 6 out of 10 for our direct service delivery work, and 4 out of 10 for both our work with policies, institutions, and social norms and for our work in promoting gender-equitable control over water and related resources.
When we presented these findings in-house, our colleagues felt that we had been too self-critical and asked how our work compared with that of others in the sector. Very well indeed, it turned out — at least in 2011 when Philanthropedia ranked CARE International #5 among 116 different nonprofits working in the field of water, sanitation, and hygiene.
So then logical follow-up questions would be: Do we stand by our ranking? (We do.); What does this say of overall performance in the WASH sector? (That there are systemic problems of which the recent sustainability dialogues are one indicator.); And, does a ranking even matter? (Yes and no, I’d contend.)
Let’s take the “yes” answer — that the grade does matter. The grade matters because, quite simply, we can do better. If the effects of our work, or even the infrastructure we set up, were consistently found present and functionial, or even sputtering along, 10 years after project implementation, there would be room to contest the grades. If achieving influence at large scale, rather than in a few hundred communities per country, was routine, the score would be higher. We have decades worth of programming that proves we have a relatively smooth operation of installing infrastructure, organizing committees, and training hygiene promoters. But we want more than this for our work and we know that good development demands more.
What does more look like? Many things is the answer, including the thorny policy and policy implementation, governance, societal norms, and knowledge gaps issues — all of which must be addressed while tackling gender equity and other forms of social injustice (all areas which led to lower scores in our assessment than our direct service delivery work). It is part science, part alchemy; it is part the ingenuity of the humble tippy tap, and part the trick to coax innovations like it into being. Don’t get me wrong, we have work that does all of these things. But if that work was consistent across the board, if the results of the work were permanent, or at least durable, and had the irresistible logic or appeal that makes something go viral, then the score would be higher.
For a look at the “no” answer, the truth is that examples of global impact (of a positive, desirable kind) abound. An impact report can grade itself on such a curve that a satisfying report is assured. But to grade tough with a worthy yardstick is about vigorously seeking change at the cost of self-exposure.
Furthermore, development work is complicated enough and the scale of problems such as depleted natural resources, social oppression, and gender bias is immense enough that even with the most cutting edge programming must strike a balance between ambition and realism. A score implies we know what the ultimate solution to the problem is and can judge our progress relative to it. Development has often come at the expense of the environment, human rights, or the advancement of other nations. We recognize all these things as important but there are wildly varying opinions of the optimum balance of these elements. Even if we were to come to a consensus on such a balance, we all know it would vary by context. And whatever agreements we can cobble together today will not serve us adequately tomorrow as our understanding of development will and must continue to evolve.
In addition, we face limitations in how we design and implement programs. Our funding cycles are not always conducive to investing in long-term results or even allowing for the ability to monitor over the long term. But we must find ways to do so. Measuring impact works muscles of self-inquiry and programming discipline. It illuminates a global picture that exposes the cobwebs in our thinking and programming while also turning up some of the gems in our work. The measurement process itself is a workout, despite the actual score.
In sum, there are reasons why the score matters and at least as many why it doesn’t. So a fair question to ask is, “how did they score themselves?”; a better question to ask is, “how much do they want to change?”
To ensure that WASH projects are successful, it’s critical to apply lessons learned from previous work in the field. Yet it can be challenging to track down relevant evaluations from multiple online and offline sources. In an effort to help our audience make better use of all of the research and knowledge accumulated in the sector, we’ve compiled a list of 20 online evaluation databases that house WASH-related evaluation reports. Some of these databases have a handful of WASH reports, while others have hundreds of reports available.
Take a look and if there’s anything we missed, leave us a note in the comments section below.
Editor’s Note:This guest blog was authored by Dr. Kerstin Danert, water and sanitation specialist at the Swiss-based Skat Foundation. Kerstin discusses country-led monitoring and why it’s important for developing country governments to lead the WASH monitoring process. An online community is being formed around country-led monitoring efforts. If you’d like to learn more about it, you may contact email@example.com.
In April 2013, I had the privilege of facilitating six sessions on country-led monitoring at the Monitoring Sustainable WASH Service Delivery Symposium in Addis Ababa. This blog is a reflection on the papers, presentations, and discussions from that event.
International statements such as the Paris Declaration, the Busan Partnership, and the New Deal for Fragile States call for country-led development. The statements also promote results-based development and highlight the importance of monitoring — specifically monitoring that is country-led.
Monitoring refers to an ongoing process by which stakeholders obtain regular feedback on the progress being made towards goals and objectives. Country-led means that the country, rather than external actors, leads the monitoring process. Institutional and individual capacity needs should be developed gradually, and as necessary, depending on what is needed.
I would argue that many rural water supply projects, whether large or small, whether short or longer-term undertake very little monitoring at all. Progress may be checked and expenditure may be compared with outputs. A report will be written for the funders — who are often very far from where the work is taking place. Monitoring ends there. In parallel, national statistics offices monitor poverty changes. Water ministries may or may not monitor systematically. There is often very little interaction between these.
Now is this a problem?
I think that it is a major problem. Firstly, for a country to make progress, for example towards safe drinking water for all, it needs to learn along the way. If good information — about the successes and challenges and about what works and what does not — remains fragmented in the hands of countless organizations, it is very hard for the country as a whole to learn. Secondly, if there is no reliable feedback about progress to political leaders and rural citizens, accountability is undermined. Democracy is undermined. Governance is undermined. Joint action is difficult.
Hope on the horizon
Fortunately the growing number of Joint Sector Reviews, which bring together a diverse range of stakeholders to reflect on progress, signals winds of change. And gradually governments, together with development partners, are trying to define what to monitor, by whom, and with what means. This may even be taking place in the country that you are working in.
The sheer cost of visiting distant and often hard to reach rural dwellers is a major barrier to understanding their needs, and reflecting on how lives can be improved as a whole. External organizations often focus on one particular group or village, reporting on what has been achieved to their funders. In parallel, local governments are generally massively under-resourced and struggle to monitor and follow up with communities once an intervention is completed.
Better understanding — stronger partnerships
Every external organization working in rural water supplies should take some time to truly understand the wider context in which they work. And they should try to engage with country institutions in a meaningful and constructive way. This means listening and talking to both local and national governments; understanding their strengths and challenges; finding ways to plan together; and being highly transparent. It may require several attempts. On the flip side, governments and other country institutions, such as religious bodies, should try to foster strong partnerships with external organizations, ensuring that all are moving in the same direction.
Often funding agencies and non-government organisations do not trust developing country governments. In addition, they report and account to their funders, bypassing country governments completely. One may consider whether this is acceptable or not, but it certainly does not enable real partnership.
Joint monitoring as a stepping stone
Finding clever ways to monitor together provides a stepping stone towards stronger partnerships. At least that is what we can learn from innovations taking place in Malawi. Here an external NGO has been explicitly working with local governments to catalyze data collection, analysis, and use of the information in planning. The collaborations have widened to join up with national government. In Uganda, the Water and Environment Sector Performance Measurement process brings diverse stakeholders together to reflect on progress for the country as a whole.
There are other interesting examples out there. If you know of any, please share them in the comments section below.
This month, the WHO/UNICEF Joint Monitoring Programme (JMP) released the Progress on Sanitation and Drinking-Water: 2013 Update, an in-depth analysis of trends in global access to safe drinking water and improved sanitation.
According to JMP’s findings, almost 1.9 billion people have gained access to an improved sanitation facility since 1990, yet fifteen percent of the world population was still practicing open defecation in 2011.
The report calls for a bigger push to meet the 2015 Millennial Development Goal (MDG) to halve the proportion of the 1990 population without access to improved sanitation. “There is an urgent need to ensure all the necessary pieces are in place — political commitment, funding, leadership — so the world can accelerate progress and reach the Millennium Development Goal sanitation target,” stated Dr. Maria Neira, WHO director for public health and environment.
The report additionally emphasizes ending open defecation by 2025. Findings highlight the success of piped drinking water in improving water access, and assess the connection between intermittent service and increased water contamination.
Post-2015 targets and indicators include universal hand-washing facilities in the home by 2030.
Read the report and additional key findings. Then, tell us what you think in the comments below.
Editor’s Note: This blog was authored by Susan Davis, executive director of Improve International, an organization focused on promoting and facilitating independent evaluations of WASH programs to help the sector improve. Susan discusses how a “services monitoring” approach can help improve and maintain WASH services. A version of this post originally appeared here.
Last month I went to the Sustainable WASH Forum and Donor Dialogues in D.C. A theme of the conversations was roles and responsibilities, especially the roles of governments. One interesting debate was about who should be responsible for monitoring. Some said that governments should be solely responsible. There are some governments who are leading the way on this, but others (myself included) believe that this doesn’t mean that development organizations shouldn’t also be accountable for their own work. If an organization visits water and toilet systems for years after they are built, they can learn from their successes and failures and make their future work better.
Since many organizations only do monitoring & evaluation (M&E) during development programs (see my thoughts after the Learn MandE conference), I think we need to use a new term like “services monitoring” to refer to the need for a way of confirming that water and sanitation services are still available to people.
Why is services monitoring important?
- 783 million people without access to improved source of water[i] 3 billion without access to safe water[ii] 4 billion without access to safe, permanent, in home water[iii]
- 2.5 billion people without adequate sanitation[iv] 4.1 billion lack access to improved sanitation[v]
- 35-50% water and sanitation systems that fail within a few years of construction[vi]
- Less than 5% water systems that are visited at least once after they are built
- Less than 1% water systems and toilets that are monitored regularly for the long-term after they are built
Long-term services monitoring is critical for the ongoing improvement of implementing organization practice and understanding, as well as donor policies. Beyond helping individual organizations learn from their experience, services monitoring could reveal geographical or sectoral trends. What if each year, USAID, other government aid agencies, development banks, and major foundations pooled a portion of their funds for water and sanitation projects? (In fact, USAID’s recently published Water and Development Strategy indicates that USAID “will seek investments in longer-term monitoring and evaluation of its water activities in order to assess sustainability beyond the typical USAID Program Cycle and to enable reasonable support to issues that arise subsequent to post-completion of project implementation.”) These funds could be used to ensure services monitoring for all (or a sample of) previous water and sanitation systems funded by those donors in a country or region.
With this information, they could identify region-wide problems and solutions. For example, declining amounts of water available from spring-fed systems in a geographic region could point to a need for investing in water source protection and installation of household water meters to reduce leaks and wastage.
A way forward
To remove some of the barriers to ongoing services monitoring, we recommend a way forward below.
- A percentage of funds (perhaps 3-5%) of each donor’s funding for water, sanitation, and hygiene programs is contributed to a pool for services monitoring each year.
- The funds could be used to monitor a sample of past programs funded by the donors. For example, programs that are 5, 10, and 15 years old. That way we get the learning now and can use it to change programs moving forward.
- Keep the monitoring indicators very basic and in line with government monitoring protocols, where present.
- Development organizations should be responsible for ensuring that services monitoring happens, but should not have to use their own staff. For example, where governments have a robust system of national monitoring, the organization could pull recent, relevant government data.
- Engage an independent auditor to verify a sample of results.
As more services monitoring data become available and accessible, we’ll get past the statistics to specifics, leading to learning, and more effective performance. Thus, people in developing countries will have a better chance at reaping the life-changing benefits of safe water for life.
The African Ministers' Council on Water, an initiative of the African Union, has announced a three-year, $2 million grant from the Bill & Melinda Gates Foundation to build its capacity for sanitation policy development, monitoring and evaluation coverage, and WASH-related advocacy across the continent.
Awarded through the foundation's global development program, the grant will be used to provide training and technical assistance in four countries working to develop and adopt effective sanitation and hygiene policies and plans; organize the fourth AfricaSan conference as a mechanism for tracking progress, refining targets, and enabling peer support and advocacy for implementation of the 2008 eThekwini Declaration and AfricaSan Action Plan; and help countries fulfill their obligations to report to the AU.
"We face tremendous challenges of diminishing access to clean water and safe sanitation," said AMCOW executive secretary Bai Mass Taal. "AMCOW is committed to working with partners such as the Bill & Melinda Gates Foundation to reduce this scourge and improve access to safe sanitation, thereby achieving our overall goal of decreasing poverty and disease in the continent."
Source: “African Ministers' Council on Water (AMCOW) Gets US$2 Million Grant to Improve Sanitation Coverage in Africa.” African Ministers' Council on Water Press Release 12/18/12.