Making an Impact: Innovative Evaluation in WASH Education

Editors Note: This guest post was authored by Nicole Rosenleaf Ritter who is the communication specialist for the Project WET Foundation, a nonprofit based in Bozeman, Montana, and active in more than 65 countries. Nicole discusses Project WET’s evaluation work, including its innovative use of mobile technology and collaboration with Engineers Without Borders. All of the WASH education materials referenced are available for free download on the Project WET web site, including formal reports regarding the northern Uganda evaluation process.

Teachers in rural northern Uganda used mobile phones to complete follow-up surveys about the impact of WASH education. Credit: © Project WET Foundation

Teachers in rural northern Uganda used mobile phones to complete follow-up surveys about the impact of WASH education. Credit: © Project WET Foundation

The Lake Victoria Primary School (LVS) in Uganda offers an object lesson for anyone curious about the importance of WASH education. As part of a water provision project, gutters and downspouts were constructed and a water tank was provided to LVS to allow the school to harvest rainwater. What wasn’t provided as part of the project was education — either about the benefits of rainwater harvesting or the way such a system would be set up. As a result, the water tank ended up unused and in storage with small, potentially contaminated containers capturing only a tiny fraction of the usable water. Meanwhile the school was shelling out US$600 per month for municipal water that was available from only one tap.

Enter Aggrey Oluka, LVS’s head science teacher. Fed up with high rates of waterborne illness and unsustainable water bills, Oluka went looking for solutions. He found the Project WET Foundation, which was launching a new program in Africa with USAID to create culturally appropriate WASH-related educator guides, children’s books, and classroom posters, and to train teachers to use them. He participated in writing workshops for the materials and enthusiastically adopted them for use at LVS. Applying the knowledge he and his students had gained, Aggrey was able to finally get the rainwater harvesting system implemented — to the benefit of the students’ health and the school’s budget. The municipal water bill dropped to just US$30 per month, and students had increased access to clean water for healthy habits such as hand-washing.

Thanks to experiences like this one, the value of WASH education is finally being recognized. The WASH Sustainability Charter gives WASH education top billing, with the preamble singling out “the lasting provision of safe water, sanitation, and hygiene education” as a “leading development priority of our time.” (For more information on the Charter, read this WASHfunders guest post.)

Given the growing influence of WASH education, finding ways to measure and evaluate educational programs and projects is also increasingly important and challenging. While it stands to reason that a populace educated in the basics of water, sanitation, and hygiene will be better able and more likely to actively participate in sustaining water provision projects, the evidence surrounding WASH education’s impact on behavior remains in early stages.

As a longtime proponent and provider of WASH education, the Project WET Foundation is exploring new and innovative ways to evaluate the results of its WASH education materials implementation. Targeting youth through school and community educators, Project WET materials, including educator guides and lesson plans, colorful children’s activity booklets, and durable classroom posters, teach children, parents, teachers, administrators, and government officials  about WASH using games, songs, role-plays, whole-body learning, subject integration, and other interactive pedagogical methods.  Educators learn to use the materials through a train-the-trainer process, and students share the lessons they learn at home and in the community. Materials have been translated and localized on three continents, reaching millions of children.

So what happens after the intervention? Getting to that answer is critical for all WASH education actors, but it is not easy. Partnerships and novel approaches can help.

Following implementation in rural northern Uganda in 2009, Project WET used traditional site visit and survey techniques to evaluate results in 2010. However, recognizing the limitations of those techniques — particularly the lack of reliable internet connectivity for e-mail follow-up, the prevalence of teacher transfer, and the time constraints for staff — Project WET teamed up with Adam Lerer, a PhD candidate from the Massachusetts Institute of Technology. Lerer’s thesis work focused on using the Open Data Kit voice system, a web-based interactive voice response technology that allows the user to design and record survey calls to be received on any kind of mobile phone.

The innovative combination of mobile phone technology plus traditional techniques, allowed successful follow-up with 92 of the 500 schools where educational materials were implemented at a cost of US$5.90 per teacher and US$.06 per student. The mobile phone surveys allowed teachers and schools to respond quickly and easily, boosting return rates. The data collected showed that of the schools who responded to the follow-up, 90 percent were still using WASH education materials with their students a year after implementation, and 92 percent reported positive changes in student behavior related to water, sanitation, and hygiene. Increased hand-washing behavior and facilities were most often reported, as well as healthy behaviors relating to water storage, cleaning of the latrine, and water sources. At least 25 percent of users reported that the materials had also been shared with the wider community, supporting the notion of schools as gardens where ideas can grow and be shared.

While mobile phone surveys provide a promising, low-cost evaluation technique, extended on-the-ground monitoring is likely to remain the gold standard for independent assessment. However, many nonprofit organizations cannot afford to carry out such programs on their own. By partnering with the Montana State University chapter of Engineers Without Borders (EWB), Project WET has been able to both wrap education around water provision projects EWB has underway in Kenya and tap EWB student leaders to carry out year-by-year site evaluations in areas where Project WET materials have been implemented. The results of this three-year study — due out in 2015 — should shed light on the effects of WASH education on water provision projects and communities.

By partnering inside and outside the WASH sector, experimenting with innovative techniques, and — most importantly — sharing lessons learned, WASH education practitioners can improve the work being done and secure education’s place at the center of WASH sustainability.