Reducing Households' Water-Related Burdens in Jordan

Program Overview

MCC’s $273 million Jordan Compact conducted a public outreach campaign targeting household water use; provided household water infrastructure; and trained female plumbers through the $4.7 million Water Smart Homes Activity (WSH) in Zarqa Governorate. The activity was built on the theory that educating households around safe water practices and providing in-home water management infrastructure would lead households to shift from expensive vendor water to network water, reducing overall expenditure.

Key Findings

Evaluation Questions

This final impact and performance evaluation was designed to answer the following key questions:

1. 1
   Did the WSH Outreach Campaign lead to significant changes in household water handling and storage that manifested in a) improved water potability and b) improved water efficiency at the household level?
2. 2
   Did the WSH Infrastructure Works Program lead to infrastructure improvements among beneficiaries?
3. 3
   Did National Aid Fund (NAF) beneficiaries experience economic benefits from these infrastructure and behavioral changes, specifically through reduced coping costs, increased household productivity, and/or improved health and well-being?
4. 4
   Did the plumbers training portion of the Water Smart Homes intervention lead to long term capacity and employment opportunities for the women who participated?

Detailed Findings

Outreach Campaign Message Recall and Response

There was no evidence that the WSH outreach campaign changed household water handling, storage, efficiency or potability in Zarqa, as it was intended to do. To start, evidence was mixed as to whether households recalled WSH messaging. Results comparing treated Zarqa households to comparison households in Amman show that the former were slightly more likely, by 3 percentage points, to recall WSH messages than controls. However, these households were also more likely to recall placebo messages not shown in their neighborhoods, by 5 percentage points, suggesting possible response bias. Treatment households were then only 1 percent more likely to report making major changes to water-related behaviors, and therefore household’s water-related costs were ultimately not reduced by the outreach campaign, which was the objective of the project.

The evaluators recommend better integration of future behavior change communication investments with related infrastructure work. During implementation of the Jordan compact, it was discovered that many households preferred shop water, erroneously believing that network water quality was poorer. The theory of change for the infrastructure investments assumed households would shift towards consumption of cheaper network water. This did not materialize, but possibly could have if the communications investments focused on it.

Household Water Infrastructure Works Receipt and Usage

The WSH Activity led to infrastructure improvements among beneficiaries. Most, but not all, households identified as needing infrastructure improvements received them through WSH. However, not all did. Booster pumps were received by only 40 percent of households that needed them and sewer improvements were received by only 8 percent that needed them. On the other hand, some households received infrastructure they did not need, including booster pumps, steel pipes, and sewer improvements. This disparity may be due to changes in households’ needs between the initial assessment in 2014 and delivery during 2015 – 2016, or could indicate incorrect delivery of infrastructure. The majority of households still had improvements four years after the program.

In terms of the resulting economic benefits, households were 9 percentage points less likely to use tankers as their primary water source compared to non-recipient households. Reduced dependency on tankers was reflected in reduced time (eight fewer minutes/week) and reduced costs (4 Jordanian Dollars less/month) for obtaining tanker water, but there was no statistically meaningful reduction in overall household water-related costs, which was the project objective. This could be due to lack of a large enough sample size to detect this change or due to compensating household behavior, such as spending more money on another water-related cost. Similarly, no changes in productivity, health, or well-being were detected though, again, the sample size was small. The evaluator recommends more granular cost-benefit analyses on future investments of this type, as some were quite costly and the outcomes they targeted not likely to be achieved.

Women Plumber Program

Evaluation results show the women plumber program led to long-term capacity and employment opportunities for the 30 women who participated. Respondents reported increased confidence in all skill areas taught, but this increase was greater in their business and marketing skills than technical plumbing skills. Most participants also reported improved employment, and the majority (17 out of 26 respondents) were still working in plumbing even four years after the program. Participants’ median personal income increased by 70 Jordanian Dinars/month, as did overall household income, though it is not possible to attribute this increase to the women plumber program due to the evaluation design, and it is known that another similar program was operating at the same time. Participants suggested the training could be improved by increasing the timing and focusing more on practical training instead of theory.

This investment did not contribute to the project objective of decreasing households’ water related costs. It was never logically related to the project’s objectives.

MCC Learning

• 
  Investment objectives and the corresponding investment design need to be agreed and documented in the compact agreement.
• 
  Be sure project designs are based on thorough due diligence.
• 
  Consult a wide range of stakeholders to ensure the theory of change is feasible.
• 
  Ensure a logical link between gender-focused investments and the objective of the project.

Evaluation Methods

The results of the outreach campaign and household infrastructure works were measured through two impact evaluations using regression analysis. The outreach campaign used Zarqa areas covered by the public messaging but outside the main compact target areas as the intervention (treatment) group, and areas in Amman that did not receive such messaging as the control. These sub-samples are drawn from the household sample used in the impact evaluation of the full Jordan Compact. The infrastructure sample consisted of NAF beneficiary households in Zarqa. Only NAF households were eligible to receive infrastructure, with the treatment group composed of recipients and the control group composed of NAF households that were screened but deemed ineligible to receive the infrastructure support based on the household’s current infrastructure conditions, land tenure, and socio-economic characteristics at the time of the assessment.

The women plumber program assessment was an ex-post performance evaluation based on interviews conducted with program trainees after compact completion.

Exposure Period: Depending on the evaluation question and associated intervention, and timing of specific data collection, respondents experienced a 20 – 49 month exposure to the program interventions.

Data Collection:

• Outreach Campaign: The endline household survey was conducted in Zarqa and Amman in mid-2018, with 3,662 interviews.
• Infrastructure Works: The survey of NAF beneficiary households was conducted in Zarqa in early 2019, with 140 interviews.
• Women Plumber Program: Interviews were conducted with 26 of the 30 participants in early 2019.