Improving Water and Sanitation in Zambia

Program Overview

MCC’s $355 million Zambia Compact (2013–2018) was designed to increase residents’ access to water, sanitation, and drainage services and improve infrastructure and state capacity in these sectors. The Zambia Compact funded the $6 million Innovation Grant Program, an initiative to support innovative opportunities and partnerships in the water, sanitation, and solid waste management sectors. The program awarded five grants during grant cycle 1 (starting in November 2015) and nine grants during grant cycle 2 (beginning in November 2016).

Key Findings

Evaluation Questions

This final performance evaluation answered a series of questions related to each phase of the Innovation Grant Program including startup, selection, implementation, and completing the award. A subset of the overarching evaluation questions includes the following:

1. 1
   Did the program achieve its objectives following the implementation model envisioned in program documentation?
2. 2
   Was grant oversight sufficient? Was it cost-effective?
3. 3
   Did the program achieve the outcomes envisioned in key peri-urban areas of Lusaka?
4. 4
   Did the grant selection process prioritize interventions based on its key objectives, and was the process an efficient and effective way to identify the “best” projects?

Detailed Findings

Achievement of Program Objectives

Community members in intervention areas perceived the greatest benefits in solid waste management, noting improvements in garbage collection, reductions in plastic debris, and improved drainage systems in Lusaka. Many beneficiaries reported higher income and used this money to meet basic needs. According to the final report from the program manager, the IGP created 955 jobs, which generated $642,493 in income. Beneficiaries involved in solid waste management interventions in particular described improvements in their income and used the money they earned to meet needs such as food, rent, and school fees.

Although public-private partnerships created by the program helped leverage private sector resources, these arrangements experienced challenges in effectively engaging with public sector actors due to the risk of leadership changes or shifts in priorities within key sector institutions.

Deviations from the Implementation Model

Due to delays in the grantee selection process, only two of the four planned grant cycles were implemented. The selection process for grant cycle 1 took a full year—longer than the planned timeline of seven months. MCC worked with the Innovation Grant Program Manager and Millennium Challenge Account–Zambia (MCA-Z) to revise the evaluation criteria in the Innovation Grant Program Operations Manual to improve the application scoring process during grant cycle 2.

Grant Oversight Under the Innovation Grant Program

Grantees, the Innovation Grant Program Manager, MCA-Z, MCC, and stakeholders agreed that grant oversight was sufficient. Cost data confirmed that program oversight was effective in identifying and suspending noncompliant grants. Three of the 14 grants, totaling $727,499, were canceled or suspended by MCA-Z. Oversight also was sufficient to ensure that grantees worked toward their cofinancing obligations, but the cofinancing data suggested that oversight may not have been sufficient to ensure that grantees fully met their obligations. Indeed, as of final grant reporting, only one grantee had fulfilled or exceeded its cofinancing obligation. Finally, grant management and oversight costs, relative to grant disbursements, were in line with comparable grant-making mechanisms: management costs represented 92 percent of the value of the productive grants disbursed through the IGP and 71 percent of the value of the total grant funds, including cofinancing. Oversight costs were comparable to those of other grant-making mechanisms relative to the amounts disbursed.

Effectiveness of the Selection Process

Three out of nine grant cycle 2 grantees could not complete their projects within the program’s lifespan and did not have a proper closeout period due to implementation delays. The fact that grantees also chronically failed to meet quarterly milestones on time suggests that the selection process was not effective in screening for high-quality projects. Yet grantees under grant cycle 1 were more successful than their grant cycle 2 counterparts in achieving monthly milestones on time, suggesting that selection was more effective during the first cycle.

MCC Learning

• 
  MCC should adopt a set of common approaches and requirements for the design and implementation of grant facilities.
• 
  Undertake greater screening and due diligence of potential grant projects during the compact development period.
• 
  Carefully assess risks of reliance on weak or unresponsive sector institutions that may be vital to implementation and sustainability.
• 
  Consider the costs and benefits of a grant facility as opposed to other implementation modalities in order to achieve project objectives.

Evaluation Methods

The evaluation used several quantitative and qualitative methods, including an exit survey of concept note workshop and proposal workshop participants from grant cycle 2; a cost analysis; key informant interviews with officials from the Lusaka City Council, Lusaka Water and Sewerage Company, Innovation Grant Program grantees, and unsuccessful program applicants; and focus group discussions with MCA-Z officials, officials from the Innovation Grant Program Manager, the Technical Evaluation Panel, the Innovation Grant Program Investment Committee, and community beneficiaries.

The evaluator surveyed 59 participants from the concept note workshop and 35 participants from the proposal workshop in April 2016. Data were also compiled on the costs of overseeing the compact’s Innovation Grant Program component, which were provided by MCA-Z and MCC. Implementation began in January 2014 and concluded in November 2018. The evaluator conducted six rounds of data collection between February 2016 and September 2018 to investigate the four phases of the Innovation Grant Program grant process, including start-up, selection, implementation, and completion. This represents an exposure period of approximately 4 years and 8 months. In total, the evaluator conducted 94 surveys, 71 key informant interviews, and 53 focus group discussions.

2020-002-2406