Reforming the Water Sector in Sierra Leone

Program Overview

MCC’s $40 million Sierra Leone Threshold Program (2016-2021) focused on establishing a foundation for the effective and financially viable provision of electricity and water services in Freetown. The $14 million Water Sector Reform Project aimed to improve sector coordination, strengthen commercial practices, and enhance service provision of the Guma Valley Water Company (Guma). The $8 million Regulatory Strengthening Project aimed to build the capacity of the new regulator, improve sector governance, and support long-term financial sustainability of the water sector.

Key Findings

Evaluation Questions

This final performance and impact evaluation was designed to answer the following questions:

1. 1
   To what extent have coordination and planning within the water sector improved?
2. 2
   What evidence is there that Guma improved its financial performance? What were the effects of the activities on business operations and strategic planning within Guma?
3. 3
   How useful were the project’s activities in realizing regulatory and tariff reforms in the water sector?
4. 4
   How did the pilot activities impact non-revenue water, service reliability, and customer satisfaction? Are there systems and results in the DMA pilot that can be sustainably replicated to other Guma service areas?

Detailed Findings

These findings build upon the interim evaluation report results published in 2022.

Coordination and Planning

Throughout implementation, the Water Sector Roadmap and Coordination Activity faced challenges in competing priorities; a lack of sector leadership, particularly at the ministry level; and a lack of funding for post-project implementation. The Threshold Program’s Accountable Entity drove sector coordination to facilitate project results, but the program did not contribute to sustained sector planning. By the end of the threshold program in 2021, neither the roadmap nor the established steering committee was active. Additionally, revisions to the water, sanitation and hygiene policy that were proposed during the project were not yet finalized. The evaluation also found no sustained improvements in coordination and planning at endline.

Utility Performance

A 2021 tariff adjustment significantly increased revenue collection, but increases did not keep pace with inflation. Monthly revenue collection increased 50 percent (by one million leones) when the tariff went into effect in October 2021. Since 2017, technical assistance and the Results-based Financing Activity have improved commercial operations and contributed to a steady increase in revenue collection of approximately 14,000 Leones per month. In the first 10 months of 2022, the utility continued to operate below cost recovery, with a negative net cash flow of 11.3 million leones.

The utility continued to use standard operating procedures in customer connections, billings, and complaint resolution, and the project developed these procedures. The utility also retained a twinning agreement with the Ghana Water Company after the program, which provided an opportunity for ongoing learning. The utility expanded its platforms for bill distribution and payment, increased its communication outlets, and decreased its response time to water leaks. However, outputs like the Guma Business and Investment Plan and the Strategic Performance Improvement Plan are no longer being used.

Technical assistance from the project and activities, like Results-based Financing, generated data and data systems that the utility institutionalized at the end of the project through the establishment of an ME&A unit. The unit uses data to identify and rectify issues, like customer non-payment, and has improved the timeliness and quality of the utility’s reporting to the regulator and auditors.

Regulatory Reform

In 2021, the regulator approved Guma’s first tariff increase in five years using policies developed under the threshold program. To enhance the utility’s capacity to collect and report on key performance indicators, institutional support to the utility built on lessons learned during the dry run and improved communication between Guma and the regulator.

Regulations approved during the program helped clarify the regulator’s role and introduced the regulator as a key sector player. However, the regulator faced challenges in providing independent monitoring and oversight of utilities. Utilities were in arrears on levy payments that were meant to strengthen the regulator’s financial independence. The regulator could also set performance targets for the utility and sup­port utility monitoring and problem-solving when issues arose, but the regulator was limited in the actions it could take when targets were not achieved.

District Metering Area (DMA) Pilot

Work in two project DMAs was completed at the end of the program due to delays in procurement and the COVID-19 pandemic. As a result, there was insufficient time to address early and critical issues in the system, including problems with customer meter readings. Over 80 percent of metered customers in the pilot DMAs have been billed a flat rate since commissioning in 2021 due to an accounting error in Guma’s system. Bulk meters that intended to capture water intake to the DMAs were bypassed, and non-revenue water was not measured. In addition, while it was outside of the project’s scope to address Guma’s limited water supply, a limited supply was not adequately accounted for in the DMA design and kiosk construction. DMA kiosks had limited operating hours, and seven out of 10 were temporarily or permanently closed during endline observations. Kiosk operators note the current model was not sustainable due to a limited piped water supply.

Customer outcomes have not improved since DMA implementation. Piped water supply decreased during the rainy season and remained unchanged during the dry season. The decrease in supply was significantly less in pilot locations than the non-project area, which was potentially attributable to the pipe works done under the project. Households across project and non-project DMAs responded to the limited supply by diversifying their water sources (including increased use of kiosks). Satisfaction with water quality remained high at 85 percent but satisfaction with the cost of water dropped in pilot locations (from 99 percent in 2021, to 73 percent in 2023). Water comprised a smaller portion of overall household expenditure in 2023 compared to 2021 suggesting that the cost of water did not rise at the same rate as other household expenditures. Finally, while Guma improved customer communications regarding outages and repairs, many focus group respondents noted a poor response to individual complaints.

MCC Learning

• 
  Threshold programs should consider the likelihood for success when incorporating infrastructure investments that are inextricably linked to larger constraints not addressed in the program.
• 
  Encourage local stakeholder leadership and buy-in to increase the sustainability of policy and institutional reform efforts.

Evaluation Methods

This final evaluation is an ex-post performance evaluation, occurring at the end of the threshold program’s water sector investments. It makes use of program monitoring data, special studies conducted by contractors and third parties, documentation, and indicator data to construct a pre-program baseline and conduct a pre-post analysis. Data collection activities occurred between November 2022 and February 2023.

Mixed methods data collection included secondary data, such as program monitoring data collected through the Sierra Leone Threshold Program Monitoring and Evaluation Plan, information available from project activities, and monthly operational data that Guma collected. Key informant interviews were conducted as semi-structured, tailored tools. An in-person survey was conducted from January to February 2023 in two project DMAs and in one comparison area. Households were tracked from the 2021 interim evaluation, though approximately 25 percent were replaced at endline due to high urban mobility. When feasible, the evaluation included 2018 baseline data collected that a third party collected from the same pool of respondents. Eight focus group discussions were conducted in February 2023 with community members from pilot DMAs. Participants were selected from household survey respondents. Direct observations were also made in project DMAs in November and December 2022.

The evaluator’s analysis triangulated information across data sources to identify emergent trends and aggregate data around common themes and findings.

2023-002-2860