Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

  • Interim Evaluation Brief
  • March 2022

Reforming the water sector in Sierra Leone

Water utility’s capacity increased, but financial performance is low

View as PDF

Program Overview

MCC’s $40.5 million Sierra Leone Threshold Program (2016–2021) focused on establishing a foundation for the effective and financially viable provision of electricity and water services in Freetown. The $13.6 million Water Sector Reform Project (WSRP) aimed to improve sector coordination, strengthen commercial practices and enhance the Guma Valley Water Company’s (Guma) service provision. The $7.6 million Regulatory Strengthening Project (RSP) aimed to build the capacity of the new regulator, improve sector governance and support the long-term financial sustainability of the water sector.

Evaluator Description

MCC commissioned Social Impact to conduct an independent interim performance evaluation of the WSRP and RSP. Full report results and learning https://data.mcc.gov/evaluations/index.php/catalog/256.

Key Findings

Coordination and Planning

  • The WSRP’s support defined sector goals and clarified stakeholder roles, but the sustainability of these outcomes and implementation of the sector roadmap appear weak.

Operational Capacity and Financial Performance

  • Trainings, system updates and technical assistance helped Guma improve collections, customer engagement, human resources and data management.
  • While collection efficiency improved, operating costs also increased and non-revenue water remained high. Thus, there is no conclusive evidence of Guma’s improved financial performance.

Regulatory and Tariff Reform

  • Guma was unable to comply with tariff application requirements until 2021. Negative press and political interference affected earlier tariff adjustment attempts.
  • The Results-Based Financing Activity under the RSP contributed to better management of pipe leaks and an increase in government collections.

Water Infrastructure Pilot and Service Quality

  • The evaluation found no evidence that pilot activities were effective in reducing non-revenue water and improving customer satisfaction.
  • Water shortages led to rationing for both connected customers and kiosk users.

Evaluation Questions

This interim performance evaluation was designed to answer the following questions to inform the final evaluation.
  1. 1 To what extent has coordination and planning within the water sector improved as a result of the WSRP?
  2. 2 What evidence is there that Guma is improving its financial performance as a result of the WSRP?
  3. 3 How have the activities impacted business operations and strategic planning within Guma?
  4. 4 How useful were the RSP’s activities in realizing regulatory and tariff reforms in the water sector?
  5. 5 How did the pilot activities impact non-revenue water, service reliability and customer satisfaction?

Detailed Findings

Coordination and Planning

Milla water tank provides water in low-income area

Milla tank in the Kingtom district metering area, which provides water in a low-income area.

The Ministry of Water Resources (MWR) led the development of the Urban WASH Roadmap (2019–2030) with WSRP support. The process went smoothly, albeit slower than expected. The WSRP also supported developing the draft WASH policy, which included revised sector targets, new institutional roles, and monitoring and reporting systems that enabled Guma to share data more effectively with donors and stakeholders.

Annual sector reviews and water sector steering committee meetings were useful platforms for coordination and transparency, but they were not held in 2020. The ownership and implementation of these activities after the WSRP appear weak due to the large scope and MWR’s limited financial and institutional capacity.

Operational Capacity and Financial Performance

Guma staff reported improvements in tracking billing and collections, targeting potential customers, and monitoring non-revenue water. Field staff and offices were equipped with new technology, allowing for a shift to digital record keeping and an increase in the frequency, accuracy and timeliness of reporting. Prior to the WSRP, Guma was two–three years behind in submitting audited accounts, but it is now submitting reports ahead of schedule. The WSRP’s targeted technical assistance, enhanced through results-based financing, that adapted to meet the utility’s needs and high institutional buy-in from management was cited as a contributing factor in raising staff performance and morale.

Despite investments in management and accounting software and new technology, along with management’s increased emphasis on achieving billing and collection targets, Guma still had low financial performance at the end of the WSRP due to high rates of non-revenue water and a low tariff.

Regulatory and Tariff Reform

Guma’s tariff, as set by the regulator, is not cost-reflective and has not increased since 2016. A tariff dry run revealed gaps in Guma’s reporting. The RSP’s technical assistance aimed to address data deficiencies. While a tariff application was submitted to and passed by the regulator in 2019, MWR did not approve the application. Political pressure to keep tariffs low made it difficult for the utility to recover its costs and provide better services. Guma’s high losses also made setting a cost-reflective tariff difficult.

The Results-Based Financing Activity allowed the regulator to oversee utility performance and shadow validation exercises for performance metrics. Building off technical support, Guma achieved or exceeded several of its targets in four of the activity’s performance areas: billing and collections, leak management, supply reliability, and regulations. Most stakeholders believed the activity had a positive impact on the utility’s internal operational efficiency through staff motivation, communication and data flow. Guma’s management continued to support the implementation of results-based targets after the activity ended.

Water Infrastructure Pilot and Service Quality

repondents reported mixed results of treatment areas versus comparison for water service

Percent of respondents who think the reliability of their water service is better, the same, or worse than it was in 2018

The pilot activities (i.e., pipeworks, metering, water kiosks and customer awareness campaigns) aimed to identify effective strategies in non-revenue water reduction and customer service improvement to replicate across Freetown. For the infrastructure pilots, there were significant project delays due to disagreements in design, planning, resettlement plans and contractor procurement, which pushed the work into 2021. A total of 1,149 customers were metered, 3.12 kilometers of pipes were updated, and 10 kiosks were constructed in two project district metering areas.

The interim evaluation’s household survey coincided with the completion of pilot activities, making it too early to assess the activities’ effectiveness and replicability. However, the evaluation identified several threats to sustainability. As of May 2021, flow meters intending to measure water supplied to project areas were bypassed to improve water flow; metered customers were not yet billed; kiosks struggled to receive adequate water for operations, which threatened their financial stability; and an unreliable water supply led to high water rations, with many survey customers dissatisfied with service. Non-revenue water across Freetown remained largely unchanged from pre-program levels.

MCC Learning

  • To better align with MCC’s focus on results, threshold programs need to specify clear, credible, and measurable targeted results (and/or program logics) that are linked to evidence-based timelines for when results can be expected.
  • Results based financing can be a valuable complementary tool contributing to institutional reform, but requires sufficient time for design and implementation
  • Simplify large implementation contracts where possible and emphasize quality rather than quantity of deliverables.

Evaluation Methods

Evaluation methodology
This interim evaluation is an ex-post performance evaluation, occurring at the end of the Threshold Program’s water sector investments. It uses program monitoring data, secondary studies, documentation and indicator data to construct a pre-program baseline and conduct a pre-post analysis. Data collection activities occurred between January–June 2021, both in-person and remotely, coinciding with the completion of project activities. The exposure period was 24 months.

Mixed methods data collection included an extensive document review, secondary data from the Threshold Program activities and Guma, key informant interviews with sector stakeholders, household surveys, focus groups with community members, and direct observations of kiosks and installations of pipes, meters and connections. The evaluator purposively selected respondents to conduct remote interviews using semi-structured, tailored tools. For the in-person survey, the evaluator selected 1,015 households from 2018 and 2019 customer mapping exercises in project areas (Aberdeen and Kingtom) and in one non-project area (Cockle Bay). The evaluator then selected survey respondents from Aberdeen and Kingtom to participate in the in-person focus group discussions.

The evaluator’s analysis triangulated information across data sources to identify emerging trends and aggregate data on common themes and findings.

Next Steps

A final evaluation covering a subset of evaluation questions is planned, and results are expected in 2023.

2021-002-2606