Promoting Policy Reform in Honduras

Program Overview

MCC’s $15.6 million Honduras Threshold Program (2013–2019) focused on improving the transparency and efficiency of public financial management (PFM) and public-private partnerships (PPPs). The program aimed to save the government money, improve service delivery, and reduce corruption by providing financial assistance to social accountability organizations and technical assistance to government institutions involved in budgeting, procurement, audits, and management of public-private partnerships.

Download Spanish translated evaluation brief.

Key Findings

Evaluation Questions

This final performance evaluation was designed to answer whether or not the Honduras Threshold program activities:

1. 1
   Improved invoice processing and cash management within the Ministry of Finance?
2. 2
   Increased accuracy of financial forecasting and budgeting within the Ministry of Finance?
3. 3
   Increased transparency and vendor confidence in public procurements?
4. 4
   Improved service delivery in institutions through government audits?
5. 5
   Improved service delivery in targeted institutions through civil society audit oversight?
6. 6
   Increased efficiency and effectiveness of processes for development and management of PPPs?

Detailed Findings

These findings build upon the interim evaluation report results published in 2019.

Strengthening budget and treasury management

SEFIN automated some invoicing processes and worked to close the loopholes permitting institutions to procure goods without financial commitments. The program also helped SEFIN and government institutions build capacity in “baseline budgeting,” programmatic budgeting, revenue projection modeling, and fiscal impact analysis (43 government institutions, accounting for most government spending, calculated basic baseline budgets for the 2019 and 2020 budget years).

Although capacity has increased, the program and its partners had not impacted arrears, consolidated a medium-term budgeting process, or widely expanded fiscal impact analysis by the program’s conclusion. Arrears have continued to peak annually at above $1.2 billion, as has been the case since the start of the program.

Improving procurement capacity, planning and controls

The procurement regulator established an evaluation unit and conducted 21 evaluations of government institutions, implemented 180-hour procurement training courses to 195 participants from 52 public entities, created a process for and certified 139 public purchasers, increased offerings in the e-catalogue, and took steps toward strengthening the overall regulatory framework. However, neither public employees nor vendors registered to sell to the government reported an increase in the perceived fairness of the procurement process. Perceived fairness scores among surveyed vendors remained consistent and within the margin of error of the survey (65 percent at baseline; 66 percent at endline).

Sustainability of the advances remained a risk at the end of the evaluation. The evaluation unit lost staff and conducted only a few evaluations each year. Proposed legislative reforms that are needed to formalize certified public purchasers stalled. An initiative to upgrade Honduras’s procurement platform HonduCompras began late in the program and did not garner adequate participation from the procurement regulator, thus further delaying implementation.

Building government audit capacity, and supporting audits by civil society

Of four pilot performance audits supported by the threshold program and assessed by the evaluation, only one led to confirmed, meaningful change. Across all four audits, only 24 percent of the detailed audit recommendations were implemented. Through an innovative social audit, the Association for a More Just Society (ASJ), with presidential support, media coverage, and a robust follow-up mechanism, found an average improvement in scores of 26 percentage points across six evaluated government institutions.

A pivot to prevent liabilities from public-private partnerships

The threshold program provided technical assistance to Honduras’ nascent PPPs, which included 10 projects worth approximately $1.4 billion as of project close. The threshold program did not gain adequate cooperation from the key PPP stakeholder COALIANZA or from the secretariat of infrastructure (INSEP), limiting the intervention’s impact. The program adapted by concentrating its support on building SEFIN’s capacity to limit the government’s financial risk and by supporting the transfer of INSEP authorities to INVEST-H, a government institution that grew out of the Millennium Challenge Account–Honduras. The program also supported a 200-hour training course, completed by 101 representatives from key PPP institutions. Sixty-nine of these participants obtained an internationally recognized certification. Based partly on concerns raised by the threshold program, the Government of Honduras worked with the Inter-American Development Bank to reassess its approach to PPPs.

MCC Learning

• 
  Be mindful of the scale of the policy and institutional reform within the design of a program. Identify a measurable objective within a specific sector and tie it directly to service delivery outcomes.
• 
  Building data and monitoring systems for partner institutions is critical to transparency and success of the PIR projects. Achieving progress on PIR requires a data-driven approach with measurable key performance metrics within the relevant organization so all parties involved are aligned along the same facts.
• 
  The political economy of the policy reform must be front and center for PIR projects to ensure that the post-program logic of the reform will be sustained.
• 
  A small number of focused evaluation questions aligned with a well-documented project logic leads to an evaluation report that is conducive for learning and accountability.

Evaluation Methods

The evaluation used a longitudinal performance evaluation design that triangulated quantitative and qualitative data sources over time.

Budget and treasury activities were evaluated using quarterly administrative data (e.g., to track changes in arrears and payment speed) and changes in public expenditure and financial accountability indicators between the periods 2009–2011 and 2016–2018. The evaluation also used document reviews and extensive qualitative interviewing to assess adoption of promoted reforms in invoicing, budgeting, and fiscal impact analysis.

The evaluation team qualitatively explored procurement strengthening activities in the health, education, infrastructure, and security ministries. The evaluation also included a pre-post assessment of changes in vendor perceptions of the procurement process between 2016 (n=850) and 2019 (n=834), prior to the program’s conclusion but with adequate time to observe the influence of evaluations, training, rule changes, and the electronic catalogue. The evaluation team also qualitatively examined PPP activities through a comparative case study of six PPPs and a one-time minisurvey of government participants in a PPP training course. Government performance audits and ASJ social audits were assessed using the TSC and ASJ’s own recommended implementation tracking and changes in ASJ scoring over time. This assessment was complemented by qualitative interviews and changes in a pre-post survey of public employees in three government institutions.

Qualitative interviewing was conducted in February 2016, August 2017, and August 2019 (three months after program close) with around 100 participants at each phase. The Threshold Program agreement was signed in 2013, and roughly a dozen subactivities either evolved or were phased in over the life of the project. Exposure period for these activities ranged from two to three months.

2020-002-2475