Closeout economic rates of return (ERRs) are estimates of project rates of return calculated at project completion, when costs and any change of project scope are known, but benefits are largely still unknown. This report documents facts on Economic Rates of Return (ERRs) on MCC supported projects to serve as a starting point for policy discussion. Key findings from this report are that (i) closeout ERRs have been produced for the majority of projects and MCC funds, (ii) closeout ERRs remain on average above MCC’s 10 percent required threshold, (iii) more than one-third of closed projects have closeout ERRs below MCC’s threshold and (iv) roughly three-fourths of projects have ERRs that decrease from the time the compact Enters Into Force (EIF) to closeout.
Authors
Sandra Ospina (Economic Analysis, Senior Director), Ospinasp@mcc.govMarissa Block (Economic Analysis, Research Assistant)
Introduction
This is MCC’s third annual report on Economic Rates of Return (ERRs). The main purpose of the report is to take stock of projected ERRs for compacts that have closed (“closeout ERRs”) and compare them with initial ex-ante ERRs (“original ERRs”). The report also presents facts on original ERRs for compacts under implementation (Open Compacts).ERRs play a central role at MCC. Original ERRs represent projections of project costs and benefits over time, based on data available at the beginning of a compact. A project’s original ERR is calculated during compact development and is used to determine whether MCC will fund the project. Closeout ERRs are updated estimates of original ERRs done shortly after the compact ends. While the costs and final scope of a project at closeout are known, the benefits are still largely unknown. Since benefits extend over 20 years after the completion of a project, closeout estimates are necessarily still projections. Therefore, closeout ERRs should not be interpreted as the final ERRs of MCC projects. It is useful to track how ERRs may vary or change in implementation by several characteristics such as country, year, sector or size. It would also be useful to derive and test hypotheses based upon any patterns documented in this report. However, this report does not attempt to do so. Nor does it attempt to test causal relationships or provide policy recommendations. Nonetheless, it is useful for learning and accountability purposes to compare them with the original estimates. Further analysis may be pursued in a subsequent report.
The key takeaways from this report are that (i) closeout ERRs have been produced for the majority of projects (63%) and MCC funds (66%), (ii) closeout ERRs are on average above MCC’s 10 percent threshold (13.2%), (iii) more than one-third of closed projects have closeout ERRs below MCC’s threshold and (iv) roughly three-fourths of projects exhibit ERRs that decrease from the time the compact Enters Into Force (EIF) to close.
The report is organized as follows. Section II describes coverage. Sections III, IV and V describe aggregate and cumulative information on ERRs. This information includes facts on ERRs by sector, year, project size and compact among other useful ways to tabulate the data.
Coverage
Table 1 provides the number of closed projects to date, funds committed and funds disbursed. It disaggregates these numbers according to projects with ERRs, with pending ERRs, projects for which ERRs are not planned and projects that were cancelled. The Economic Analysis (EA) division started updating ERRs upon compact closeout in 2011. At end-2016, EA has estimated and published closeout ERRs for 86 projects on MCC’s external website.[[Jordan’s compact ended by end-2016 but closeout ERRs are due July 2017.]] As Table 1 shows, MCC has completed 137 projects with a disbursed value at compact closeout of approximately US$6.5 Billion. These projects span 21 compacts. Once the remaining seven closeout ERRs are updated, 77 percent of all funds disbursed will have a closeout ERR (68 percent of all projects).There may be various reasons why some projects may not have a closeout ERR. We tabulated the stated reasons provided by country economists assigned to each compact for the lack of a closeout ERR in Table 6 in Annex II.[[Table 6 also tabulates stated reasons for the lack of original ERRs. When projects lack both the original ERR and the closeout ERR, the stated reasons usually coincide.]] The three broad explanations given were compact termination and cancelled projects, insufficient data at closeout, or small project size. Specifically, five projects were cancelled due to termination of the Mali and Madagascar compacts. In several cases, there was insufficient data at closeout to update the ERR. Some projects were considered too small (i.e. less than $10 million) to justify the estimation of a closeout ERR.[[A caveat on the stated reasons in Table 6 is that they are survey-based. As such, they may not represent true underlying causes.]]
Table 1
| Number | Funds Committed | Funds Disbursed | |
|---|---|---|---|
| Total Closed Projects | 137 | 6,560.47 | 6,503.98 |
| Projects with Closeout ERR | 86 | 3,888.59 | 4,260.64 |
| Projects without Closeout ERR | 51 | 2,671.88 | 2,243.34 |
| Closeout ERR pending | 7 | 926.23 | 733.7 |
| Projects for which no ERR Will Be Calculated | 44 | 1,745.65 | 1509.64 |
| Completed, but no ERR Planned | 28 | 895.15 | 888.52 |
| Terminated or Cancelled Projects | 16 | 850.5 | 621.12 |
Characteristics
We note that in principle ERRs are estimated at the lowest level of disaggregation – where any activity or cluster of sub-activities can be separated in terms of delivering defined benefit streams and therefore for which a decision on investment can be made separately. The lowest possible level of logical disaggregation may occur at the sub-activity level. When reporting summaries for compact statistics—as we do here—ERRs are sometimes aggregated by project or by activity using funds as weights. The majority of reported ERRs, 64 out of 85 closeout ERRs, are calculated at the activity level while the remaining 21 are calculated at the project-level.Table 2 shows weighted and unweighted ERR averages for all closed projects with an ERR and by selected categories. It also shows the number of projects included in the respective average calculation and the funds disbursed. The table shows the overall average, averages by sector, by closing year, by three different project sizes, and by compact. The compact category is organized according to GDP per capita in the year he country was selected.
The bottom line entry in Table 2 shows the average ERR for the 86 projects. The weighted average (by disbursed funds) at compact closeout is 13.2 percent. The ERRs reported in Table 2 range from -5.5 to 34.7%.
While the reported averages per sector and per project size usually do not vary dramatically, the averages by year and by country compact show high variability. The average closeout ERR for compacts that closed in 2015 is lower than other years. The compacts that closed in 2015 are Moldova and Senegal. In the compact category the weighted average ERRs range from -5.5 to 20.3%. Moldova’s ERR at -5.5 percent is the lowest overall followed by Nicaragua, Burkina Faso, Tanzania, Senegal, and Lesotho. El Salvador, Morocco, Ghana and Namibia’s weighted closeout ERR stand out at around 20 percent. We note that the number of compacts reporting all ERRs planned is 17.[[Note that these differences across countries, years, and sectors in Table 2 should not be interpreted as statistically significant or indicative of meaningful comparative differences in performance across countries due to the very small sample size.]]
Figure 1 depicts all projects with an original and a closeout ERR. The interactive feature of this graph shows specific project information. The projects are presented from left to right in ascending order according to the original ERR. The red solid line represents the original ERRs and the blue solid line represents the closeout ERRs. The solid black line is drawn at MCC’s current 10% ERR threshold. All 30 projects with a closeout ERR below this threshold, nearly 34 percent of all projects, are labeled.
Figure 1 shows that the majority of the original project ERRs are above their corresponding closeout ERRs. Future work should investigate the proximate reasons for why this is so. More than one third of closed projects have ERRs below MCC’s threshold and roughly three-fourths of projects exhibit ERRs that decrease from EIF to close. Of the 80 projects for which there are original and closeout ERRs, the weighted average for original ERRs is 17.9 percent compared to the 12.8 percent for corresponding closeout ERRs. Figure 1 also reveals substantial variation across project closeout ERRs. They range from -8 percent to 109 percent.
Figure 2 shows the difference in percentage points between closeout ERRs and original ERRs. The figure is organized from lowest to highest difference (closeout ERR – original ERR). The red bars depict the difference in ERRs for projects with ERRs that fell below the threshold while the green bars depict this difference for projects with a closeout ERR above the threshold. Figure 2 shows that for most projects the ERRs decreased, sometimes exceeding a fall of 20 percentage points. The right hand tail in Figure 2 shows the 17 projects for which the closeout ERR increased relative to their original ERR (21% of all projects with original and closeout ERRs). The average of the absolute value of the gap is 5.9 percentage points.
Table 2
| Sector | Weighted Closeout ERR | Unweighted Closeout ERR | Total Projects | Funds Disbursed (US$M) |
|---|---|---|---|---|
| Agriculture and Irrigation | 14.5% | 15.6% | 21 | 1031.20 |
| Cap Bldg & Inst Dev | 34.7% | 34.7% | 1 | 54.90 |
| Energy | 15.1% | 13.7% | 4 | 101.60 |
| Finance, Invest., Trade | 11.5% | 11.0% | 5 | 94.33 |
| Health, Ed. & Community | 15.1% | 21.7% | 14 | 540.94 |
| Land | 12.1% | 13.8% | 6 | 190.90 |
| Other* | 14.2% | 15.3% | 3 | 66.10 |
| Transport | 11.6% | 9.8% | 18 | 1763.83 |
| WASH | 11.7% | 12.7% | 14 | 416.85 |
| Closeout Year | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
| 2011 | 11.5% | 9.9% | 10 | 748.22 |
| 2012 | 20.2% | 14.5% | 15 | 665.51 |
| 2013 | 15.1% | 16.0% | 38 | 1472.91 |
| 2014 | 9.6% | 16.4% | 16 | 543.90 |
| 2015 | 5.0% | 4.4% | 5 | 655.20 |
| 2016 | 19.5% | 23.7% | 2 | 174.90 |
| Size at Closeout (US$) | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
| < 50M | 15.5% | 15.5% | 58 | 1241.15 |
| 50M - 100M | 13.2% | 13.9% | 16 | 1023.25 |
| > 100M | 11.7% | 11.3% | 12 | 1996.25 |
| Country | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
| Mozambique | 15.2% | 20.7% | 8 | 310.80 |
| Burkina Faso | 3.4% | 2.9% | 5 | 332.00 |
| Tanzania | 10.0% | 6.8% | 3 | 77.20 |
| Benin | 12.8% | 11.2% | 3 | 278.13 |
| Senegal | 7.0% | 6.9% | 4 | 553.6 |
| Lesotho | 8.7% | 9.8% | 6 | 297.72 |
| Ghana | 20.1% | 13.9% | 7 | 248.41 |
| Nicaragua | 2.1% | 2.1% | 1 | 57.80 |
| Moldova | -5.5% | -5.5% | 1 | 101.60 |
| Georgia | 12.2% | 8.0% | 3 | 309.70 |
| Mongolia | 12.4% | 13.0% | 7 | 223.39 |
| Philippines | 19.5% | 23.7% | 2 | 174.90 |
| Armenia | 12.3% | 14.6% | 2 | 36.90 |
| Morocco | 20.2% | 19.8% | 14 | 563.80 |
| Vanuatu | 10.3% | 10.3% | 1 | 65.69 |
| El Salvador | 20.3% | 15.1% | 8 | 417.10 |
| Namibia | 19.2% | 22.5% | 11 | 211.90 |
| All Projects | 13.2% | 14.6% | 86 | 4260.64 |
Figure 1
Figure 2
Projects with closeout ERRs below the threshold
Table 3 disaggregates projects with closeout ERRs that fall below 10 percent in different categories. The table shows the number of projects per sector, year, project size and country. The top line entry for the table shows that the average ERR for the 30 projects is 3.2 percent corresponding to US$1.6 billion or 25 percent of disbursed funds. The table shows that more than half (57 percent) of the projects are small projects (<US$50M). However, only a quarter of all small projects fall below the threshold, whereas half of all projects in the US$50-$100M category fall below the threshold Finally, one third of the disbursed funds correspond to three high value projects that sum up to US$533.75M. The three projects are Senegal’s RN6 ($262.7M), Mozambique’s Rehabilitation/Construction of Roads Project (US$127.7M), and Lesotho’s Health Sector project (US$128.5M).For Nicaragua, Moldova, and Burkina Faso, all projects fall below the threshold. Almost all sectors have at least one project with an ERR below the threshold. The transport sector has the largest number of projects below the ERR threshold amounting to 56 percent of total transport projects and 46 percent of the value of the sector’s portfolio. Water, Sanitation and Hygiene (WASH) projects are similarly high – 43 percent of projects and 38 percent of the value of the sector’s portfolio. Land projects that fall below the threshold account for 55 percent of the value of the sector’s portfolio, the highest of all sectors. Two of the four energy projects fall below the threshold and account for 36 percent of the value of energy’s portfolio. The three health, education, and community projects account for 50 percent of the sector’s total (US$432.6M), and one of these projects, the Lesotho project on health, accounts for 33 percent of the sector’s total. On the other hand, less than 30 percent of agriculture and irrigation projects fall below the threshold, which accounts for only one quarter of the value of the sector’s disbursed funds.
Table 3
| Sector | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
|---|---|---|---|---|
| Agriculture and Irrigation | -0.1% | 1.1% | 6 | 258.30 |
| Cap Bldg & Inst Dev | - | - | 0 | 0.00 |
| Energy | 7.7% | 8.2% | 2 | 36.90 |
| Finance, Invest., Trade | -0.9% | -0.9% | 1 | 15.10 |
| Health, Ed. & Community | 6.5% | 6.4% | 3 | 218.75 |
| Land | 5.8% | 5.8% | 2 | 105.40 |
| Other | - | - | 0 | 0.00 |
| Transport | 3.5% | 3.3% | 10 | 818.50 |
| WASH | -0.2% | -0.3% | 6 | 160.20 |
| Closeout Year | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
| 2011 | 3.6% | 4.1% | 4 | 183.30 |
| 2012 | 1.0% | 1.2% | 3 | 73.70 |
| 2013 | 4.8% | 2.2% | 11 | 469.95 |
| 2014 | 4.5% | 4.3% | 8 | 427.30 |
| 2015 | 0.4% | 1.5% | 4 | 458.90 |
| 2016 | - | - | 0 | 0.00 |
| Size at Closeout (US$) | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
| < 50M | 1.7% | 1.9% | 17 | 337.30 |
| 50M - 100M | 5.5% | 5.1% | 8 | 539.50 |
| > 100M | 2.1% | 2.2% | 5 | 736.35 |
| Country | Weighted ERR | Unweighted ERR | Total Projects | Funds Disbursed (US$M) |
| Mozambique | 5.4% | 1.7% | 3 | 171.70 |
| Burkina Faso | 3.4% | 2.9% | 5 | 332.00 |
| Tanzania | 2.9% | 2.7% | 2 | 31.50 |
| Benin | 5.8% | 5.8% | 1 | 39.20 |
| Senegal | 2.1% | 3.8% | 3 | 357.30 |
| Lesotho | 3.1% | -1.4% | 2 | 171.55 |
| Ghana | 0.0% | 0.0% | 2 | 53.90 |
| Nicaragua | 2.1% | 2.1% | 1 | 57.80 |
| Moldova | -5.5% | -5.5% | 1 | 101.6 |
| Georgia | 3.6% | 4.2% | 2 | 86.30 |
| Mongolia | 9.4% | 9.0% | 2 | 76.50 |
| Philippines | - | - | 0 | 0.00 |
| Armenia | - | - | 0 | 0.00 |
| Morocco | -0.9% | -1.0% | 2 | 18.70 |
| Vanuatu | - | - | 0 | 0.00 |
| El Salvador | 3.6% | 3.6% | 1 | 19.80 |
| Namibia | 8.3% | 6.7% | 3 | 95.30 |
| All Projects | 3.2% | 2.8% | 30 | 1613.15 |
Original ERRs for Compacts under Implementation
Table 4 shows the number of projects under implementation as well as the funds committed. The table disaggregates the number of projects according to projects with original ERRs, with pending ERRs and projects for which no ERR will be calculated. The table shows that 80 percent of the 25 open projects have an original ERR. They correspond to 83 percent of committed funds. The majority of the funds for which ERRs are not estimated correspond to the Indonesia compact (US$M 383).[[See Annex II for further explanation as to why an ERR was not originally calculated.]] The pending ERR is for Liberia.Table 4
| Number | Funds Committed | |
|---|---|---|
| Total Open Projects* | 25 | 2,406.56 |
| Projects with Original ERR | 20 | 1986.46 |
| Projects without Original ERR | 5 | 420.1 |
| Original ERR pending | 1 | 21.1 |
| Projects for which no ERR Will Be Calculated | 4 | 399 |
Figure 3 shows compact original ERR averages weighted by committed funds for all compacts that have Entered Into Force (EIF) when the five year time limit for completing project disbursements begins. The figure organizes the compacts according to EIF year starting with the earliest compact to EIF. The green bars denote the closed compact original ERRs and the yellow bars the open compacts’ original ERRs The figure shows that for open compacts, except Malawi, the average of the original ERRs appears to be lower than the average of the ERRs for closed compacts. The dotted trending line has been added for emphasis on the apparent downward trend on original ERRs.
Table 5 shows the number of projects under implementation as well as the funds committed. The table shows the number of projects and average ERRs per sector, year, project size and country. The table includes the Jordan compact for which the closeout process is underway. As with closed compacts, countries are ordered based on GDP per capita in the year selected. The weighted average ERR for all 20 projects is 21 percent. If Malawi’s original ERR is replaced with 18.7 percent—the ERR revised one year later—the weighted average ERR falls to 16.3 percent.
Figure 3
Table 5
| Sector | Weighted ERR | Unweighted ERR | Total Projects | Funds Committed (US$M) |
|---|---|---|---|---|
| Agriculture and Irrigation | - | - | 0 | 0.00 |
| Cap Bldg & Inst Dev | - | - | 0 | 0.00 |
| Energy | 30.3% | 25.8% | 6 | 739.8 |
| Finance, Invest., Trade | 18.6% | 18.6% | 1 | 42.4 |
| Health, Ed. & Community | 12.5% | 15.3% | 6 | 348.3 |
| Land | 22.0% | 22.0% | 1 | 18.75 |
| Other | - | - | 0 | 0.00 |
| Transport | 20.3% | 20.3% | 1 | 109.6 |
| WASH | 16.4% | 16.0% | 4 | 672.61 |
| EIF Year | Weighted ERR | Unweighted ERR | Total Projects | Funds Committed (US$M) |
| 2011 | 16.0% | 16.5% | 2 | 275.01 |
| 2012 | 43.2% | 28.0% | 3 | 393.75 |
| 2013 | 15.9% | 15.0% | 2 | 486.3 |
| 2014 | 12.7% | 16.8% | 4 | 116.1 |
| 2015 | 16.5% | 16.8% | 3 | 252.7 |
| 2016 | 15.7% | 20.3% | 5 | 407.6 |
| Size at EIF (US$) | Weighted ERR | Unweighted ERR | Total Projects | Funds Committed (US$M) |
| < 50M | 19.0% | 20.6% | 9 | 204.55 |
| 50M - 100M | 8.0% | 8.0% | 1 | 56.5 |
| > 100M | 22.1% | 19.2% | 9 | 1670.41 |
| Country | Weighted ERR | Unweighted ERR | Total Projects | Funds Committed (US$M) |
| Liberia | 11.0% | 11.0% | 1 | 201.6 |
| Malawi | 48.1% | 48.1% | 1 | 332.2 |
| Zambia | 17.0% | 17.0% | 1 | 354.8 |
| Ghana II | 19.7% | 21.8% | 4 | 261 |
| Indonesia | 13.0% | 13.0% | 1 | 131.5 |
| Georgia II | 12.7% | 16.8% | 5 | 116.1 |
| Cabo Verde II | 16.4% | 18.0% | 2 | 61.55 |
| El Salvador II | 16.5% | 16.8% | 3 | 252.7 |
| Jordan | 16.0% | 16.5% | 2 | 275.01 |
| All Projects | 21.2% | 19.1% | 20 | 1986.46 |
Figure 4
Annex I
2016 Updates to the Closeout ERR Report
The following list details the changes and additions of projects from the dataset used to calculate the statistics found in the 2016 Report on Closeout ERRs. Overall, the total number of closed projects increased from 122 to 137 and twelve new closeout ERRs were added. The total number of open projects decreased from 35 to 25.Updates
- The funds at closeout for Ghana’s Trunk Roads is $29.9 million, not $33.2 million. As a result, Ghana’s closeout ERR is slightly higher.
- One of Morocco’s Fruit Tree Productivity activities was double counted. At EIF, there were two separate original ERRs for the intensification and rehabilitation and the intensification and expansion activities, but at closeout these activities were combined into one ERR.
- Benin’s Access to Markets project was changed from the Finance, Investment, and Trade sector to the Transport sector.
- Three activities were added to Burkina Faso’s Roads Project, all of which do not have closeout ERRs.
- El Salvador’s Community Infrastructure Activity was changed from the Health, Education, and Community sector to the Transport sector. The activity rehabilitated a small number of rural roads and bridges.
- Three projects for Moldova were moved from open to closed. The original ERR for the Access to Agricultural Finance was removed as this ERR was never officially cleared for public release. Additionally, two activities within the Transition to High Value-Added Agriculture Project were added. Costs for both of the activities are included in the ERR for the Rehabilitation of the Centralized Irrigation System Activity.
- Four closeout ERRs were added for Burkina Faso, two for Philippines, four for Senegal, and one for Moldova.
- The original ERR for Liberia’s Energy Project is now included. The original ERR for the Roads project is pending.
- Original ERRs for the three projects in El Salvador II are now included.
- Six projects for Ghana II are included in the open compact data and there are three original ERRs. Funds for the Regulatory Strengthening and Capacity Building Project are accounted for in the ERRs.
Annex II
Table 6 - Stated Reasons for Projects that lack an ERR*
Projects without Closeout ERR: 50 Projects for which no ERR will be Calculated: 44| Country | Project | Activity | Reason |
|---|---|---|---|
| Armenia | Irrigated Agriculture | Irrigation Infrastructure | A revised ERR was calculated in the third year of the compact (2008). At closeout, it was intended that the independent evaluation would include a post-compact ERR. However, this did not occur. |
| Benin | Access to Justice | N/A | Insufficient data on benefits at closeout to calculate an updated ERR. The original model’s assumptions could not be validated. As the model did not provide specific information to estimate benefits, a simple update on costs ay closeout is misleading. |
| Burkina Faso | Agricultural Development | Diversified Agriculture | The compact was approved without an ERR for this activity. At closeout, there was insufficient data and information to produce an ERR. |
| Burkina Faso | Bright 2 Project | N/A | Insufficient data and information to quantify benefits at closeout. A post-compact ERR was estimated by the independent evaluator two years post-compact, with the estimate ranging from 3% to 8%. |
| Burkina Faso | Roads Project | Development of Rural Roads | Insufficient data and information. The compact was approved without an ERR for this activity. At closeout no data was available on traffic volume, road roughness, or other data required for a cost-benefit analysis. |
| Burkina Faso | Roads Project | Capacity Building and Technical Assistance for Road Maintenance | Insufficient data and information. The compact was approved without an ERR for this activity. This was a $5.1 million technical assistance activity with unclear outputs. |
| Burkina Faso | Roads Project | Incentive Matching Fund for Periodic Road Maintenance | Insufficient data and information. The compact was approved without an ERR for this activity. At closeout no data was available on traffic volume, road roughness, or other data required for a cost-benefit analysis. |
| Cabo Verde | Infrastructure | Port of Praia | An independent evaluation is being finalized. A post-compact ERR was expected to be included in the evaluation. However, a post-compact ERR was not included in the 2017 summary report presented to MCC. |
| Cabo Verde | Infrastructure | Roads and Bridges | Insufficient data and information; initial estimates at the time of closeout indicate that the ERR would be below the 10% threshold; however, an ERR is expected to be calculated as part of an ex-post evaluation. |
| Cabo Verde | Watershed Management & Agricultural Support | N/A | Insufficient data and information at closeout. An evaluation team is assessing the use of the irrigation structures five years after the closing of the first compact. |
| Georgia | Enterprise Development | Agribusiness Development | Insufficient data and information. |
| Georgia | Enterprise Development | Georgia Regional Development Fund | Insufficient data and information. |
| Ghana | Agricultural Development | Farmer and Enterprise Training, Credit & Land Tenure | There is little evidence of expansion and intensification of farming and significant restructuring of project components weaken the capacity to calculate a relevant closeout ERR. |
| Ghana | Agricultural Development | Post-Harvest Handling | Insufficient data and information at closeout to calculate an updated ERR and significant redesign of the project make it difficult to calculate a relevant closeout ERR. |
| Ghana | Rural Development | Community Services: Electrification | The project was significantly de-scoped and at closeout the budget was too small to calculate an ERR. |
| Ghana | Rural Development | Financial Services | Insufficient data and information. |
| Ghana | Rural Development | Procurement | Activity was too small to calculate an ERR ($1.5M). |
| Ghana | Transportation | Rehabilitation of Ferry Services | Insufficient data and information to calculate an updated ERR at the time of closeout. After re-evaluation, an ERR is planned. |
| Honduras | Rural Development | N/A | Original ERR was dependent on assumptions that could not be backed up with data. |
| Honduras | Transportation | N/A | Insufficient data and information at closeout to calculate an updated ERR. A post-compact ERR was requested to be incorporated in the impact evaluation. |
| Lesotho | Water Sector | Wetlands and Restoration Conservation | Activity was too small to calculate an ERR ($4M). |
| Lesotho | Private Sector Development | Gender Equality in Economic Rights | Activity was too small to calculate an ERR ($1.04M). |
| Moldova | Transition to High Value-Added Agriculture | Access to Agricultural Finance | Same as reason given for no original ERR; Data needed to calculate an ERR was never made available by the MCA. |
| Namibia | Education | Regional Study and Resource Centers | Same as reason given for no original ERR; There is no evidence that libraries can be linked to student outcomes. |
| Nicaragua | Rural Business Development | N/A | Original ERR was dependent on assumptions that could not be backed up with data. |
| Tanzania | Energy | Kigoma Solar Power | Insufficient data and information |
| Country | Project | Activity | Reason |
|---|---|---|---|
| Burkina Faso | Agricultural Development | Access to Rural Finance | The activity was terminated after MCC determined it had not made sufficient progress in achieving project targets. |
| Cabo Verde | Private Sector Development | Financial Sector Reform | N/A |
| Lesotho | Private Sector Development | Credit Bureau and National ID | The GoL officially requested removing the National ID card aspects of the National ID and Credit Bureau Activity from the compact, to align the activity with its own national e-passport initiative. |
| Lesotho | Private Sector Development | Payment and Settlement Systems | N/A |
| Madagascar | Agricultural Business Investment | N/A | Compact was terminated in 2009 due to the military coup. |
| Madagascar | Financial Sector Reform | N/A | Compact was terminated in 2009 due to the military coup. |
| Madagascar | Land Tenure | N/A | Compact was terminated in 2009 due to the military coup. |
| Mali | Airport Improvement | N/A | Compact was terminated in 2012 due to the military coup. |
| Mali | Alatona Irrigation | N/A | Compact was terminated in 2012 due to the military coup. |
| Mali | Industrial Park | N/A | Compact was terminated in 2012 due to the military coup. |
| Mongolia | Rail | N/A | The Government of Mongolia requested to withdraw the Rail Project from the compact due to an inability to fulfill certain conditions for transparency by the Mongolian rail company’s Russian partners. |
| Mongolia | Roads | Bayanzurkh Bridge | Project was cancelled due to two failed procurements and insufficient time remaining within the compact to initiate a third procurement. |
| Mongolia | Roads | UB-Nalaikh Road | Project was cancelled due to two failed procurements and insufficient time remaining within the compact to initiate a third procurement. |
| Mozambique | Water & Sanitation | Nacala Urban Water Supply | Cancelled due to under-performing contractor. |
| Nicaragua | Property Regularization | N/A | Compact was partially terminated in response to actions by the Nicaraguan government inconsistent with MCC’s eligibility criteria; as a result, funding for this activity stopped. |
| Tanzania | Energy | Malagarasi Hydro (Energy) | Cancelled due to environmental considerations. |
| Country | Project | Activity | Reason | Closeout ERR Status |
|---|---|---|---|---|
| Georgia II | Improving General Education Quality | Education Assessment Support | There is no literature or standard practice on methods to quantify the benefits of testing. | It is too early in the compact to determine. |
| Ghana II | Access Project | N/A | At the time of the investment decision, the project was not fully designed. | It is too early in the compact to determine. |
| Indonesia | Green Prosperity | N/A | As two of the four activities with the project are grant facilities, the size and composition of the individual grants at EIF were unknown and it was not possible to calculate an original ERR. The two remaining activities are complementary to the grant facilities. | As a grant facility, there will likely be no project-level closeout ERR. However, ERRs for individual grants will be calculated. |
| Indonesia | Procurement Modernization | N/A | Insufficient information and data to quantify benefits. | A closeout ERR is planned. |
| Country | Project | Activity | Reason |
|---|---|---|---|
| Ghana II | Regulatory Strengthening and Capacity Building Project | N/A | The costs of this project are included in the total costs of the ECG Project and the Demand Side Management Project. |
| Country | Project | Activity | Reason |
|---|---|---|---|
| Burkina Faso | Agricultural Development | Access to Rural Finance | Compact approved without a cost-benefit analysis for this activity. |
| Burkina Faso | Agricultural Development | Diversified Agriculture | Compact approved without a cost-benefit analysis for this activity. |
| Burkina Faso | Rural Land Governance | Compact approved without a cost-benefit analysis for this activity. In 2011 an ERR was calculated for the phasing decision at 10.8%. | |
| Burkina Faso | BRIGHT 2 Project | Compact approved without a cost-benefit analysis for this activity. This project built on MCC’s 2005 threshold program. | |
| Burkina Faso | Roads Project | Development of Rural Roads | N/A |
| Burkina Faso | Roads Project | Capacity Building and Technical Assistance for Road Maintenance | N/A |
| Burkina Faso | Roads Project | Incentive Matching Fund for Periodic Road Maintenance | N/A |
| Cabo Verde | Private Sector Development | Partnership to Mobilize Investment | Activity was considered too small to construct and ERR ($5M). |
| El Salvador | Productive Development | Finance and Investment Support | This was a grant facility, so there was no data ex-ante to construct an original ERR. |
| Ghana | Rural Development | Financial Services | Activity was considered too small to construct an ERR ($24M). |
| Ghana | Rural Development | Procurement | Activity was considered too small to construct an ERR ($2M). |
| Lesotho | Water Sector | Wetlands and Restoration Conservation | Activity was considered too small to construct an ERR ($5M). |
| Lesotho | Private Sector Development | Payment and Settlement Systems | Activity was considered too small to construct an ERR ($1.7M). |
| Lesotho | Private Sector Development | Gender Equality in Economic Rights | Activity was considered too small to construct an ERR ($1.04M). |
| Madagascar | Agricultural Business Investment | N/A | |
| Mali | Industrial Park | N/A | |
| Moldova | Transition to High Value-Added Agriculture | Access to Agricultural Finance | Data needed to calculate an ERR was never made available by the MCA. |
| Namibia | Education | Regional Study and Resource Centers | There is no evidence that libraries can be linked to student outcomes. |
| Country | Project | Activity | Reason |
|---|---|---|---|
| Mongolia | Energy & Environment | MC Energy Investment Fund | This project is a result of funds being reallocated from the cancelled Rail project. |
| Morocco | Artisan and Fez Medina | Literacy & Vocational Training | This project is the result of a reallocation of funds away from other projects. |
| Morocco | Fruit Tree Productivity | Catalyst Fund | The activity was added after a re-scoping decision. The funds for the activity are included in the other Fruit Tree Productivity activity closeout ERRs. |
| Namibia | Education | Vocational and Skills Training, VGTF | The activity was added to complement the National Training Fund. |
| Tanzania | Energy | Kigoma Solar Power | This was added at the re-scoping decision following the cancellation of the Malagarasi Hydro Activity. |
Table 7
| Country | Cancelled Projects | Projects with No ERR Planned | Pending ERRs | As a % of Total Projects | As a % of Total Funds Disbursed |
|---|---|---|---|---|---|
| Armenia | - | 1 | - | 33.3% | 76.7% |
| Benin | - | 1 | - | 25% | 6.5% |
| Burkina Faso | 1 | 5 | 1 | 58.3% | 35.2% |
| Cabo Verde | 1 | 4 | - | 100% | 100% |
| El Salvador | - | - | - | 0% | 0% |
| Georgia | - | 2 | - | 40% | 14.4% |
| Ghana | - | 6 | - | 46.2% | 41.5% |
| Honduras | - | 2 | - | 100% | 100% |
| Lesotho | 2 | 2 | - | 40% | 2.3% |
| Madagascar | 3 | - | - | 100% | 100% |
| Mali | 3 | - | - | 100% | 100% |
| Moldova | - | 1 | 1 | 40%** | 55%** |
| Mongolia | 3 | - | - | 30% | 0.20% |
| Morocco | - | - | - | 0% | 0% |
| Mozambique | 1 | - | - | 11.1% | 16% |
| Namibia | - | 1 | - | 8.3% | 8.2% |
| Nicaragua | 1 | 1 | - | 66.7% | 40.8% |
| Philippines | - | - | 1 | 33%*** | 55.6%*** |
| Senegal | - | - | - | 0% | 0% |
| Tanzania | 1 | 1 | 4 | 66.7%**** | 85.9%**** |
| Vanuatu | - | - | - | 0% | 0% |
Table 8A
| Country | Project/Activity | Reduced Scope | Increased Costs | Reduced Beneficiaries | Project Design | Poor Data at EIF | Low Carbon Credit Prices | Government/Third-Party Progress |
|---|---|---|---|---|---|---|---|---|
| Benin | Access to Land | X | X | |||||
| Burkina Faso** | Rural Land Governance | X | ||||||
| Burkina Faso | Roads | |||||||
| Burkina Faso | Di Irrigation | |||||||
| El Salvador* | WASH | X | X | X | ||||
| Georgia | Regional Infrastructure Dev. | X | X | |||||
| Georgia* | Energy Rehabilitation: North/South Gas Pipeline | X | X | |||||
| Ghana | Irrigation | X | ||||||
| Ghana | Trunk Roads | X | X | |||||
| Lesotho | Health Sector | X | X | X | ||||
| Lesotho | Rural Water Supply | X | ||||||
| Moldova | Rehabilitation of CIS | X | X | |||||
| Mongolia | Choir-Sainshand Road | X | ||||||
| Mongolia | Wind Park | X | X | |||||
| Morocco* | Enterprise Support | X | X | X | ||||
| Morocco* | Artisan Production | X | ||||||
| Mozambique | Quelimane WASH | X | X | |||||
| Mozambique | Roads | X | X | |||||
| Mozambique | Mocuba WASH | X | ||||||
| Namibia | National Training Fund | X | ||||||
| Namibia | Improving the Quality of General Education | X | ||||||
| Namibia | Land Access and Management Activity and Livestock Activity | X | ||||||
| Nicaragua* | Transportation | X | ||||||
| Senegal | Podor Irrigation | |||||||
| Senegal | RN2 | X | ||||||
| Senegal | RN6 | X | ||||||
| Tanzania | Morogoro Water Supply | X | ||||||
| Tanzania | Mafia Airport | X |
Table 8B
| Benin | Access to Land | While the target number of land titles were surveyed, only a quarter of them were expected to be issued, greatly reducing project scope. The project also experienced moderate cost increases. |
| Burkina Faso** | Rural Land Governance | |
| Burkina Faso | Roads | Approved at the time of the investment decision with ERRs below the hurdle rate |
| Burkina Faso | Di Irrigation | Approved at the time of the investment decision with ERRs below the hurdle rate |
| El Salvador* | WASH | The original model assumed water-borne disease would be completely eliminated by provision of clean water, which was not the case. |
| Georgia | Regional Infrastructure Dev. | Inadequate Water Supply |
| Georgia* | Energy Rehabilitation: North/South Gas Pipeline | Gas losses, the main anticipated benefit, were substantially reduced before the rehabilitation work began due to maintenance that was completed by the government after original economic analysis, but prior to the start of MCC-funded projects. |
| Ghana | Irrigation | Initial assessments indicate that two components of this project are not adequately maintained and that the third has deteriorated considerably. Given the current state of the infrastructure, it is unlikely that the project will have a positive return. |
| Ghana | Trunk Roads | |
| Lesotho | Health Sector | The ERR reduced after factoring in increased costs, borne by the Government of Lesotho, decreased benefits due to incorrect baseline assumptions, as well as adding other donor costs for the benefit streams not attributable to MCC funding alone. |
| Lesotho | Rural Water Supply | Limited economic potential in target community |
| Moldova | Rehabilitation of CIS | The closeout ERR is significantly lower than the original estimates due to higher construction costs than expected, higher costs due to an unexpected exchange rate depreciation, and less optimistic assumptions about transition rates of land to HVA (due in part to limited access to redit and limited access to foreign markets). |
| Mongolia | Choir-Sainshand Road | Lower-than-expected traffic |
| Mongolia | Wind Park | Government inaction; construction delays |
| Morocco* | Enterprise Support | Costs were greater than originally expected, which was the primary reason the decision was made not to scale up. Additionally impacts on value-added growth and business survival rates are difficult to detect and measure after a single year. |
| Morocco* | Artisan Production | Artisans were both reluctant to adopt the new kilns or unable to obtain financing for their 20% contribution to the purchase of the kiln. As a result, at the time of the evaluation, only two gas kilns were operational compared to the target of 144 kilns. |
| Mozambique | Quelimane WASH | The original model was estimated based on old data and the project was put on hold. Once reliable data was obtained, project was restarted with leftover funds, but the scope was greatly reduced. |
| Mozambique | Roads | The project was reduced from four road segments totaling 491 km to two segments totaling 253 km. |
| Mozambique | Mocuba WASH | The project was significantly de-scoped in order to focus on an emergency activity targeted at a hospital. |
| Namibia | National Training Fund | There were delays in launching the fund. |
| Namibia | Improving the Quality of General Education | |
| Namibia | Land Access and Management Activity and Livestock Activity | |
| Nicaragua* | Transportation | The project was partially terminated and funding ceased to activities not already contracted. Of those activities that were funded, costs increased. |
| Senegal | Podor Irrigation | Exogenous factors (i.e. weather) delayed harvest season, subsequently delaying realization of benefits |
| Senegal | RN2 | Incorrect baseline assumptions of IRI; re-measured mid-compact by MCC independent engineer |
| Senegal | RN6 | Incorrect baseline assumptions of traffic counts |
| Tanzania | Morogoro Water Supply | Due to construction delays, the treatment plants were completed after the compact by the Tanzanians, impacting the percentage of O&M costs recovered and the volume of water produced by the plants at compact end. |
| Tanzania | Mafia Airport |
In Tables 8A and 8B above, twelve evaluations are underway, and seven evaluations are planned. Three projects currently do not have any evaluations planned.
Annex III
Excerpts from MCC’s Summary of Findings
I. Measuring Results of the Morocco Enterprise Support Project
Economic Rate of Return (ERR) Estimates and Decision Not to Scale-up the Enterprise Support PilotAs there was insufficient empirical evidence on the extent to which post-creation business training would increase revenues, productivity and survival rates of new enterprises, the Enterprise Support Project was structured as a two-year pilot program subject to a rigorous impact evaluation using a randomized design. Scale-up of the project was made contingent upon the results of the pilot program producing an estimated ERR above 10 percent for the scale-up project. The ERR prior to the implementation of the pilot program was estimated at 13.5 percent. Upon results one year after implementation, the overall ERR including scale-up phase projections was estimated at 1.21 percent while the ERR of the scale-up phase alone was estimated at 7.21 percent.[[Given the nature of post-creation business support, impacts on value-added growth and business survival rates are difficult to detect/measure after a single year. However, costs were greater than originally expected, which was the primary reason the decision was made not to scale up. While the results of the impact evaluation as summarized in this report further supports MCC’s final decision not to scale-up (as well as the fact that costs at closeout were even higher than estimated at the time of the scale up decision), previous interventions in post-creation business support would suggest that some benefits from support may accrue several years after implementation, longer than the total of two years given to the pilot program to be evaluated. Unfortunately, due to concerns about attrition rates (e.g. difficulty in following up with businesses) and possible provision of support services to control group businesses, further impact evaluation of this Project is not feasible.]] MCC decided not to proceed with scale-up of the Enterprise Support Project as the revised ERR estimate did not pass the hurdle-rate of 10 percent and was not estimated to generate sufficient benefits to justify the costs of the project.
ERR at Closeout (Project-wide, not exclusive to Support to Small and Medium Enterprises (SMEs) Activity)
At closeout, the results of the impact evaluation became available and it was possible to revisit some of the benefit assumptions that could not be updated at the time the scale up decision was made. The impact evaluation found SME failure rates were reduced by 30%, rather than the 20% reduction assumed by the original ERR. However, this has essentially no significant impact on the ERR (an interesting problem of itself that we have yet to resolve). The only other significant impact the impact evaluation found was an improvement in value added for the AGR-INDH sub-activity of 30%, vs. 5% as was assumed in the original ERR. Making this change to the parameter increased the ERR for the AGR-INDH sub-activity from -3.5% to 4.4%, a significant improvement but still not higher than the 10% threshold. The overall closeout ERR was -0.9%.”
II. Measuring Results of the Artisan and Fez Medina Project Artisan Production and Promotion and Fez Medina Activities
“Even though potters were convinced of the relevance of replacing traditional kilns with less polluting gas kilns, this activity faced serious reluctance for many reasons: (i) the 20% contribution of potters to the imported gas kiln was beyond the financial capacity of potters; (ii) potters were reluctant to contract loans from banks for various reasons, among which the reimbursement conditions and (iii) Banks were not willing to finance the majority of potters operating informal, unregistered businesses; (iii) the maintenance cost of imported kilns were perceived as a long-term burden. The project design did not take into account the potential of the local gas kiln manufacturers and favored imported kilns from the start. The project should have considered developing a local kiln based on local usage and adapted to the needs and capacities of Moroccan potters.”III. Measuring Results of the Nicaragua Transportation Project
“The calculation of the ex-post ERR utilized the Roads Economic Decision model (RED) developed by the World Bank. This approach captures the project’s flow of net benefits to road users defined as benefits minus costs which are discounted across time to yield a single value, using pre- and post-construction traffic counts and estimates of future traffic counts. The ex-post ERR estimated by the independent evaluator showed that the roads failed to meet the 10 percent hurdle rate. The average for the project, as a whole, was estimated to be 2.1 percent. Actual capital costs were, on average, 2.2 times greater than those estimated in the feasibility studies.”IV. MCG Gas Pipeline Rehabilitation: Project Summary and ERR Model Revised
“Before the rehabilitation work was undertaken, the ERR was estimated to be 11.7% based on a forecast of a reduction in the amount of gas leakage from the pipeline and a assumed corresponding sale of carbon credits due to the reduction in methane emissions to the atmosphere. However, the high gas leakage rate was reduced before the rehabilitation work was undertaken, and it was not possible to sell carbon credits.”V. Measuring Results of the El Salvador Water and Sanitation Sub-Activity
“High costs per beneficiary can kill even the best projects. The economic rate of return for the El Salvador Water and Sanitation Sub-Activity was estimated as 3.6% at the close of the Compact, due largely to low population densities and high costs. While the execution of water, sanitation, and hygiene projects can be improved using the above lessons, it must be kept in mind that if we target expensive or low density regions, the benefits are unlikely to justify the project.”“‘Higher than expected costs per borehole reduced the number built during the compact, while low population densities reduced the number of households using each borehole. Combined, these led to a measurable drop in beneficiaries.”
Bibliography
Cronshaw, Mark, and Gustavson Associates. MCG Gas Pipeline Rehabilitation: Project Summary and ERR Model Revised. Rep. 23 June 2010. Web.Millennium Challenge Corporation “Measuring Results of the Morocco Enterprise Support Project”, MCC, Summary of Findings, 2015
Millennium Challenge Corporation “Measuring Results of the Artisan and Fez Medina Project Artisan Production and Promotion and Fez Medina Activities”, MCC, Summary of Findings, 2015
Millennium Challenge Corporation “Measuring Results of the Nicaragua Transportation Project”, MCC, Summary of Findings, 2014.
Millennium Challenge Corporation “Measuring Results of the El Salvador Water and Sanitation Sub-Activity”, MCC, Summary of Findings, 2017.
Millennium Challenge Corporation. "Economic Rates of Return: Country Spreadsheets." Millennium Challenge Corporation. Web.
Ospina, Sandra, Aaron Mitchell, and Aaron Szott. 2014 Report on Closeout ERRs. March 2015.
Ospina, Sandra and Marissa Block. 2015 Report on Closeout ERRs. June 2016.