2015 Report on Closeout ERRs

The main purpose of this report is to concisely document facts on ERRs that have been updated upon compact closeouts as of end-2015. The report‘s first three sections describe coverage, characteristics of all closeout ERRs and characteristics of closeout ERRs that fall below MCC’s ten percent threshold. Section IV presents original ERRs for open compacts using data available as of end-2015. The data used to estimate these closeout ERRs reflects final cost data as available at closeout. As the benefits stream for most projects may only start after compact closeout, these closeout ERRs remain an ex-ante estimate of economic return.

Authors

Sandra Ospina (Economic Analysis, Senior Director), Ospinasp@mcc.gov

Marissa Block (Economic Analysis, Research Assistant)

Table 1

Overall Description of Data - Closed Compacts
	Number	Funds Committed	Funds Disbursed
Total Closed Projects	122	5,482.10	5,283.80
Projects with Closeout ERR	76	3,059.28	3,232.71
Projects without Closeout ERR	46	2,422.81	2,051.09
Closeout ERR Pending	9	702.77	634.10
Projects for which no ERR will be calculated	37	1,720.04	1416.99
Completed, but no ERR planned	22	865.02	825.01
Cancelled Projects	15	855.03	591.98

Characteristics

Closeout ERRs are on average above MCC’s 10 percent threshold. ERRs are estimated at the lowest possible level of disaggregation. When reporting summaries for compact statistics—as we do here—ERRs are sometimes aggregated by project or by activity using funds as weights. The majority of reported ERRs, 59 out of 76 closeout ERRs, are calculated at the activity level while the remaining 16 are calculated at the project-level. Table 2 shows weighted and un-weighted average ERRs for different categories. The table shows no significant differences in these averages, with most ERRs fluctuating between 12 and 17 percent. The exceptions are in the compact category with Nicaragua’s at 2.10 percent—the lowest overall—Burkina Faso’s at 5.8 percent, and Lesotho’s at 8 percent. Morocco, El Salvador, Namibia, and Ghana’s weighted closeout ERR stand out at around 20 percent. The average ERR for the 76 projects weighted by disbursed funds at compact closeout is 15.2 percent. While it is not explicitly shown in Table 2, it should be noted that the total number of projects in Table 2 excludes projects for which no ERR will be calculated and projects with pending ERRs. Burkina Faso and Tanzania have five and four projects respectively with pending ERRs, and therefore the compact ERR will ultimately change. Additionally, Cabo Verde and Honduras have three and two projects respectively, but both consist of projects for which no ERR will be calculated, and are thus excluded from the table. See Table 7 and 8 in Annex II.

Table 2

All Projects with Closeout Data
Sector	Weighted Closeout ERR	Unweighted Closeout ERR	Total Projects	Funds Disbursed (US$M)
Agriculture and Irrigation	18.7%	18.0%	18	676.80
Energy	15.1%	13.7%	4	101.63
Finance, Invest., Trade	13.0%	11.3%	6	312.73
Health, Ed. & Community	15.7%	21.6%	14	432.64
Land	12.1%	13.8%	6	190.90
Other*	14.2%	15.3%	3	66.10
Transport	15.8%	12.5%	11	1022.84
WASH	11.1%	12.7%	14	429.08
Closeout Year	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
2011	11.5%	9.9%	10	748.22
2012	20.1%	14.5%	15	668.82
2013	14.8%	15.9%	39	1537.57
2014	16.0%	21.2%	12	278.10
Size at Closeout (US$)	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
< 50M	15.7%	15.9%	57	1255.01
50M - 100M	14.0%	14.5%	12	757.45
> 100M	15.6%	15.5%	7	1220.25
Country	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
Armenia	12.3%	14.6%	2	36.90
Benin	12.8%	11.2%	3	278.13
Burkina Faso	5.8%	5.8%	1	66.20
El Salvador	20.3%	15.1%	8	417.10
Georgia	12.2%	8.0%	3	309.70
Ghana	19.8%	13.9%	7	251.72
Lesotho	8.0%	9.8%	6	309.95
Mongolia	12.4%	13.0%	7	223.42
Morocco	19.5%	19.2%	15	616.20
Mozambique	15.2%	20.7%	8	310.80
Namibia	19.2%	22.5%	11	211.90
Nicaragua	2.1%	2.1%	1	57.80
Tanzania	10.0%	6.8%	3	77.20
Vanuatu	10.3%	10.3%	1	65.69
All Projects	15.2%	15.7%	76	3232.71

*This includes Namibia's tourism projects.

Closeout ERRs have final cost data but do not include final benefits. Final benefits will in most cases take a number of years to be realized although it is possible at compact closure to have an updated estimation of the benefits. Notwithstanding that closeout ERRs are still not final ERRs, it is useful to compare closeout ERRs with their original estimates. As Chart 1a shows, the norm is that original project ERRs are above their corresponding closeout ERRs. Of the 70 projects for which there are original and closeout ERRs, the weighted average for original ERRs is 19.6 percent compared to the 15.1 percent for corresponding closeout ERRs.

Chart 1a

Chart 1b shows the difference in percentage points between closeout ERRs and original ERRs. The chart shows that for most projects the ERRs decreased, sometime exceeding a fall of 20 percentage points. The right hand tail in Chart 1b shows the 14 projects for which the closeout ERR increased relative to their original ERR (20% of all projects with original and closeout ERRs). The average of the absolute value of the gap (closeout ERR - original ERR) is 6.3 percentage points.

Chart 1b

Chart 1a

Chart 1b

More than one fourth of closeout projects have ERRs below MCC’s threshold and more than three-fourths of projects exhibit ERRs that decrease from EIF to close. Chart 1a reveals substantial variation across closeout ERRs. They range from -8 percent to 109 percent. Chart 1a identifies the 21 projects with an ERR below the threshold (nearly 30 percent of all projects) in red. Differences in ERRs range from a 67 percentage point decrease to a 29 percentage point increase as seen in Chart 1b.

Projects with ERRs below the threshold:

Projects with ERRs below the threshold are mostly small in size, concentrated in 2013, more prevalent in water and sanitation and transport projects, and evenly distributed across compacts. Table 3 disaggregates projects with closeout ERRs that fall below 10 percent in different categories. The average ERR for the 22 projects is 4.5 percent corresponding to approx. US$904M or 28 percent of disbursed funds. The table shows that more than two thirds (68 percent) of the projects are small projects (< US$50M). However, only a quarter of all small projects fall below the threshold, whereas 42% of all projects in the US$50-$100M category fall below the threshold. Roughly 30 percent of the disbursed funds correspond to two high value projects that sum up to US$271M. The two projects are Mozambique’s Rehabilitation/Construction of Roads Project (US$127.7M) and Lesotho’s Health Sector project (US$128.5M).

Almost all sectors have at least one project with an ERR below the threshold. The water and sanitation sector has the most number of projects amounting to 40 percent of the value of the sector’s portfolio.

The transport sector’s five projects amount to the highest value of disbursed funds, though this amounts to 30 percent of the value of the sector’s portfolio. Land projects that fall below the threshold account for 55 percent of the value of the sector’s portfolio, the highest of all sectors. Two of the four energy projects fall below the threshold and account for 36 percent of the value of energy’s portfolio. The three health, education, and community projects account for 50 percent of the US$432.6M for the sector’s total, and one of these projects, the Lesotho project on health, accounts for 33 percent of the sector’s total. On the other hand, less than 20 percent of agriculture and irrigation projects fall below the threshold. Their value corresponds to seven percent of the value of the sector’s portfolio.

Table 3

Closed Projects with Closeout ERR below 10% Threshold
Sector	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
Agriculture and Irrigation	2.5%	1.7%	3	47.80
Energy	7.7%	8.2%	2	36.90
Finance, Invest., Trade	-0.9%	-0.9%	1	15.10
Health, Ed. & Community	6.5%	6.4%	3	218.75
Land	5.8%	5.8%	2	105.40
Other	N/A	N/A	0	0.00
Transport	5.8%	4.0%	5	307.61
WASH	-0.8%	-0.3%	6	172.43
Closeout Year	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
2011	3.6%	4.1%	4	183.30
2012	0.9%	1.2%	3	77.01
2013	4.5%	2.2%	11	482.18
2014	7.3%	6.5%	4	161.50
Size at Closeout (US$)	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
< 50M	1.5%	2.0%	15	309.34
50M - 100M	5.9%	5.4%	5	323.30
> 100M	6.2%	6.3%	2	271.35
Country	Weighted ERR	Unweighted ERR	Total Projects	Funds Disbursed (US$M)
Armenia	N/A	N/A	0	0.00
Benin	5.8%	5.8%	1	39.20
Burkina Faso	5.8%	5.8%	1	66.20
El Salvador	3.6%	3.6%	1	19.80
Georgia	3.6%	4.2%	2	86.30
Ghana	0.0%	0.0%	2	57.21
Lesotho	2.3%	-1.4%	2	183.78
Mongolia	9.4%	9.0%	2	76.50
Morocco	-0.9%	-1.0%	2	18.70
Mozambique	5.4%	1.7%	3	171.70
Namibia	8.3%	6.7%	3	95.30
Nicaragua	2.1%	2.1%	1	57.80
Tanzania	2.9%	2.7%	2	31.50
Vanuatu	N/A	N/A	0	0.00
All Projects	4.5%	3.2%	22	903.99

Increased costs and reduced scope are the most important reasons behind closeout ERRs falling below the threshold. Table 4 identifies the reasons for reduced ERRs for every project below the threshold. The single most listed category is increased costs, followed by reduced scope. Other reasons include poor data at EIF and poor project design. The reasons listed are mainly provided by the compact-assigned lead economists. The reasons for four projects are taken from the “Summary of Findings” of their corresponding evaluation reports or from the reports themselves. Annex III provides excerpts of the available “Summary of Findings” for projects with evaluations. For the remaining projects in Table 4, seven evaluations are underway, and six evaluations are planned. Three projects currently do not have any evaluations planned.

Table 4

Major Drivers of Reduced ERRs at Closeout
Country	Project/Activity	Reduced Scope	Increased Costs	Reduced Beneficiaries	Poor Data at EIF	Low Carbon Credit Prices	Government/ Third-Party Progress	Project Design	Additional Information
Benin	Access to Land	X	X						While the target number of land titles were surveyed, only a quarter of them were expected to be issued, greatly reducing project scope. The project also experienced moderate cost increases.
Burkina Faso**	Rural Land Governance		X
El Salvador	WASH	X		X	X				The original model assumed water-borne disease would be completely eliminated by provision of clean water, which was not the case.
Georgia	Regional Infrastructure Dev.			X				X	Inadequate Water Supply
Georgia*	Energy Rehabilitation: North/South Gas Pipeline					X	X		Gas losses, the main anticipated benefit, were substantially reduced before the rehabilitation work began due to maintenance that was completed by the government after original economic analysis, but prior to the start of MCC-funded projects.
Ghana	Irrigation						X		Initial assessments indicate that two components of this project are not adequately maintained and that the third has deteriorated considerably. Given the current state of the infrastructure, it is unlikely that the project will have a positive return.
Ghana	Trunk Roads	X	X
Lesotho	Health Sector		X		X		X		The ERR reduced after factoring in increased costs, borne by the Government of Lesotho, decreased benefits due to incorrect baseline assumptions, as well as adding other donor costs for the benefit streams not attributable to MCC funding alone.
Lesotho	Rural Water Supply							X	Limited economic potential in target community
Mongolia	Choir-Sainshand Road		X						Lower-than-expected traffic
Mongolia	Wind Park					X			Government inaction; construction delays
Morocco*	Enterprise Support		X		X			X	Costs were greater than originally expected, which was the primary reason the decision was made not to scale up. Additionally impacts on value-added growth and business survival rates are difficult to detect and measure after a single year.
Morocco*	Artisan Production							X	Artisans were both reluctant to adopt the new kilns or unable to obtain financing for their 20% contribution to the purchase of the kiln. As a result, at the time of the evaluation, only two gas kilns were operational compared to the target of 144 kilns.
Mozambique	Quelimane WASH		X		X				The original model was estimated based on old data and the project was put on hold. Once reliable data was obtained, project was restarted with leftover funds, but the scope was greatly reduced.
Mozambique	Roads	X	X						The project was reduced from four road segments totaling 491 km to two segments totaling 253 km.
Mozambique	Mocuba WASH	X							The project was significantly de-scoped in order to focus on an emergency activity targeted at a hospital.
Namibia	National Training Fund							X	There were delays in launching the fund.
Namibia	Improving the Quality of General Education							X
Namibia	Land Access and Management Activity and Livestock Activity		X
Nicaragua*	Transportation	X							The project was partially terminated and funding ceased to activities not already contracted. Of those activities that were funded, costs increased.
Tanzania	Morogoro Water Supply		X						Due to construction delays, the treatment plants were completed after the compact by the Tanzanians, impacting the percentage of O&M costs recovered and the volume of water produced by the plants at compact end.
Tanzania	Mafia Airport		X

*Projects with completed evaluations. **No original ERR.

Original ERRs for Open Compacts

Original ERRs have been estimated for the majority of the open projects. Table 5 shows that 63 percent of the 35 open projects have an original ERR. They correspond to 73 percent of committed funds. The majority of the funds for which ERRs are not estimated correspond to the Indonesia compact (US$M 383).[[See Annex II for further explanation as to why an ERR was not originally calculated.]] The nine pending ERRs correspond to El Salvador II and to Liberia.

Table 5

Overall Description of Data - Open Projects
	Number	Funds Committed
Total Open Projects*	35	3,195.68
Projects with Original ERR	22	2331.28
Projects without Original ERR	13	864.40
Original ERR pending	10	475.40
Projects for which no ERR Will Be Calculated	3	389.00

* Total open projects is defined as all projects of compacts which have entered into force. Therefore, Liberia and El Salvador II are included, but Ghana II and Benin II are NOT included.

Comparison between currently open and closed compacts reveals a downward trend in original ERRs per compact. Chart 2 shows that for open compacts, except Malawi, the per-compact average original ERRs are lower than 20 percent. For half of 18 closed compacts, the per-compact average original ERR exceeds 20 percent. Table 6 shows weighted ERRs of different categories for open projects, including the Moldova and the Senegal compact for which the closeout process is underway. The weighted average ERR for all 22 projects is 20 percent. If Malawi’s original ERR is replaced with 18.7 percent—the ERR revised one year later—the weighted average ERR falls to 16 percent.

Chart 2

Table 6

All Open Projects with Original ERRs
Sector	Weighted ERR	Unweighted ERR	Total Projects	Funds Committed (US$M)
Agriculture and Irrigation	14.9%	12.1%	4	232.39
Capacity Building	40.3%	40.3%	1	54.3
Energy	48.1%	48.1%	1	332.2
Health, Ed. & Community	12.8%	16.0%	6	245.6
Land	22.0%	22.0%	1	18.75
Other*	12.6%	12.6%	1	120
Transport	14.2%	14.2%	4	653.43
WASH	16.4%	16.0%	4	672.61
EIF Year	Weighted ERR	Unweighted ERR	Total Projects	Funds Committed (US$M)
2010	14.7%	13.1%	7	623.39
2011	16.4%	19.9%	5	711.74
2012	43.2%	28.0%	3	393.75
2013	15.9%	15.0%	2	486.3
2014	12.5%	16.8%	5	114.1
Size at EIF (US$)	Weighted ERR	Unweighted ERR	Total Projects	Funds Committed (US$M)
< 50M	7.8%	7.9%	7	372.34
50M - 100M	16.2%	16.9%	5	316.3
> 100M	20.9%	18.6%	10	1934.14
Country	Weighted ERR	Unweighted ERR	Total Projects	Funds Committed (US$M)
Cabo Verde II	16.4%	18.0%	2	61.55
Georgia II	12.7%	16.8%	5	116.1
Indonesia	13.0%	13.0%	1	131.5
Jordan	16.0%	16.5%	2	275.01
Malawi	48.1%	48.1%	1	332.2
Moldova**	18.2%	15.6%	3	220.86
Philippines	16.7%	22.2%	3	436.73
Senegal**	12.8%	11.2%	4	402.53
Zambia	17.0%	17.0%	1	354.8
All Projects	20.2%	17.7%	22	2331.28

*Other refers to Philippines KALAHI-CIDSS **Moldova and Senegal Compacts have closed, but at the time of this report they were still undergoing the closeout process.

Two compacts—Senegal and Georgia II—have projects with original ERRs below the threshold. Chart 3 plots committed project funds and their corresponding original ERRs for open compacts, including Moldova and Senegal. Two projects, Senegal’s Podor Irrigation and Georgia II’s Education Infrastructure corresponding to 9.5 percent of all open projects or 2.7 percent of funds committed for all open projects, have original ERRs below 10 percent.

Chart 3

Annex I

2015 Updates to the Closeout ERR Report

The following list details the changes and additions of projects from the dataset used to calculate the statistics found in the 2015 Report on Closeout ERRs. Overall, the total number of closed projects increased from 94 to 122 and eighteen new closeout ERRs were added (11 for Namibia, 3 for Tanzania, 2 for Lesotho, and one each for Burkina Faso, and Nicaragua). The total number of open projects decreased from 45 to 35.

Updates

The status of the Closeout ERR for Ghana’s Post Harvest Handling Activity is that no ERR will be calculated, not pending.
The closeout ERR for Ghana’s Irrigation Activity changed from 13% to 0%. The operation of the schemes is not active.
The original ERR for Ghana’s Irrigation Activity is 15%. It was previously listed as 22%, the activity-level ERR consisting of Land Tenure, Farmer and Enterprise Training, Irrigation, and Credit Sub-Activities.
The original ERR for Zambia is 17%, not 13.7%
The following changes were made to Lesotho’s Closeout ERRs:
- Rural Water Supply & Sanitation: -7.9%
- Health Sector: 5.2%
- Wetlands and Restoration Conservation: None planned
Eleven Namibia projects were moved from open to closed and two additional projects are included. Two out of thirteen projects will not calculate an ERR.
Nine Burkina Faso projects were moved from open to closed. Two projects will not calculate an ERR, one project was cancelled, and one project has a closeout ERR. At EIF, the original ERRs of the roads project were disaggregated by road segments for a total of seven ERRs. At closeout, the ERRs will be calculated at a more aggregated level on primary roads, for a total of three ERRs. Including roads, there are five pending ERRs.

Additions

Mongolia’s Baynzurkh Bridge and UB-Nalaikh Road Activities were added to the list of projects. These two activities have an original ERR, but were ultimately cancelled and thus have no closeout ERR.
The closeout ERR for Nicaragua’s Transportation project was calculated by the evaluator and is now included.
The closeout ERRs for Lesotho’s Private Sector Development Project were added at the activity-level for the Civil Legal Reform Activity and Land Administration Reform Activity. The Credit Bureau and National ID activity and Payment and Settlement Systems activity are included, but both were cancelled. The Gender Equality in Economic Rights activity is now included, but no ERR is planned.
Tanzania’s Water & Sanitation Project was split into two activities. The original and closeout ERRs for the Morogoro Water Supply Activity and Lower Ruvu Plant Expansion Activity are now included.
The closeout ERR for Tanzania’s Mafia Airport is now included.
Four activities for El Salvador II are included in the open compact data. The compact has entered into force. The ERR status is pending.
Six activities for Liberia are included in the open compact data. The compact has entered into force. The ERR status is pending.

Annex II

Table 7 - Explanation for Projects without an ERR

Projects without Closeout ERR: 46 Projects for which no ERR will be Calculated: 37

Completed, but no ERR planned: 22
Country	Project	Activity	Reason
Armenia	Irrigated Agriculture	Irrigation Infrastructure	A revised ERR was calculated in the third year of the compact (2008). At closeout, it was intended that the evaluation would calculate a post-compact ERR. However, this did not occur.
Benin	Access to Justice		Insufficient data on benefits at closeout to calculate an updated ERR.
Burkina Faso	Agricultural Development	Diversified Agriculture	The compact was approved without an ERR for this activity. At closeout, there was insufficient data and information to produce an ERR.
Burkina Faso	Bright 2 Project		Insufficient data and information to quantify benefits.
Cabo Verde	Infrastructure	Port of Praia	An ex-post ERR will be calculated as part of the evaluation.
Cabo Verde	Infrastructure	Roads and Bridges	Insufficient data and information; initial estimates at the time of closeout indicate that the ERR would be below the 10% threshold; however, an ERR is expected to be calculated as part of an ex-post evaluation.
Cabo Verde	Watershed Management & Agricultural Support		Insufficient data and information
Georgia	Enterprise Development	Agribusiness Development	Insufficient data and information
Georgia	Enterprise Development	Georgia Regional Development Fund	Insufficient data and information
Ghana	Agricultural Development	Farmer and Enterprise Training, Credit & Land Tenure	There is little evidence of expansion and intensification of farming and significant restructuring of project components weaken the capacity to calculate a relevant closeout ERR.
Ghana	Agricultural Development	Post-Harvest Handling	Insufficient data and information at closeout to calculate an updated ERR and significant redesign of the project make it difficult to calculate a relevant closeout ERR.
Ghana	Rural Development	Community Services: Electrification	The project was significantly de-scoped and at closeout the budget was too small to calculate an ERR.
Ghana	Rural Development	Financial Services	Insufficient data and information
Ghana	Rural Development	Procurement	Activity was too small to calculate an ERR ($1.5M).
Ghana	Transportation	Ferry	Insufficient data and information to calculate an updated ERR.
Honduras	Rural Development		Decision to not conduct an ERR was made prior to the current economist.
Honduras	Transportation		Insufficient data and information at closeout to calculate an updated ERR.
Lesotho	Water Sector	Wetlands and Restoration Conservation	Activity was too small to calculate an ERR ($4M).
Lesotho	Private Sector Development	Gender Equality in Economic Rights	Activity was too small to calculate an ERR ($1.04M).
Namibia	Education	Regional Study and Resource Centers	There is no evidence that libraries can be linked to student outcomes.
Nicaragua	Rural Business Development		Decision to not conduct an ERR was made prior to the current economist
Tanzania	Energy	Kigoma Solar Power	Insufficient data and information

Cancelled Projects: 15
Country	Project	Activity	Reason
Burkina Faso	Agricultural Development	Access to Rural Finance	The activity was terminated after MCC determined it had not made sufficient progress in achieving project targets.
Cabo Verde	Private Sector Development	Financial Sector Reform	N/A
Lesotho	Private Sector Development	Credit Bureau and National ID	The GoL officially requested removing the National ID card aspects of the National ID and Credit Bureau Activity from the compact, to align the activity with its own national e-passport initiative.
Lesotho	Private Sector Development	Payment and Settlement Systems	N/A
Madagascar	Agricultural Business Investment		Compact was terminated in 2009 due to the military coup.
Madagascar	Financial Services, Land Tenure, & Agri-business Investment		Compact was terminated in 2009 due to the military coup.
Mali	Airport Improvement		Compact was terminated in 2012 due to the military coup.
Mali	Alatona Irrigation		Compact was terminated in 2012 due to the military coup.
Mali	Industrial Park		Compact was terminated in 2012 due to the military coup.
Mongolia	Rail		The Government of Mongolia requested to withdraw the Rail Project from the compact due to an inability to fulfill certain conditions for transparency by the Mongolian rail company’s Russian partners.
Mongolia	Roads	Bayanzurkh Bridge	Project was cancelled due to two failed procurements and insufficient time remaining within the compact to initiate a third procurement.
Mongolia	Roads	UB-Nalaikh Road	Project was cancelled due to two failed procurements and insufficient time remaining within the compact to initiate a third procurement.
Mozambique	Water & Sanitation	Nacala Urban Water Supply	Cancelled due to under-performing contractor
Nicaragua	Property Regularization		Compact was partially terminated in response to actions by the Nicaraguan government inconsistent with MCC’s eligibility criteria; as a result, funding for this activity stopped.
Tanzania	Energy	Malagarasi Hydro (Energy)	Cancelled due to environmental considerations.

Projects without Original ERR: 31 Open Projects without Original ERR: 13

Projects for which No ERR will be Calculated: 3
Country	Project	Activity	Reason	Closeout ERR Status
Georgia II	Improving General Education Quality	Education Assessment Support	There is no literature or standard practice on methods to quantify the benefits of testing.	It is too early in the compact to determine.
Indonesia	Green Prosperity		As two of the four activities with the project are grant facilities, the size and composition of the individual grants at EIF were unknown and it was not possible to calculate an original ERR. The two remaining activities are complementary to the grant facilities.	As a grant facility, there will likely be no project-level closeout ERR. However, ERRs for individual grants will be calculated.
Indonesia	Procurement Modernization		Insufficient information and data to quantify benefits.	There will likely be no closeout ERR for the same reason listed for the original ERR.

Projects Included in Other ERRs: 1
Country	Project	Activity	Reason	Closeout ERR Status
Georgia II	Industry-led Skills & Workforce Development	Strengthening Sector Policy and Provider Practice	The costs are bundled together with the Competitive Program Improvement Grants activity ERR, as the two activities are complementary.

Closed Projects without Original ERR: 18
Country	Project	Activity	Reason
Burkina Faso	Agricultural Development	Access to Rural Finance	Compact approved without a cost-benefit analysis for this activity.
Burkina Faso	Agricultural Development	Diversified Agriculture	Compact approved without a cost-benefit analysis for this activity.
Burkina Faso	Rural Land Governance		Compact approved without a cost-benefit analysis for this activity. In 2011 an ERR was calculated for the phasing decision at 10.8%.
Burkina Faso	BRIGHT 2 Project		Compact approved without a cost-benefit analysis for this activity. This project built on MCC’s 2005 threshold program.
El Salvador	Productive Development	Finance and Investment Support	This was a grant facility, so there was no data ex-ante to construct an original ERR.
Ghana	Rural Development	Financial Services	Activity was considered too small to construct an ERR ($24M).
Ghana	Rural Development	Procurement	Activity was considered too small to construct an ERR ($2M).
Lesotho	Water Sector	Wetlands and Restoration Conservation	Activity was considered too small to construct an ERR ($5M).
Lesotho	Private Sector Development	Payment and Settlement Systems	Activity was considered too small to construct an ERR ($1.7M).
Lesotho	Private Sector Development	Gender Equality in Economic Rights	Activity was considered too small to construct an ERR ($1.04M).
Madagascar	Agricultural Business Investment		N/A
Mali	Industrial Park		N/A

Projects were added after EIF: 5
Country	Project	Activity	Reason
Mongolia	Energy & Environment	MC Energy Investment Fund	This project is a result of funds being reallocated from the cancelled Rail project.
Morocco	Artisan and Fez Medina	Literacy & Vocational Training	This project is the result of a reallocation of funds away from other projects.
Morocco	Fruit Tree Productivity	Catalyst Fund	The activity was added after a re-scoping decision. The funds for the activity are included in the other Fruit Tree Productivity activity closeout ERRs.
Namibia	Education	Vocational and Skills Training, VGTF	The activity was added to complement the National Training Fund.
Tanzania	Energy	Kigoma Solar Power	This was added at the re-scoping decision following the cancellation of the Malagarasi Hydro Activity.

Table 8

Compact-Level Summary of Projects without a Closeout ERR
Country	Cancelled Projects	Projects with No ERR Planned	Pending ERRs	As a % of Total Projects	As a % of Total Funds Disbursed
Armenia	-	1	-	33.3%	76.7%
Benin	-	1	-	25%	6.5%
Burkina Faso	1	2	5	89%*	84.6%*
Cabo Verde	1	3	-	100%	100%
El Salvador	-	-	-	0%	0%
Georgia	-	2	-	40%	14.4%
Ghana	-	6	-	46.2%	41.5%
Honduras	-	2	-	100%	100%
Lesotho	2	2	-	40%	2.3%
Madagascar	2	-	-	100%	100%
Mali	3	-	-	100%	100%
Mongolia	3	-	-	30%	0.20%
Morocco	-	-	-	0%	0%
Mozambique	1	-	-	11.1%	16%
Namibia	-	1	-	8.3%	8.2%
Nicaragua	1	1	-	66.7%	40.8%
Tanzania	1	1	4	66.7%**	85.9%**
Vanuatu	-	-	-	0%	0%

*This will become 33.3% of total projects and 23.1% of total funds disbursed when the five pending ERRs become available. **This will become 22.2% of total projects and 18.2% of total funds disbursed when the four pending ERRs become available.

Annex III

Excerpts from MCC’s Summary of Findings

I. Measuring Results of the Morocco Enterprise Support Project

Economic Rate of Return (ERR) Estimates and Decision Not to Scale-up the Enterprise Support Pilot

As there was insufficient empirical evidence on the extent to which post-creation business training would increase revenues, productivity and survival rates of new enterprises, the Enterprise Support Project was structured as a two-year pilot program subject to a rigorous impact evaluation using a randomized design. Scale-up of the project was made contingent upon the results of the pilot program producing an estimated ERR above 10 percent for the scale-up project. The ERR prior to the implementation of the pilot program was estimated at 13.5 percent. Upon results one year after implementation, the overall ERR including scale-up phase projections was estimated at 1.21 percent while the ERR of the scale-up phase alone was estimated at 7.21 percent.[[Given the nature of post-creation business support, impacts on value-added growth and business survival rates are difficult to detect/measure after a single year. However, costs were greater than originally expected, which was the primary reason the decision was made not to scale up. While the results of the impact evaluation as summarized in this report further supports MCC’s final decision not to scale-up (as well as the fact that costs at closeout were even higher than estimated at the time of the scale up decision), previous interventions in post-creation business support would suggest that some benefits from support may accrue several years after implementation, longer than the total of two years given to the pilot program to be evaluated. Unfortunately, due to concerns about attrition rates (e.g. difficulty in following up with businesses) and possible provision of support services to control group businesses, further impact evaluation of this Project is not feasible.]] MCC decided not to proceed with scale-up of the Enterprise Support Project as the revised ERR estimate did not pass the hurdle-rate of 10 percent and was not estimated to generate sufficient benefits to justify the costs of the project.

ERR at Closeout (Project-wide, not exclusive to Support to IGAs Activity)

At closeout, the results of the impact evaluation became available and it was possible to revisit some of the benefit assumptions that could not be updated at the time the scale up decision was made. The impact evaluation found SME failure rates were reduced by 30%, rather than the 20% reduction assumed by the original ERR. However, this has essentially no significant impact on the ERR (an interesting problem of itself that we have yet to resolve). The only other significant impact the impact evaluation found was an improvement in value added for the AGR-INDH sub-activity of 30%, vs. 5% as was assumed in the original ERR. Making this change to the parameter increased the ERR for the AGR-INDH sub-activity from -3.5% to 4.4%, a significant improvement but still not higher than the 10% threshold. The overall closeout ERR was -0.9%.”

II. Measuring Results of the Artisan and Fez Medina Project Artisan Production and Promotion and Fez Medina Activities

“Even though potters were convinced of the relevance of replacing traditional kilns with less polluting gas kilns, this activity faced serious reluctance for many reasons: (i) the 20% contribution of potters to the imported gas kiln was beyond the financial capacity of potters; (ii) potters were reluctant to contract loans from banks for various reasons, among which the reimbursement conditions and (iii) Banks were not willing to finance the majority of potters operating informal, unregistered businesses; (iii) the maintenance cost of imported kilns were perceived as a long-term burden. The project design did not take into account the potential of the local gas kiln manufacturers and favored imported kilns from the start. The project should have considered developing a local kiln based on local usage and adapted to the needs and capacities of Moroccan potters.”

III. Measuring Results of the Nicaragua Transportation Project

“The calculation of the ex-post ERR utilized the Roads Economic Decision model (RED) developed by the World Bank. This approach captures the project’s flow of net benefits to road users defined as benefits minus costs which are discounted across time to yield a single value, using pre- and post-construction traffic counts and estimates of future traffic counts. The ex-post ERR estimated by the independent evaluator showed that the roads failed to meet the 10 percent hurdle rate. The average for the project, as a whole, was estimated to be 2.1 percent. Actual capital costs were, on average, 2.2 times greater than those estimated in the feasibility studies.”

IV. MCG Gas Pipeline Rehabilitation: Project Summary and ERR Model Revised

“Before the rehabilitation work was undertaken, the ERR was estimated to be 11.7% based on a forecast of a reduction in the amount of gas leakage from the pipeline and a assumed corresponding sale of carbon credits due to the reduction in methane emissions to the atmosphere. However, the high gas leakage rate was reduced before the rehabilitation work was undertaken, and it was not possible to sell carbon credits.”

Bibliography

Cronshaw, Mark, and Gustavson Associates. MCG Gas Pipeline Rehabilitation: Project Summary and ERR Model Revised. Rep. 23 June 2010. Web.

Millennium Challenge Corporation “Measuring Results of the Morocco Enterprise Support Project”, MCC, Summary of Findings, 2015

Millennium Challenge Corporation “Measuring Results of the Artisan and Fez Medina Project Artisan Production and Promotion and Fez Medina Activities”, MCC, Summary of Findings, 2015

Millennium Challenge Corporation “Measuring Results of the Nicaragua Transportation Project”, MCC, Summary of Findings, 2014.

Millennium Challenge Corporation. "Economic Rates of Return: Country Spreadsheets." Millennium Challenge Corporation. Web.

Ospina, Sandra, Aaron Mitchell, and Aaron Szott. 2014 Report on Closeout ERRs. March 2015.