- $67,100,000Original Compact Project Amount
- $8,400,000Total Disbursed
Estimated benefits at compact closure correspond to $8.4 million of project funds, where cost-benefit analysis was conducted
Project Economic Analysis
MCC computed economic rates of return for the Rural Roads Rehabilitation Project as summarized in the table below.
Activity | Time | Estimated Economic Rate of Return (ERR) over 20 years | Estimated beneficiaries over 20 years | Estimated net benefits over 20 years (NPV discounted at 10% per annum in million USD) |
---|---|---|---|---|
Rural Roads Rehabilitation Project | At the time of signing | 25.9% | 142,905 | 80.52 |
At compact closure | 17.6% | 6,356 | 2.24 |
The benefits of the Rural Road Rehabilitation Project (RRRP) are attributable to (1) operating cost savings for vehicles and wear and tear on the roadbed, and (2) travel time-savings, as improvements in road condition permit higher average driving speeds. These reductions in the cost and time of transport translate to enhanced economic growth via reductions in the cost of production and distribution and by making other factors of production more productive for enterprises and individual farmers alike.
Project Summary
The objective of the Rural Roads Rehabilitation Project (RRRP) (original budget: $67.1 million; total disbursed: $8.4 million) was to expand communities’ access to agricultural markets, non-farm income opportunities, and social infrastructure by improving the condition of rural roads. To accomplish this, the project was designed to rehabilitate about one-third of the high priority rural roads in the lifeline network, including 943 kilometers of rural roads in 85 road segments throughout the country. This included 321 km of secondary roads and 622 km of local roads, for a total cost of USD $67.1 million. All rehabilitation works were to be on existing infrastructure and would include pavement rehabilitation, improvements on up to 19 bridges, drainage facilities, and road safety features.
Initially, it was estimated that approximately 360,000 rural inhabitants in 260 rural communities would benefit. However, during compact implementation, the RRRP underwent significant changes, primarily due to the devaluation of the US dollar against the Armenian dram and construction price escalation.
In 2007, MCA-A initiated the fast-track construction of roads and irrigation infrastructure (tertiaries) to show some work in progress. For those pilot constructions, MCA-A used ready-to-go designs prepared by other donors since its own feasibility and design activities were still underway. However, during the feasibility and design stage later that year, it was determined that MCA-A would not be able to fund all the roads that were originally included in the program.
To determine next steps, MCA-A worked with the Armenian Roads Directorate, the RRRP implementing entity responsible for technical oversight of project implementation, to restructure the project. The revised packaging was based on new ERR calculations derived from feasibility studies and designs submitted by a design consultant. As a result, in November 2007, the project scope was reduced by roughly two-thirds. Around 300 kilometers of rural roads, all of which had ERRs of at least 12.5 percent, were selected for rehabilitation, with the same total cost of USD $67.1 million. Overall, 84 communities located in different regions of Armenia were slated to benefit.
Accordingly, by June 2009 only the pilot H17 Armavir - Isahakyan - Gyumri road section (24.4 km) was fully rehabilitated using MCC funding (this pilot had been underway and completed prior to the events in March 2008). This road section was constructed under the MCA-A fast track initiative. It connects a number of villages in the Armavir region with Gyumri, the country's second biggest city. The rehabilitated road serves 12 rural communities and reaches 6,356 beneficiaries.
Following the funding hold in May 2008, the GoA decided to fund the project and allocated USD $16,800,000 to rehabilitate initially planned road sections. By June 2009, some works, such as earthworks, pothole patching, asphalt concrete layering, and rehabilitation of sidewalks, were already implemented under the re-scoped RRRP through the GoA funding. Prior to rehabilitation works, the MCA-A roads team, together with Armenian Roads Directorate and the feasibility and design consultant, had conducted a series of community outreach meetings to inform the local authorities and actual beneficiaries—farmers—about the future construction works.
As a result of the hold and the end of MCC funding for roads construction and rehabilitation, on September 25, 2009, MCA-A proceeded to transfer the RRRP to the Ministry of Transport and Communication of Armenia. Also, MCA-A, the contractor, and the Ministry of Transport and Communication signed agreements to amend and transfer the two contracts for the construction of the Armavir-Isahakyan-Gyumri road sections. On the same date, the assets and documents related to RRRP were transferred to the Ministry of Transport and Communication. MCA-A’s contract with the Armenian Road Directorate, the RRRP implementing entity, was terminated on September 30, 2009.
Even though this project was cut short due to the above mentioned circumstances, it did accomplish several achievements, including the rehabilitation of 24.4km of rural roads; development of designs for 575 km of rural roads; increased knowledge of best maintenance practices through study tours to Sweden, Finland, and Estonia; and production of guidelines for winter maintenance contracts.
The impact of this project was further extended by the fact that the World Bank financed the construction or rehabilitation of approximately 150 km of rural roads using the MCA-A funded detailed designs.
Key performance indicators and outputs at compact end date
Indicators | Baseline (2006) | Actual Achieved (2008) | End of Compact Target (2011) | Percent Complete[[MCC uses the following formula to calculate Percent Complete for all numerical indicators, (Actual – Baseline)/(Target-Baseline)*100]] |
---|---|---|---|---|
Government budgetary allocations for rehabilitation of road sections in the road lifeline network | Not available | 7,227,000 Armenian drams | 3,310,000 Armenian drams | 218% |
Government budgetary allocations for routine maintenance of the entire road network | Not available | 6,656,000 Armenian drams | 6,290,000 Armenian drams | 106% |
Average daily traffic on pilot roads | 637 | 735 | 706 | 142% |
International Roughness Index for pilot roads[[International Roughness Index (IRI): Roughness is a measure of the irregularity of the road surface. It affects the operation of a vehicle (safety, comfort and speed of travel) and costs of operation through vehicle wear, fuel consumption and the value of human and asset time spent in transit. This affects the economic evaluation of proposed road maintenance and upgrading expenditures.]] (meters per kilometer) | 14.16 | 3.47 | 4 | 105% |
Pilot road sections rehabilitated (kilometers) | 0 | 24.4 | 24.4 | 100% |
Explanation of Results:
The average daily traffic on pilot roads increased; however, it can not necessarily be attributed to the road rehabilitation project without having an estimate of how much traffic would have changed in the absence of the rehabilitation. Traffic could have increased (or decreased) due to other factors outside of the RRRP.[[Mathematica’s Evaluation of a Rural Road Rehabilitation Project in Armenia Report. March 12, 2015.]]
Evaluation Findings
Even though funding was put on hold, eventually indefinitely, for the RRRP, MCC continued with the independent impact evaluation, which covered roads designed using MCC funds but built as part of the World Bank project. During the compact, MCC funded an increase in the sample size of the Integrated Survey of Living Standards (ISLS), the Armenia national household survey, conducted by the National Statistical Service of Armenia on an annual basis. MCC’s independent evaluators planned on using the ISLS to evaluate the impact of RRRP before the project was placed on indefinite hold. Since the increase in sample of the ISLS was not tied to the RRRP, the data collection continued until the end of the compact and was provided to MCC. Both the GoA and the World Bank expressed interest in having MCC’s independent evaluator analyze the ISLS data as previously planned to provide insights into the impacts of rural road rehabilitation. In addition, MCC saw value in completing the evaluation because of the large percentage of MCC’s portfolio invested in roads.
Component | Status |
---|---|
Endline Report | Completed in 2015. Report and de-identified data are public. |
The original RRRP was designed to improve the quality of the lifeline road network in Armenia in order to enhance the economic performance of the agricultural sector. It was expected that improved road quality would reduce transportation costs and increase vehicular activity, which would increase access to markets and social infrastructure. As a result, farmers would be able to access agricultural inputs at cheaper prices and an increased number of retailers and buyers of agricultural products could access the communities, thus creating conditions for farmers to sell their agricultural production at a higher price. These changes were expected to incentivize farmers to invest more, thereby increasing employment and production, leading to improved performance of the agricultural sector and poverty reduction.
The evaluation included the following research questions:
- Did rehabilitating roads affect the quality of roads?
- id rehabilitating roads improve access to markets and social infrastructure?
- Did rehabilitating roads improve income from employment?
- Did rehabilitating roads affect agricultural productivity and profits, and if so, by how much?
- Did rehabilitating roads improve household well-being for communities served by these roads, especially income and poverty?
The independent impact evaluation found that road rehabilitation efforts can improve road quality and increase the use of roads in the short term. There was a 39 percentage point increase in favorability rating of regional roads and a 20 percentage point decrease in market access difficulties. However, those changes were not sufficient to stimulate increases in agricultural production and sales in a timeframe of one to two years. For additional information on the road evaluation, see the Evaluation Brief.
Evaluation Learning[[The full MCC Learning documents, which detail the motivation for the lesson and how MCC plans to apply it, are found in MCC’s Evaluation Catalog. These lessons were developed at the time of the evaluation publication and are framed within this context.]]:
Several key lessons learned from the Rural Roads Rehabilitation Project evaluation contributed to a broad set of lessons derived from other roads evaluations and the findings of MCC’s Transport Sector Practice Group’s internal reviews. These lessons included:
- Base evaluation decisions on a clear program logic.
- Set realistic time horizons and keep data collection plans flexible.
- Ensure sufficient statistical power.
- Conduct road network analysis.
- Consider alternatives.
- Address road maintenance constraints.
- Improve road investment economic assessments.
- Ensure collection of high-quality data.
- Enforce design review standards.