Program Overview
MCC’s $65.4 million Vanuatu Compact (2006-2011) funded the $58.3 million Infrastructure Activity. The activity aimed to reduce travel times and vehicle operating costs in rural areas by reconstructing and sealing 149.7 km of two national roads: the Efate Ring Road and the Santo East Coast Road. This was originally expected to stimulate economic activity in the tourism and agriculture sectors, though MCC now values the economic benefits across all types of road users.
Evaluator Description
Vanuatu’s Millennium Challenge Account commissioned Transport Research Laboratories to lead a performance evaluation of the Infrastructure Activity. Full report results and learning: https://data.mcc.gov/evaluations/index.php/catalog/102.
Key Findings
Road Extension and Condition
- Prior to the intervention, the condition of the unpaved coral roads was very poor – with erosion gullies and deep depressions – equivalent to an international road roughness index of 15.
- Following the double surface dressing, the road condition improved to a smoother surface with only minor depressions – equivalent to an international road roughness of 3.
Traffic Volume
- Traffic volumes increased between 5 percent and over 300 percent from 2008 to 2010 across various road segments after the roads were sealed.
Vehicle Operating Costs and Sustainability
- 3 and 4-axle trucks accrued 70 percent of the total vehicle operating cost savings.
- The analysis reports an economic rate of return of 10.3 percent when considering the full package of road segments, though this was largely driven by higher economic rates of return for road segments with greater than 400 annual average daily traffic counts.
- Resealing the roads within the next 20 years was considered not cost effective with a negative economic rate of return value between -30 percent and -80 percent.
Evaluation Questions
The analysis examined whether the road upgrades:
- 1
Reduced road roughness? - 2
Increased traffic volume? - 3
Decreased vehicle operating costs?
Detailed Findings
Road Extension and Condition
The activity extended specific road segments 10-25 kilometers each and sealed the road segments with thin Double Bituminous Surface Treatment (DBST). The purpose of the seal was to improve surface drainage rather than strengthen the structural bearing capacity of the pavement, while also reducing road roughness. Roughness is generally defined as irregularities in the pavement surface that affects not only ride quality but also vehicle delay costs, fuel consumption, and maintenance costs. The analysis estimated the international road roughness index for road segments decreased from 15 to 3 for sealed project roads.
Traffic Volume
Sealing roads led to an average increase in traffic volumes of almost 200 percent between 2008 and 2010 on the Efate Ring Road, while the Santo Road had an average increase of almost 60 percent. Interestingly, the evaluators found a substantial range of increased traffic results between road segments on the same national roads. For example, the traffic on the Rentabo road segment on the Efate road increased by 328 percent, while the traffic on the Klehm’s Hill segment increased a mere 8 percent suggesting a difference in cost effectiveness by segment rather than the national road when analyzing the impact of the road sealing.
Vehicle Operating Costs and Sustainability
For the Highway Development and Maintenance (HDM-4) model, vehicle operating costs are specified as economic unit costs – costs before import duty, excise duty, VAT, etc are added – and costs to complete the roads upgrade are specified in both economic and financial terms. The vehicle types with the largest reductions in vehicle operating costs were Empty Truck (3-axle), Loaded Truck (3-axle), and Truck Trailer (4-axle).
Laying pavement.
The analysis reports it was cost effective to seal the individual road sections with annual average daily traffic counts (AADT) greater than 400. The sealing of individual road sections with AADT between 300-400 was considered marginally cost effective, however sealing road sections with AADT less than 300 was found to be not cost effective, particularly with the very low proportion of heavy vehicles.
Economic Rate of Return
- 24.2%
MCC Original Estimate - 10.3%
Evaluation-Based Estimate
The original Infrastructure Activity included civil works for the reconstruction or construction of priority infrastructure including roads, wharfs, airstrips and warehouses, with a total of 11 different infrastructure sub-projects on eight islands. The projected economic rate of return for this group of activities, calculated in April 2006, was 24.2 percent. In 2011, the economic analysis estimated sealing both Santo and Efate roads was economically justified with an economic rate of return of 10.3 percent. While sealing all the roads in Santo was justified with an ERR of 14.4 percent, sealing all the roads in Efate was not considered economically justifiable with an ERR of 7.4 percent.
MCC Learning
Collect accurate baseline data to properly document traffic volume and road roughness.
Although not relevant for the Ring Road given engineering requirements to have a uniform design standard, segment-specific analysis is important when considering a road investment as it may not be cost-effective to upgrade every segment of the road.
It is critical to incorporate sensitivity analysis into ex-ante cost-benefit analysis to understand what level of risk MCC is taking in its assumptions about key parameters, such as traffic counts.
For future MCC road investments to produce the highest return for the largest number of beneficiaries, they should be identified through an analysis of the relevant road network that assesses key criteria, such as current traffic volume and road roughness.
Evaluation Methods
The performance evaluation for the Infrastructure Activity relied on using the Highway Development and Maintenance (HDM-4) model - one of MCC’s standard tools to evaluate roads. It models the condition of the road with the upgrade and without the upgrade over time. The transportation costs (vehicle operating costs and travel times) in each scenario are modelled, and the difference between the two is the savings that accrues as a result of the investment. The analysis used the following data:
- Traffic surveys conducted before road sealing in 2008 and after road sealing in 2010 in five locations for Efate and four locations for Santo; and
- Direct observation of sealed road segments in 2010.
2020-002-2373