Those familiar with MCC will know that we commit to independently evaluate all of our investments to ensure accountability of results and resources to the American taxpayers and MCC’s partner governments and stakeholders. We publish all of these evaluations on the MCC website — no matter the findings — as a part of our commitment to the highest standards of transparency. Another important reason for our commitment to independent evaluations may be less known; MCC uses evaluations as a key tool for agency and sector learning. As part of this process of learning to achieve better development results, MCC recently published an exhaustive analysis of our road investments to date: Lessons from MCC’s Investments in Roads, the latest in our Principles into Practice series.
Road investments are attractive to partner governments and donors alike because of their potential to improve access to goods, services, markets, and information. Since MCC’s establishment in 2004, we’ve built or rehabilitated roads in 16 countries in Latin America, West Africa, Eastern Europe, and the Pacific. These roads investments have totaled more than $2 billion — close to 30 percent of our total compact investments. Using USAID and OECD data, we estimate that MCC is one of the world’s largest bilateral donors in the transportation sector. The graphic below depicts MCC’s roads compacts to date, detailing the size and scope of MCC’s investments, as well as the latest economic rates of return.
Differentiating Economic Rates of Return (ERRs): Original ERRs are forecasts of the likely economic benefits and costs of roads investments, conducted prior to the start of a compact. Closeout ERRs update original ERRs and are produced by MCC after compacts close; they are still forecasts, as some benefits do not start until after a compact is completed. Evaluation-based ERRs are developed as part of MCC’s independent evaluation process, better capturing costs and benefits not seen at initial compact closeout.
The paper identifies seven lessons, applying to all phases of the project lifecycle, with the goal of improving road project design, implementation, and evaluation. The lessons aim to advance implementation consistency across investments, better maximize investment funds, reduce risk and cost overages, and improve the sustainability of investments after MCC’s five-year compacts end. Below we highlight two lessons that could be particularly useful for prospective and current development practitioners alike.
The Project Development Lesson: Understand Your Problem, and Let the Problem Guide You
Without a thorough understanding of the development challenge, how it can be solved, and how it can be evaluated, it is hard to develop a project whose success can be comprehensively measured. Lacking a clear connection between what a project is doing (build a road) and what a project aims to impact (say, increase local incomes through increased commerce) makes it more difficult for a project to achieve impact, and less likely that a project’s success can be measured. A coherent, well-researched, evidence-supported theory of change that clearly identifies the problem and accounts for expected and measurable outcomes can make project development (and evaluation) far stronger. It can also help MCC identify possible complications and problems to improve estimates of a project’s costs and benefits and achieve long-term success.The Evaluation Lesson: Carefully Consider Your Evaluation Choice
In the development space, randomized control trials and impact are constant bywords. However, our analysis of our roads investments found that road projects do not necessarily lend themselves well to impact evaluations (the evaluation approach that compares what happened with an MCC investment to what would have happened without it). There are a few straightforward reasons for this:- it’s difficult to develop a counterfactual (that is, how the area would have changed, or not, if we didn’t build a road);
- it’s difficult to measure impacts on economic growth from a road (often because the benefits are individually small but widely dispersed across a road network); and
- it’s uncertain when data collection should occur following road completion (are effects realized after 2 years? 5 years? 10 years?)
Moving Forward, with Lessons in Hand
MCC currently has active road projects in Côte d’Ivoire, Liberia, and Nepal, and MCC is applying lessons derived from this analysis in each of them. Project development in Nepal has been enhanced by a review of the strategic road network to identify investment opportunities with the highest potential for economic benefits. In Liberia, sustainability is the focus with the compact focused on strengthening the government’s road maintenance regime. Perhaps most importantly, we have redesigned 10 road evaluations to match our new strategy of analyzing roads investments.MCC’s rigorous monitoring and evaluation of its projects is key to helping MCC achieve its mission of reducing poverty through growth. Evaluations equip our partner countries with vital information needed to continue to build on compact achievements in transportation and other key sectors and lead their own development. We look forward to hearing about your experiences from the field of road upgrades, and what lessons you have gathered along the way to enhance your work in this important sector.