The Millennium Challenge Corporation is releasing its first set of independent impact evaluations, which use rigorous statistical methods to measure changes in project participants’ farm and household incomes. These first five impact evaluations—for farmer training activities in Armenia, El Salvador, Ghana, Honduras, and Nicaragua—reflect a small portion of both MCC’s investment and evaluation portfolios but offer valuable lessons and a first look at how MCC uses evaluations for accountability, learning and for improving its work.
Independent evaluations are an important part of MCC’s evidence-based approach. MCC uses evidence—both existing evidence and evidence built through monitoring and evaluation—to guide decision making on country selection, program design, appraisal, and implementation. Coupled with transparency and a culture of learning, this approach allows MCC to continuously improve its own programs while also benefitting other U.S. agencies and the international development community.
MCC delivers results along a “continuum of results,” from policy reforms associated with MCC compact eligibility to inputs, outputs, outcomes, and ultimately to measurable increases in farm and household incomes. Impact evaluations compare results from the people who participated in MCC-funded programs to those who did not. This indicates whether gains were the direct result of MCC’s investment. Independent firms conduct the evaluations to ensure objectivity, and results are made public to help foster accountability and learning.
According to monitoring data, MCC was very successful in meeting or exceeded its output and outcome targets for the evaluated activities. The average completion rate of output and outcome targets is: Ghana (103 percent), Armenia (103 percent), Nicaragua (112 percent), El Salvador (131 percent), and Honduras (158 percent).
These five impact evaluations also provide encouraging news about MCC program successes:
- In El Salvador, the evaluators found that dairy farmers doubled their farm incomes.
- In Ghana, northern region farmers’ annual crop income increased significantly relative to the control group, over and above any impacts recorded in the other zones.
- In Nicaragua, project participants’ farm incomes went up 30 percent after two to three years of project support.
In fact, these evaluations show increases in farm income in three out of the four countries where methodologically sound evaluations were possible. While MCC was successful in exceeding its output and outcome targets and saw increases in farm incomes in three of these countries, none of the five evaluations were able to detect changes in household income. The possible reasons behind this vary across countries, and may include limited time to observe whether changes in farmer behavior occurred, delays in implementation of other program components closely linked to training activities, or evaluations that were unable to maintain credible comparisons between farmers that participated in the training activities and those that did not. These findings also raise interesting questions about traditional assumptions of how program interventions lead to increased household income (as opposed to farm income) and the challenges associated with producing and measuring changes in household income.
MCC is applying lessons from these impact evaluations to improve the effectiveness of its future program investments and evaluation practice, and has already found opportunities to apply lessons to its current portfolio with course corrections of ongoing programs and evaluations. Many more evaluations are in process or planned, and the results of these and other evaluations will help MCC and our partners more efficiently reduce poverty through economic growth.
For more information, including full independent evaluation reports and MCC issue briefs on each report, visit http://www.mcc.gov/pages/results/evaluations.