Program Overview
MCC’s $474 million Indonesia Compact (2013-2018) included a $228 million Green Prosperity (GP) Project that aimed to increase productivity and reduce land-based greenhouse gas emissions. The Green Prosperity cocoa portfolio had three grants totaling $26 million (60 percent compact and 40 percent private funding). The grants offered farmers inputs and training to increase yields, quality, and income; to improve natural resource use; and to lower greenhouse gas emissions.Evaluator Description
MCC commissioned Social Impact (SI) to conduct an independent interim performance evaluation of the Green Prosperity cocoa portfolio. Full report results and learning: https://data.mcc.gov/evaluations/index.php/catalog/206.
Key Findings
Farmer Training
- Most of the trained farmers applied what they learned, but they needed ongoing mentoring and more time to see results from their improved practices.
Progress toward Targeted Results
- The Sustainable Cocoa Production Program (GP-SCPP) and Cocoa Revolution (CR) grants performed well against their training targets and achieved short term behavior changes. However, they faced obstacles in trying to create sustainable markets for cocoa farming inputs (seedlings, fertilizer, etc.) and extension services (farmer training). The third grant, Economic, Quality, and Sustainability Improvement (EQSI), had implementation delays.
Sustainability of Practices
- With MCC funding, the GP-SCPP grant grew its operations from 13 to 50 districts and will continue operating post-Compact. The other two grants, CR and EQSI, will not continue post-Compact.
Program Design
- Weather extremes and market conditions significantly undermined project results. Implementers will need to develop climate resilience and risk mitigation measures to achieve higher yields.
Evaluation Questions
This interim performance evaluation was designed to answer the following questions to inform the final evaluation:- 1 To what extent have the three GP cocoa grants’ training approaches proven successful in improving farmers’ knowledge, attitudes, and practice of good agricultural practices and good environmental practices?
- 2 How has each grant progressed in achieving its short- and medium-term outcomes (phase 1) and long-term outcomes (phase 2)?
- 3 What evidence is there that results or outcomes of the GP cocoa grants will be further scaled and sustained, and what results appear to be less sustainable? Why?
- 4 What aspects of the GP cocoa grant approaches have proven to be most relevant in meeting the needs of the Indonesian cocoa sector?
Detailed Findings
Farmer Training
Farmer trainings in good agricultural, environmental, and financial management practices, as well as seminars dedicated to cocoa farming as a business, were built on grantees’ current understanding of best practices. Though farmers learned, appreciated, and adopted the training lessons, they did not find the information to be new. The repeated content from international donors and training fatigue meant farmers had difficulty remembering which entities provided specific content. Additionally, participating farmers felt that not enough time had passed
Cocoa beans ferment in large boxes at a cocoa fermentation center.
Progress toward Targeted Results
The grants had varying levels of success in achieving expected results. GP-SCPP made solid progress against training targets and leveraged investments from international cocoa companies. CR developed and demonstrated innovations (local fertilizer, solar dryers, and climate-smart agricultural techniques), but the implementation timeframe was too short to achieve uptake and obtain sufficient product to justify ongoing investments in the innovations. EQSI experienced implementation delays from a disagreement about the program approach. While EQSI effected changes in farmer practices, the intended fermented market chain did not proceed due to technical problems at the processing plant. Challenges to achieving and measuring longer-term outcomes remain for all three grantees, particularly for CR and EQSI:- Farmer incomes were unverifiable at the time of data collection. All three grantees had tools to measure yields and incomes, but the tools relied on estimates, self-reporting, and forecasting. CR and EQSI data on income was not available at the time of data collection.
- Certified farmers still sold their beans to local traders due to convenience, accessibility and their inability to produce the high-quality, certified beans that large cocoa processors require. This prevented the farmers from earning the premium that large processors pay.
- While the grantees aimed to provide necessary inputs for improved cocoa production to farmers, only CR had a specific mandate to provide affordable inputs. Participating farmers of all three grantees were unable to access inputs of suitable quality due to high prices and low availability. Also, the late start of the grantees’ provision of inputs or aid to farmers to buy inputs meant that farmers were largely unable to benefit from the schemes.
Sustainability of Practices
GP-SCPP has expanded implementation from 13 to 50 districts due to Compact support. GP-SCPP is expected to maintain its current scope of implementation with funds from consortium partners at least until 2020 and could be fully funded by private sector partners by 2020. CR does not plan to continue funding incentives and field facilitators post-Compact. EQSI has no further plans to fund any activities. Trained farmers may continue to mentor neighbor farmers, but without any monetary incentives from EQSI.Program Design
The evaluation found certain weaknesses in the design of the cocoa grants. Indonesian cocoa farmers are diversifying their crops and reducing cocoa production because of sharply declining cocoa prices and inclement weather. Climate-smart and resilient strategies might provide opportunities to assist cocoa farmers. Grant designers did not seem to take into account lessons learned from previous cocoa interventions. For example, fermentation and solar dryers had already been attempted in Indonesia with poor results. Finally, farmer certification did not automatically lead to certified products being sold to processing companies for a premium.
Cocoa farmers learn grafting techniques alongside grant-funded trainers.
Cocoa farmers attend Farmer Field School.
MCC Learning
- Understand the complete cocoa value chain and farmer behaviors when designing the intervention: MCC should examine the whole value chain to better understand whether the market will accommodate the proposed improvement and enable long-term sustainability.
- Design cocoa sector interventions with market conditions and other key factors in mind: Laws, regulations, institutions, incentives, weather patterns, intercropping, and the prevalence of pests should be considered when designing cocoa sector projects.
- Promote traceability: MCC, in partnership with implementers, should ensure that cocoa traceability systems are properly functioning and strengthen them when necessary.
- Re-examine cocoa certification: Farmers continued to sell their certified cocoa beans to middlemen, thereby forfeiting premiums. MCC should explore ways to better understand what prevents farmers from responding to these incentives.
- Project design and MCC’s learning agenda: MCC should consider designing projects guided by a learning agenda, as well as M&E perspectives and requirements, thus creating opportunities for more rigorous evaluations to accurately measure results.
Evaluation Methods
Pre-Post Performance Evaluation
A freshly cut cocoa pod.
The data collection sampling is convenience-driven at the provincial and district levels. The primary data is supported by document review and the use of grantee monitoring data on inputs and outputs. The evaluation is heavily dependent on respondents’ memories and perceptions.
Next Steps
The final evaluation covering all of the evaluation questions and ultimate impacts on farmer productivity and incomes will occur in late 2019.2018-002-1950