Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

  • Final Evaluation Brief
  • June 2020

Sustainable Cocoa Production in Indonesia

Enhanced productivity and income challenged by conditions in a mature cocoa sector

View as PDF

Program Overview

MCC’s $474 million Indonesia Compact (2013–2018) included the $228 million Green Prosperity Project, which aimed to boost productivity and reduce land-based greenhouse gas emissions. Three grants under this project—with total funding of $26 million (60% compact and 40% private)—aimed to address the long-term decline in Indonesian cocoa production and support the development of a sustainable cocoa industry. The grants provided agricultural inputs as well as training and coaching to increase yields, quality, and income; improve natural resource use; and lower greenhouse gas emissions.

Evaluator Description

MCC commissioned Social Impact to conduct an independent final performance evaluation of the Green Prosperity sustainable cocoa portfolio. Full report results and learning:https://data.mcc.gov/evaluations/index.php/catalog/206.

Key Findings

Program Design and Theory of Change

  • The Sustainable Cocoa Production Program (SCPP) grant leveraged implementing partner Mars’s technical assistance delivery model.
  • Grantees Cocoa Revolution and Economic, Quality, and Sustainability Improvement (ESQI) lacked a strong foundation and suffered from a combination of logistical challenges and time constraints.

Implementation and Adoption

  • Producers stressed the need for technical assistance that integrates coaching into capacity building to address productivity and farm management challenges.
  • Few producers accessed the financial credit the grants offered for production systems because they believed they would get a low return on investment. Yet some buyers perceived improvement in cocoa quality.

Knowledge Management Systems

  • Knowledge management systems mostly supported implementation adjustments but were not accessible to all key stakeholders, especially Government of Indonesia staff.

Sustainability

  • Activities based on beneficiary demand are probably sustainable. Other sustainability mechanisms, such as fermentation and prevention of pest and disease outbreaks, were abandoned because they did not raise income.

Evaluation Questions

This final performance evaluation was designed to answer the following evaluation questions for the cocoa portfolio as a whole and for each individual cocoa grant.
  1. 1 Program Design and Theory of Change. To what extent were the theories of change valid in achieving the overall project objectives?
  2. 2 Implementation and Adoption. To what extent have the cocoa grants’ (SCPP, Cocoa Revolution, and EQSI) approaches and activities improved farmers’ knowledge, attitudes, and implementation of good agricultural and environmental practices?
  3. 3 Knowledge Management. How did the cocoa grantees monitor grant progress toward results and outcomes during implementation, and how did they use this information to manage performance?
  4. 4 Sustainability. Which results or outcomes of the cocoa grants are likely to be sustainable and scalable, and which do not appear to be sustainable and scalable?

Detailed Findings

These findings build upon the interim evaluation report results published in 2019.

Program Design and Theory of Change

Mars Technical Assistance Delivery Model and Map of GP Cocoa Deographic Coverage
All three grants deployed large-scale technical assistance to improve smallholder productivity practices. The technical assistance, coupled with activities promoting environmental sustainability and access to producer business and finance skills, aimed to facilitate more capital-intensive practices.

Adoption of promoted practices correlated closely with the presence of technical assistance infrastructure and pre-existing or prior supply chain interventions. Specifically, SCPP benefited from its experience with technical assistance from the Mars Community Development Centers/Cocoa Village Clinic model.

Implementation and Adoption

Grant-provided training in good agricultural practices reinforced trainees’ knowledge. Producers also stressed the need for coaching to address intermittent challenges such as pest and disease outbreaks and soil nutrition problems.

Business and finance training did not motivate producers to use credit to invest in their production systems. Producers often did not have time for or interest in keeping financial records. They were also reluctant to apply for loans, citing concerns about productivity and their ability to repay. Poor transparency in cocoa bean price setting also limited market access improvements.

Nonetheless, some buyers noted that cocoa bean quality improved through the life of the grant as producers reduced waste and met certification standards for moisture content and bean count.

Knowledge Management Systems

The SCPP system overcame initial challenges to provide timely data to guide management decisions. Likewise, Cocoa Revolution’s system faced challenges but adjusted the program in response to monitoring and evaluation data. ESQI had insufficient time to initiate a comprehensive data management system.

For all grants, limited access to these systems by partners and the Government of Indonesia resulted in frustration. After the project, the partners reverted to the pre-project scenario of proprietary data control and competition.

Sustainability

A woman sorting bean samples at the Indonesia Coffee and Cocoa Research Institute research station in Jember, East Java.

Sorting bean samples at the Indonesia Coffee and Cocoa Research Institute research station in Jember, East Java

For SCPP and Cocoa Revolution, sustainability certification proved a viable strategy for responding to growing market demand for sustainably sourced cocoa and for raising incomes. Fermentation was not viable because of logistical constraints and marginal price differentials. Nurseries responded successfully to producer demand.

One remaining challenge is the need to foster smallholder capital investment in higher-yield farming systems to reach estimated productivity levels for profitable cocoa production.

Economic Rate of Return

MCC considers a 10% economic rate of return (ERR) the threshold to proceed with investment. Although the evaluator did not recalculate the ERRs, they provided feedback on the validity of the ERRs produced by MCC (GP-SCPP – 17.25%; Cocoa Revolution – 32.92%; ESQI – 39.48%) in light of the evaluation findings. MCC modeled all three grants’ benefits on net farmer revenue to generate the ERR, estimating benefits in higher yields and income over 20 years and focusing on the long-term benefits of training in farming techniques rather than on seedling distribution alone. In addition, the model assumes that achieving yields greater than 2 t/ha requires investment into fertilizer and pesticides without considering tree age, access to inputs, and soil conditions—all key issues affecting productivity and income in the short and long term. The evaluation’s qualitative findings—particularly the limited smallholder capital investments into production systems and the extreme fragility of farmer productivity due to exogenous variables such as climate and price—call into question whether ERR targets are achievable.

MCC Learning

  • MCC investments should have a sound economic justification and undergo proper problem identification.
  • MCC should deeply consider its value added when investing in a crowded sector where the private sector, the partner government, and other donors are already intervening.
  • Evaluations of individual grants from a grant facility present unique evaluability challenges. These challenges affected the cocoa evaluation and should be contemplated before including grant facilities in future MCC programming.

Evaluation Methods

Evaluation data collection details.
This mixed-methods final performance evaluation collected primary data from 62 key informant interviews with MCC staff, grantees, staff of subgrantees’ private sector companies, buyers and traders, community leaders, and producers and from direct observation of 25 program-related sites. In addition, the evaluation team conducted 20 focus group discussions with producers and a quantitative mini-survey of 115 farmers who had participated in focus group discussions in Sulawesi and East Nusa Tenggara.

The data collection sampling was purposive at the provincial and district levels. Primary data were supplemented by document review and grantee monitoring data of inputs and outputs. The evaluation relied heavily on respondents’ memories and perceptions.

The exposure period varied by grant from 14 to 27 months. Data were collected in October and November 2019, approximately 1.5 years after the compact closed.

2020-002-2350