Understanding the Sustainability of Off-grid Energy in Indonesia

Program Overview

MCC’s $474 million Indonesia Compact (2013-2018) included the $288 million Green Prosperity (GP) Project that aimed to increase economic productivity and reduce land-based greenhouse gas emissions. The project funded 23 community-based off-grid renewable energy grants totaling $85.3 million. Some were designed as community-owned renewable energy projects, while others were renewable energy components of natural resource management projects. These grants sought to substitute renewable energy for fossil fuels in remote and rural communities, opening opportunities for social and economic improvements through access to electricity.

Evaluator Description

MCC commissioned Social Impact to conduct an independent final performance evaluation of this Grant Portfolio. Full report results and learning: https://mcc.icpsr.umich.edu/evaluations/index.php/catalog/207.

Key Findings

Renewable Energy Infrastructure Functionality

The portfolio fell short of the objective of reducing poverty through low-carbon economic growth due to challenges in maintaining renewable energy technology operations.
Many technologies are no longer in use. Technology associated with 6 grants is completely non-operational and a further 13 grants are operating sub-optimally.

Domestic and Economic Use of Electricity

Outcomes related to energy substitution (and related greenhouse gas emissions reductions), electricity access and use are largely mediated by the generation capacity of infrastructure.
Provision of renewable energy has encouraged domestic economic activities, though the aggregate effects have been modest due to limited growth in businesses providing higher- value goods and services.
Grantee interventions promoting productive use of renewable energy have largely not been effective.

Sustainability

Only two grants have a good sustainability outlook, while almost half have a poor outlook.
Common factors influencing sustainability include operations financing, maintenance and repairs, the operating environment with on-grid alternatives, and ownership transfer challenges.

Evaluation Questions

This final performance evaluation was designed to answer the following questions.

1 To what extent are renewable energy assets functioning as intended four years post-compact?
2 How has energy access, consumption, and use for households and businesses changed in response to the provision of renewable energy? To what extent do these changes favor the reduction of greenhouse gas emissions?
2 What are the prospects for long-term sustainability for each grant?

Detailed Findings

These findings build upon the results published in 2020.

Renewable Energy Infrastructure Functionality

Multiple grants of roughly half the disbursements are not operational. Almost another half are of sub-optimal function. Only two small grants are functioning optimally. — Functional Status of renewable energy infrastructure, by grant. *Note: Relative size of boxes reflect the relative magnitudes of renewable energy disbursement. Names of smaller grants have been omitted.*

There were significant challenges in maintaining optimal renewable energy functioning in the four years post-compact. Renewable energy technology associated with 26% of grants (6 of 23) was completely non-operational at endline. Only four grants (17%) were functioning optimally across the entire grantee portfolio. The most common reason cited for non-operational status was unaddressed major repairs caused by factors such as climatic events and unexpected infrastructure quality issues. Lack of funding was often the primary barrier to repairs; however other barriers included lack of interest in utilizing the infrastructure and lack of clarity on infrastructure ownership.

Domestic and Economic Use of Electricity

According to compact monitoring reports, 9,095 renewable energy users were added by the 15 grants which targeted provision of renewable energy electricity for household use. Accounting for non-operational infrastructure and updated user numbers from functional technologies, the number of users with access to grant-funded renewable energy electricity at endline was less than 3,000. Technical issues have further influenced energy access by either making the technology inoperable, reducing hours of operation, or reducing the number of users with access to electricity. Where technology is functional, it is being used for lighting and/or powering appliances. Grants which provided renewable energy to substitute non-renewable energy sources or improve existing renewable energy sources were most likely to be contributing to reduced greenhouse gas emissions (48 percent of the portfolio, or 11 out of 23 grants), though this could not be substantiated by the evaluation.

While renewable energy provision has encouraged economic activities, these mostly produce items for direct consumption by the local community, such as refrigerated goods or snacks, and are unlikely to significantly change the local economic conditions. In a few locations, new renewable energy-powered businesses such as furniture, carpentry, and motorcycle workshops were reported, though these were not common. The evaluation also found limited evidence that grant-promoted economic use of renewable energy through provision of production houses, equipment, and training, had been effective, with only 5 of 15 production houses operational and in use. Key constraints to pursuing grant-promoted activities included insufficient working capital to purchase raw inputs, producer preferences to sell unprocessed goods, and lack of market linkage to facilitate sale of processed products.

Sustainability

Only 9% of grants (2 of 23) have a good sustainability outlook, with solid track records of operations and systems in place for generating sufficient funding for operations and maintenance (O&M). 48% of grants have a poor outlook. The majority of these are grants where the renewable energy infrastructure is already non-functional, and a few are grants that exhibit serious concerns about the operational viability of the systems. The remaining grants lie somewhere in the middle of the spectrum on sustainability, though many have yet to face their first major O&M hurdle.

Financing renewable energy is a common constraint as technologies in the portfolio are largely reliant on funding from users; however, the ability to regulate user fees is subject to user willingness and ability to pay and tariff regulatory requirements. Ensuring sufficient demand and payment compliance are also challenges to financing. A few grants receive ongoing support from grantees or subsidization from government, but these support mechanisms are not guaranteed in the long-term. Therefore, when faced with a serious O&M hurdle, grant-funded technologies have often been abandoned. Local knowledge and accessibility of vendors to support maintenance and repairs were also key factors for sustainability, as well as the arrival of alternative sources of energy to substitute for grant-funded renewable energy. In some cases, transfer of ownership created additional sustainability challenges, particularly where there was lack of clarity on infrastructure ownership. This was cited as the main reason why needed repairs had not been completed, particularly for village government-owned infrastructure.

Economic Rate of Return

MCC considers a 10% economic rate of return (ERR) as the threshold to proceed with investment. The evaluator provided feedback on the validity of the ex-ante ERRs in light of the evaluation findings for the six endline case study grants. Five of the six case study grants had known estimated ex-ante ERRs ranging from 14.20 % to 34.90%. However, renewable energy operational issues combined with limited evidence on reduced energy expenditures and income gains suggest that benefits are likely to be lower than anticipated across the six case studies.

More broadly, the sub-optimal functional status of technologies across the portfolio at endline calls into question whether many of the grants would cross the ten percent threshold. Moreover, in the absence of a portfolio-wide ERR estimation, there is limited quantitative evidence to justify MCC investment.

MCC Learning

MCC should conduct comprehensive demand and value chain analyses prior to the investment decision.
Joint ownership through a special purpose vehicle (SPV) between grantees, operators of the grid and the community does not guarantee sustainable outcomes.
Quality control mechanisms, including post-compact ownership models, should be entrenched early in the compact design and implementation to ensure sustainable outcomes for MCC infrastructure investments.
Evaluations of infrastructure investments, particularly small infrastructure, should inspect infrastructure prior to launching household or business data collection.
Grant facility investments should clearly articulate what success looks like and ensure this definition of success is shared between MCC and country partners.
MCC should carefully consider approaches to ensure the long-term operations and maintenance of newly introduced technologies.

Evaluation Methods

This final evaluation is primarily an ex-post performance evaluation, occurring four years post installation of most renewable energy assets. This is within the timeframe that renewable energy outcomes related to economic use were expected to have been realised. A planned endline follow-up to the impact evaluation of the Anekatek grant in Berau and a quantitative performance evaluation of the Akuo Energy grant in East Sumba conducted at interim were not feasible due to COVID-19 constraints on in-person data collection and complete equipment failure. The number of grants included in the evaluation portfolio were revised from 26 to 23 due to cancellation of renewable energy components under two grants (Lombok Utara Hijau Consortium and WWF Indonesia) and the reduced scope of the renewable energy component of a third grant (YLBHL). Endline data collection consisted of a portfolio review and case studies.

2022-002-2736