Measuring the Results of the Philippines community-driven development program, the Kalahi-CIDSS Program

In context

The MCC compact with the Government of the Republic of the Philippines is a five-year (2011-2016), $434 million investment. The compact is intended to support reforms and investments to modernize the Bureau of Internal Revenue, expand and improve a community-driven development project, Kalahi-CIDSS, and rehabilitate a secondary national road in Samar province. 

The Kalahi-CIDSS project, $120 million, will a) provide grants to communities for small-scale infrastructure and services associated with community-selected and managed sub-projects, b) strengthen community participation in development and governance activities at the village and municipal level, and c) improve responsiveness of local government to community needs. The project will build on and support the participatory planning, implementation, and evaluation methodology developed by the Philippines Department of Social Welfare and Development in collaboration with the World Bank.  The Department of Social Welfare and Development will implement this project, overseen by a National Steering Committee that includes representatives from government departments and NGOs, and in collaboration with local governments.

Program Logic

The Philippines lags significantly behind other countries in the region with respect to government development expenditures as a percentage of GDP and infrastructure investment and quality. The Asian Development Bank’s 2007 growth diagnostic report found that inadequacies in infrastructure are a critical constraint to growth and that the availability of basic infrastructure (water, sanitation, roads, electricity) is regressive. Provision and use of education and health services were found to vary across regions, particularly as a function of incomes.

Community driven development projects are a strategy for addressing these constraints and providing community empowerment and poverty reduction. In the past, they have been used to support a wide range of community priority needs including provision of water supply and nutrition programs for women and children; building of school classrooms, day care and health facilities, farm to market roads, foot bridges, and drainage systems; and support for productive enterprises such as pre- and post-harvest facilities as well as community capacity building.

KALAHI-CIDSS is a community driven development project implemented by DSWD of the Philippines. Through KC, communities (“barangays” or villages) are trained, together with their local governments, both at the barangay and the municipal level, to choose, design and implement sub-projects that are intended to address their most pressing needs. This is done through a three-year, three-cycle program, which includes “social preparation” training for communities, barangays, and municipalities, and sub-project implementation. The KALAHICIDSS project to be funded by MCC is an expansion of an initial KALAHI-CIDSS project

In sum, the project aims to foster economic growth and reduce poverty by improving the responsiveness of local governments to community needs; encouraging communities to engage in development activities; and delivering benefits to barangay residents through individual sub-projects.

Measuring Results

MCC uses multiple sources to measure results, including monitoring data during Compact implementation, and independent evaluations, which in many cases are continued Post Compact. Monitoring data is typically generated by the implementers, and specifically covers the ‘treatment’ group of communities who received KC under the Compact.  The table below includes the key performance monitoring indicators, a sub-set of all of the monitoring indicators that are being tracked during implementation.

Evaluation Summary and Results

A keystone poverty reduction initiative of the Government of the Republic of the Philippines (GRP) is the Kapit-bisig Laban sa Kahirapan-Comprehensive and Integrated Delivery of Social Services (Kalahi-CIDSS or KC), a community driven development project implemented in the country’s 48 poorest provinces. The program gives representative volunteer teams from barangays (villages) the power to select, design and implement the public projects which they most need. In 2011, KC received US$120 million in funding from the United States government’s Millennium Challenge Corporation (MCC) Compact in the Philippines and $59 million in loan funding from the World Bank.

The MCC contracted Innovations for Poverty Action (IPA) to perform an impact evaluation of KC.  The evaluation endeavors to provide an independent assessment of KC’s impact and to contribute to broader research about the impacts of CDD programs.  In order to isolate KC’s impact, the evaluation makes use of the randomized-controlled-trial methodology.  The impact evaluation sample consists of 198 municipalities (with 33% to 69% poverty incidence), spread over 26 provinces and 12 regions. The 198 municipalities were paired based on similar characteristics (99 pairs) and then randomly assigned into treatment and control groups through public lotteries. The sample size is large enough to be able to detect MCC’s projected 8% change in household income as well as other smaller effects.

As part of the impact evaluation, baseline quantitative and qualitative data were collected in the study area from April to July 2012. The quantitative data came from 5,940 household surveys in 198 barangays (one from each municipality) and 198 barangay surveys implemented in these same barangays. Data for the qualitative section were collected in a subsample of 24 municipalities (12 pairs), spread across 12 provinces. The qualitative data came primarily from key informant interviews with barangay and municipal leaders and focus group discussions with barangay residents.  The baseline survey data confirmed the overall balance of treatment and control communities along key household and barangay level indicators (see baseline balance findings in Appendix 1).

In early 2014, interim data were collected in a randomly selected sub-sample of 80 municipalities.  The purpose of this round of data collection was to capture short-term outcome variables related to governance, empowerment and community participation after 1 to 1.5 cycles of KC (out of 3 cycles). In addition, to barangay and household surveys the interim data collection included a structured community activity (SCA) implemented in all 80 barangays.  SCAs complement household-level survey questions by creating the opportunity to directly observe the community decision-making processes in a real-world setting.  Furthermore, in order to understand participation dynamics in formal government structures, the interim data collection also included barangay assembly (BA) observations, implemented in a sub-sample of 10 barangays.

IPA has prepared this interim report to present all findings related to the outputs and near-term impacts of KC.  The research team finalized a pre-analysis plan in September 2014, prior to analysis of any follow-up data.  The pre-analysis plan explicitly states the hypothesized areas of KC’s impact.  The hypotheses are grouped into three different domains: socio-economic, governance and community-empowerment.   The pre-analysis plan also specified a hypothesis on the match between development projects and the preferences of community members.  Table 0.1 below summarizes the results from each hypothesis test.  Those hypotheses with statistically-significant differences between treatment and control, i.e. that demonstrate impact due to the KC project, are highlighted in bold

Table 0.1 Summary of Evaluation Results

The socio-economic domain considers impacts related to the economic welfare improvements resulting from the implementation of KC subprojects in the community.  Hypothesis 1 tests whether KC reduced household travel time and costs to access basic services.  At interim, we do not find any overall improvement in access to basic services due to KC.  However, our subgroup analyses results indicate that KC did improve travel time and cost to key services for barangays that had lower levels of baseline governance.  We also find that indigenous peoples (IP) benefited more than non-IP from the impact of KC on this domain.  This latter result signifies that vulnerable subgroups are also benefitting from the subprojects implemented through KC.

The governance domain considers the changes in awareness and participation in local governance caused by the KC experience.  Hypothesis 2/3 examines both KC outputs and impacts.  Results for Hypothesis 2/3 indicate that KC succeeded at increasing the quantity and quality of participation in local governance.  KC broadened the base of participation through barangay residents’ active participation in KC barangay assemblies.  Furthermore, KC increased participation in KC community efforts related to the implementation of KC subprojects.   However, at interim, KC did not increased participation in community efforts outside of KC.  Hypothesis 4 also captures KC outputs, and results indicate that KC increased knowledge and awareness of local governance.  This finding suggests that residents in KC areas became more familiar with their barangay leadership and barangay development plans when they attended KC barangay assemblies.  Subgroup analyses results show that KC increases household participation in and knowledge of formal and informal structures more in barangays that had lower participation at baseline.  KC also improved barangay information sharing and inclusiveness more so for poorer households and barangays as well as for barangays that were not affected by Typhoon Yolanda.  Additionally, KC increased confidence and self-efficacy more in barangays with higher levels of baseline governance and barangays not affected by Typhoon Yolanda.  Lastly, KC had a more positive impact on governance in barangays that had lower participation, knowledge and awareness of local governance at baseline.

The community empowerment domain explores changes in community interactions and collective action caused by KC.   The various KC activities taking place during each cycle offer barangay residents multiple opportunities to interact with one another.  Results for Hypothesis 5 show that KC was in fact successful at increasing interaction among peers.  Additionally, we find through the testing of Hypothesis 6 that KC significantly increased participation in existing community organizations or support.  As a result of KC, IPs experienced a higher increase in participation in existing community organizations and interactions among peers than non-IPs.  KC also had a larger impact on increasing interactions among peers for households not classified as poor than households who were classified as poor.

We also explore KC’s influence on how well development projects implemented in barangays match the types of development projects prioritized by barangay residents (Hypothesis 7).  We find that KC did not improve the degree to which development projects correspond to baseline (ex-ante) preferences, but this may be because there was already such a high correspondence at baseline. 

Interim results also show that KC had similar impacts on females and males across all domains.

Overall, interim findings suggest that KC was implemented as intended and this is encouraging since it is not the case for every development program.  Moreover, KC had larger positive impacts on the more vulnerable subgroups, such as poorer households and IPs.  There is always a risk that development programs will primarily impact more fortunate respondents in the sample, it is reassuring to find that this is not the case with KC.  After 1 to 1.5 cycles of KC, there is some evidence of program impacts in community participation in groups/organizations.  We will assess longer-term impacts via subsequent data collection and analysis efforts planned for 2015.  By this time treatment areas will have completed 2 to 3 cycles of KC.

Next Steps

This compact and evaluation are both on-going.  The compact ends in May 2016.  The next steps for the evaluation include:

• Third-round data collection in July-December 2015
• Third-round report on KC impact in December 2016