- $145,670,000Original Compact Project Amount
- $153,716,022Total Disbursed[[The project budget was increased based on the bids received for the pumping station, which revealed the need to increase the contingency budget for construction contracts.]]
Estimated benefits at compact closure correspond to $153.7 million of project funds, where cost-benefit analysis was conducted:
MCC computed economic rates of return (ERR) for both the Infrastructure Activity and the Water-to-Market Activity as summarized in the table below.
| Activity | Time | Estimated Economic Rate of Return (ERR) over 20 years | Estimated beneficiaries over 20 years | Estimated net benefits over 20 years (NPV discounted at 10% per annum in million USD) |
|---|---|---|---|---|
| Infrastructure Activity | At the time of signing | 27.5% | 126,000 | 167.1 |
| At compact closure[[The compact closure ERR was completed shortly after the compact end date, before the evaluation results were available, and therefore does not reflect findings from the final evaluation, which were released in 2016.]] | 24.4% | 421,407 | 145.5 | |
| Water-to-Market Activity | At the time of signing | 15.5% | 124,000 | 19.1 |
| At compact closure[[The compact closure ERR was completed shortly after the compact end date, before the evaluation results were available, and therefore does not reflect findings from the final evaluation, which were released in 2013.]] | 11.5% | 28,831 | 2.7 |
Economic analysis of the Irrigated Agriculture Project estimated the benefits attributable to each investment component. In particular, the analysis quantified expected incremental increases in income from newly irrigated land, the increase in high value-added crop cultivation, higher yields, lower production costs, and energy and water savings.
Project Summary
The Irrigated Agriculture Project aimed to increase the productivity of the agricultural sector by extending and improving the quality of the irrigation system, strengthening the entities that managed the system and enabling farmers to commercialize their products. This project included two activities: the Infrastructure Activity and the Water-to-Market (WtM) Activity.
Infrastructure Activity
The Infrastructure Activity (original budget: $113.2 million; total disbursed: $121.5 million) sought to improve dilapidated irrigation infrastructure in order to expand the agricultural land area under irrigated production and improve the overall efficiency of sourcing and delivering water to farmers. The activity addressed these issues by rehabilitating major sections of the country’s main canal systems, modernizing some of the most urgently needed pumping stations, introducing new gravity irrigation schemes, rebuilding tertiary canals, and restoring sections of the drainage system in the Ararat Valley.
During the initial project development stage, MCA-A hired the Water Sector Development and Institutional Strengthening Project Implementation Unit (Irrigation PIU under the State Water Committee of Armenia) to implement the Infrastructure Activity. However, progress was slow, and in 2008, MCA-A terminated the contract and took back responsibilities for implementation.
In 2008, the Infrastructure Activity was re-scoped due to the devaluation of the US dollar against the Armenian dram and construction price escalation. As a result, MCC and the GoA decided to develop a new program ensuring the most optimal mix of components to reduce poverty in the rural areas. The re-scoped activity components included rehabilitation of: (i) main canals, (ii) gravity systems, (iii) tertiary canals, (iv) drainage systems, and (v) pumping stations.
The main canal works included the rehabilitation of six canals to reduce water losses in the main canals, increase irrigated lands, and improve water distribution in the irrigation system. Over 40 kms of new canal lining were refurbished and over 260 water structures were installed. By the end of the compact, $22.7 million was invested in the canals.
The gravity systems[[Gravity sytems are a cost effective way to provide irrigated water utilizing the natural water pressure created by gravity.]] works included construction of five gravity systems to help save electricity and provide stable water supply to the communities. Construction of the gravity-fed irrigation systems included almost 50 kms of canal. By the end of the compact, around $8.8 million was invested in the rehabilitation of five gravity systems.
Tertiary canals were rehabilitated in more than 100 communities all over Armenia. These communities were selected from a list of priority communities identified by the GoA and became eligible under a 15 percent co-financing contribution towards the overall construction costs. Approximately 220 kms of tertiary canals were rehabilitated. By the end of the compact, $15.8 million was invested in tertiary canals.
The rehabilitation of drainage systems reduced ground water levels in the Ararat and Armavir regions, increased crop productivity and arable land in the drainage area, and helped to regulate groundwater levels. This rehabilitation included cleaning 470 kms of canals. By the end of the compact, over $23.1 million was invested in the rehabilitation of drainage systems in the Ararat and Armavir regions.
Rehabilitation of pumping stations in six regions in Armenia ensured more reliable water supply and increased irrigated lands. By the end of the compact, $40.6 million was invested in the rehabilitation of the pumping stations.
Water-to-Market Activity
The Water-to-Market Activity (WtM) (original budget: $32.4 million; total disbursed: $32.2 million) was designed to help farmers use irrigation improvements to introduce new technologies and shift to high-value agriculture (HVA), build the management capacities of the Water Supply Agencies (WSAs) and Water User Associations (WUAs), and provide training and access to credit for member farmers to transition to more profitable, market-oriented agriculture which would improve their income and lead to future economic growth in rural areas. To tackle these issues, the activity was separated into two sub-activities: (a) Improving Profitability of WUA Member Farmers; and (b) Strengthening Irrigation System Entities. These sub-activities are described further below.
During the program implementation, the WtM Activity also underwent other modifications based on recommendations made through different programmatic studies. These changes resulted in improved synchronization of the training component, established linkages between producers and agribusinesses, and increased efficiency of project management.
Improving Profitability of WUA Member Farmers Sub-Activity
The objective of the WtM’s Improving Profitability of WUA Member Farmers Sub-Activity was to ensure that the Irrigation Infrastructure Activity was sustained through a combination of farmer training, access to credit, and support to small agribusinesses. The sub-activity included three components: (1) Farmer Training, (2) Credit, (3) and Post-Harvest, Processing and Marketing.
The Farmer Training component was the strategic tool of the WtM Activity. Overall, two types of trainings were implemented:
- On-Farm Water Management trainings focused on introducing farmers to the newest technologies for on-farm water management in order to secure water savings.
- High-Value Agriculture (HVA) trainings introduced farmers to the new technologies, approaches, and practices for transition to higher-value agriculture.
It was assumed that the adoption of both practices would increase farming income. Farmers were first required to take the on-farm water management training and then the HVA training. The training component was scheduled in a way that a program farmer moved from one training to the next within a relatively short period of time to maintain high levels of interest and ensure a higher degree of adoption. Demonstration sites were directly linked to classroom training components as an extension education program (visits to demonstration sites versus viewing of demonstration videos) and a prerequisite to receiving a training certificate. Combined training demonstration sites were established to ensure the effectiveness of investments and higher income. Directional signage was introduced to promote the usage of demonstration sites and to increase accessibility for villagers.
The Credit component was to support the increase of affordable, longer-term credit to the WtM trainees. The main participants were the farmers who passed the on-farm water management and HVA trainings and wanted to make investments into new technologies. The total amount of credit provided to the banks and credit organizations for five years was USD $8.5 million, with a maturity period of up to seven years and with an annual interest rate of a maximum of 12 percent.
More than 79 percent of compact-funded loans were lent by Universal Credit Organizations, small lenders that are important players in the rural areas and compete with Armenia banks. These loans went to end-borrowers for any productive agricultural purpose associated with HVA crops and technologies promoted by the WtM Activity or directly related agribusiness activity in all regions of Armenia, excluding Yerevan. This component was monitored and implemented by the Rural Finance Facility Project PIU. Eligibility for credit was linked to the training activities, since insufficient access to finance proved to be one of the impediments for farmer adoption of new techniques introduced during the trainings.
Over the course of the compact, credit objectives became more explicit and targeted towards the following agricultural activities: adoption of on-farm water management and new irrigation technologies; establishment of orchards, vineyards, greenhouses, and processing/sale centers; establishment of collection and consolidation centers; and adoption of food safety and quality management systems. By the end of the compact, repaid loan funds were lent again resulting in a total of $12.7 million ($3 billion AMD equivalent) of credit distributed to over 1,000 farmers through existing Armenian banks and other financial institutions. Given its success, after the compact the GoA continued the administration of the credit program (expected through at least September 2030), using the repaid funds from the initial set of loans.
In November 2019, the extended program was transferred from the Ministry of Finance’s Foreign Financing Projects Management Center to the Ministry of Economy’s Department of Implementation of Agricultural Programs. The Ministry of Economy reports that as of March 2020, a total of 4,444 loans in the amount of 17.3 billion AMD have been provided to finance the below investments:
| Activity Type/Sector | % |
|---|---|
| Livestock | 27.5 |
| Orchards and vineyards | 10.1 |
| Food processing | 1.6 |
| Greenhouses and cooling facilities | 35.6 |
| Leasing | 5.7 |
| Vegetable growing and potato cultivation | 1.8 |
| Other purposes | 17.6 |
| Total | 100.0 |
The Post-Harvest, Processing and Marketing component established linkages between the producers and agribusinesses. To ensure successful operations and to strengthen the capacity of the processing enterprises, this component provided assistance to increase the quality of supplied products; facilitated the establishment of collection points and consolidation centers to store, sort and package produce; and linked farmers to consolidators.
Through these activities, food processors, wholesalers, and institutional buyers were linked to agricultural associations and communities. Project-supported consolidation centers and collection points started their operations in late 2009. More than 15 were established and became operational in 2010 to provide farmers with increased opportunities to sell their produce. By the end of the compact in September 2011, 24 collection centers and three consolidation centers were established, which provided farmers with appropriate conditions to store, sort, and sell their produce close to their land plots, without the transportation cost required to get to markets. Supermarkets (e.g., STAR supermarket) were linked to the Federation of Agricultural Associations, other grower organizations, and producers. A market information system, ARMIS, was developed to provide real-time price and marketing information to all interested parties—from farmers to processors and supermarkets. Exporters were supported under a unified “Fruits from Armenia” brand and a trial shipment of modern packed and palletized produce was initiated to Russian supermarkets. In 2009, MCA-A initiated two big public events—Buy Armenian Campaign and WtM-EXPO—to increase interest toward Armenian-made products. In December 2009, the GoA also adopted legislation on agricultural cooperatives, thereby meeting one of the compact’s conditions precedent.
Strengthening Irrigation System Entities Sub-Activity
Through the WtM’s Strengthening Irrigation System Entities Sub-Activity, MCA-A supported management reforms in the irrigation sector by developing and strengthening the capacity and effectiveness of irrigation system management entities, including WUAs and WSAs. The sub-activity sought to enhance the institutional capacity of 44 WUAs and three WSAs to maintain the rehabilitated infrastructure. MCA-A strengthened these institutions by helping them develop and implement Management Improvement Plans. In addition, WUAs received improved water management tools, including 180 computers with sophisticated budgeting, accounting and georgraphic information system software; 44 units of welding equipment; and 38 backhoe loaders to increase the efficiency of their maintenance operations. It was expected that as a result, WUAs could better monitor and manage water supply and demand, improve collection of water fees and membership dues, and enhance water user awareness of and involvement in the WUA management.
At completion, the Irrigated Agriculture Project represented one of the largest investments to date into the rehabilitation of Armenia’s vital irrigation infrastructure as well as provided extensive technical and financial assistance to rural farmers and to the government irrigation entities that support them.
However, during implementation, the various project activities were disjointed in targeting project beneficiaries and sequencing. The irrigation infrastructure rehabilitation activities were significantly reduced in scope and faced construction delays of two to three years. Despite irrigation construction scope changes and delays, farmer training in improved technologies commenced for farmers that already had some access to reliable water and/or would benefit from irrigation rehabilitation efforts. The estimates of reliable access to water for farmers trained ahead of the irrigation rehabilitation proved to be inaccurate for some communities. Credit on-lending and training of financial institutions began approximately at the same time as farmer training efforts. Post-harvest, processing, and marketing activities scaled up approximately two years after farmer training began and completed major initiatives, such as the creation of consolidation centers, in the final year of implementation. Finally, water user association capacity building occurred alongside other project activities with strong complementarities with the on-farm water management training content, although actual access to water varied. Infrastructure rehabilitation also was completed only in the final year of the compact.
Key performance indicators and outputs at compact end date
| Indicators | Baseline (2006) | Actual Achieved (2011) | End of Compact Target[[The targets included are the targets after the project was re-scoped, not the original targets.]] (2011) | Percent Complete[[MCC uses the following formula to calculate Percent Complete for all numerical indicators, (Actual – Baseline)/(Target-Baseline)*100]] |
|---|---|---|---|---|
| Primary canals rehabilitated | 0 | 41.8 km | 41.8 km | 100% |
| Pumping stations renovated | 0 | 17 | 17 | 100% |
| Conversion from pumping to gravity completed | 0 | 5 | 4 | 125% |
| Tertiary canals rehabilitated | 0 | 232.8 km | 220 km | 106% |
| Drainage canals cleaned | 0 | 470 km | 470 km | 100% |
| Government budgetary allocations for maintenance of irrigation system | Unavailable | 1,385,538 Armenian drams | 1,500,000 Armenian drams | 92% |
| Training/technical assistance provided for on-farm water mgmt (Total) | 0 | 45,639 participants | 45,000 participants | 101% |
| Training/technical assistance provided for on-farm water mgmt – Female | 0 | 14,520 women | No Target | No Target |
| Training/technical assistance provided for HVA (Total) | 0 | 36,070 participants | 36,000 participants | 100% |
| Training/technical assistance provided for HVA – Female | 0 | 12,189 women | No Target | No Target |
| Loans provided | 0 | $12,700,000 | $8,500,000 | 150% |
| Loan borrowers (Total) | 0 | 1,008 | No Target | No Target |
| Loan borrowers – Female | 0 | 106 | No Target | No Target |
| Training/technical assistance provided for post-harvest, processing, and marketing | 0 | 227 businesses | 225 businesses | 101% |
| Recovery of WUA operations and maintenance cost by water charges | 36.7% | 48.1% | 60% | 49% |
| Increased collection of irrigation service fee for the water used | 50% | 68.7% | 55% | 374% |
| Management Improvement Plans developed for WUAs | 0 | 44 | 44 | 100% |
Explanation of Results:
WUAs did not meet the targeted operations and maintenance cost recovery rates during implementation or after. Improvements were made, but the cost recovery rates leveled off after 2010. Cost recovery rates were not expected to improve enough to meet the target without increasing water charges.
Water user associations increased their irrigation service fee collection rates. As noted in the WtM evaluation, further discussed below, the sharp increases in service fee collection in 2009 and 2010 may have been linked to the GoA’s decision to provide free irrigation water in April and May of 2009 and 2010 in an effort to alleviate agricultural hardship during those years. Because water users’ service fee obligations were substantially lower during these years, the overall service fee collection rate increased despite an actual decrease in total revenues from water payments during this time period. However, the increase in service fee rates achieved in 2010 were sustained from 2011-2013. This suggests that water user associations were able to increase irrigation service fee rates materially with improved administration, and the increases were not solely due to the GoA’s decision to provide free irrigation water in some months.[[Mathematica’s Impact Evaluation of the Irrigation Infrastructure Activity in Armenia Final Report. August 2016.]]
Evaluation Findings
The Irrigated Agriculture Project was evaluated through multiple evaluations. An initial evaluation report was produced in 2013 to report on results from the WtM Activity. Then a second evaluation report was produced in 2016 covering the Infrastructure Activity and additional analysis on the related training and institutional strengthening interventions from the WtM Activity.
Water-to-Market Activity
| Component | Status |
|---|---|
| Baseline Report | Completed in 2008. Report is public. |
| Endline Report | Completed in 2013. Report and de-identified data are public. |
Irrigation Infrastructure Activity
| Component | Status |
|---|---|
| Baseline Report | Completed in 2010. Report is public. |
| Endline Report | Completed in 2016. Report and de-identified data are public. |
The program logic for the Irrigated Agriculture Project assumed that when farmers had reliable access to irrigated water; were trained on water management and productivity techniques; had access to credit, stronger post-harvest and marketing channels, improved irrigation delivery, and administrative management services provided by water user associations; they would apply techniques and investment that transitioned their farm operations from low-value to high-value agricultural production. This was expected to result in additional farm income from increased agricultural productivity and market access, which was assumed to lead to an increase in overall household income.
For each of the four components of the WtM Activity, the WtM Activity evaluation examined the following two broad sets of questions:
- How was the component implemented? What were the characteristics of each component’s participants, and how were these participants identified and recruited? What assistance was provided to participants through the component?
- What were the impacts of the component? What were the impacts on practices or use of new technologies as a result of the component? What were the impacts on key outcomes such as household income and poverty?
The Irrigation Infrastructure Activity evaluation examined the following research questions:
- Did the program affect the quantity and reliability of irrigation water provided to Armenian farmers?
- Did farmers adopt new agricultural practices as a result of the program? Further, is there evidence that farmers who received Water-to-Market training postponed adopting new agricultural practices until after the irrigation infrastructure had been rehabilitated?
- Did the program affect agricultural productivity?
- Did the program improve household well-being for farmers served by the rehabilitated infrastructure, especially income and poverty?
- Is there evidence that the infrastructure investments will be sustained after rehabilitation was complete? Are the water user associations themselves financially sustainable?
- Were the program effects large enough to justify its costs?
The WtM evaluation report found that the Farmer Training component did not result in an increase of on-farm water management practices, transition to high-value crops, productive income, or household income. Some positive impacts were detected on simple high-value practices such as soil preparation and purchase of pesticides from a licensed store. The Infrastructure Activity was significantly delayed during the compact period, and the farmer training implementation and initial evaluation continued without the new and improved infrastructure (a projected amount of 9,000 new hectares under irrigation and 38,000 hectares of improved irrigation). This was a fundamental breakdown in the program logic and a key lesson learned for MCC.
For the Credit component, the independent performance evaluation detected potential effects on adoption, crop production, and household income. WtM credit recipients were more likely to make agricultural investments, had higher production, and realized larger incomes. However, the evaluation has some important limitations (like not accounting for unobserved differences between treatment and comparison groups, such as motivation), which means that the results are likely upwardly biased.
For the Post-Harvest, Processing, and Marketing component, the independent performance evaluation found that the majority of recipients of this assistance reported positive outcomes, including improved product and service quality, increased productivity, and increased sales. However, the evaluation has some important limitations: all results are ex-post results reported by assisted enterprises and the results do not take into account what would have happened without the assistance, nor do they take into account any baseline information.
For the Strengthening Irrigation System Entities component, the independent performance evaluation reports that membership rates and membership fee payment rates increased moderately during implementation. In addition, WUAs improved their financial standing, but they were not yet approaching financial self-sufficiency. These improvements cannot be attributed fully to the component, because many other factors affect these outcomes, such as weather and national policies. For additional information on the WtM evaluation, see the Evaluation Brief.
The Infrastructure Activity evaluation found that while farmers near tertiary canals perceived improvements in the timeliness and reliability of irrigation water, others near rehabilitated large infrastructure did not. There was also no evidence that farmers irrigated more land, that production of HVA crops or their yields increased, or that household income and consumption increased. For additional information on the Irrigation Activity evaluation, see the Evaluation Brief.
The results from the WtM Activity training evaluation were disappointing; however, data was collected in 2013 before all of the irrigation system improvements were completed. Therefore, the 2016 Infrastructure Activity evaluation revisited the training and institutional strengthening components. For communities that received WtM training in 2008, the evaluation did not find that adoption of on-farm water management practices was substantially higher for farmers in 2013 than it was in 2010. The levels of adoption of profitable new practices remained low. In addition, WUAs’ financial status had leveled off since 2010, with little additional progress toward financial self-sustainability.
Evaluation Learning[[The full MCC Learning documents, which detail the motivation for the lesson and how MCC plans to apply it, are found in MCC’s Evaluation Catalog. These lessons were developed at the time of the evaluation publication and are framed within this context.]]:
Lessons learned from the WtM Activity evaluation included:
- Always return to the program logic. It is especially important in integrated projects that the rollout is coordinated with complementary activities.
- Balance ambitious targets with training effectiveness.
- The randomized roll-out evaluation approach has risks.
- A multifaceted development approach requires proactive and visionary management.
Lessons learned from the Irrigation Infrastructure Activity evaluation included:
- Root cause analysis is critical to successful project design, especially where behavior change is integral to the program logic.
- Supporting the development of new institutions, such as WUAs, is inherently difficult and can be a long-term undertaking, so MCC projects should account for the five-year timeline and anticipate necessary follow-on activities.
- During a project re-scoping, the program logic, economic analysis, potential beneficiaries, and evaluation plan should be re-assessed in a cohesive way by a coordinated project team.
- Better water monitoring tools could help WUAs while also providing better measurement of outcomes.
- An irrigation project where demand for the intervention exceeds available funding can provide an opportunity for random assignment (or other allocation mechanisms that are fair, efficient and informative) that can improve the level of rigor of evaluation results.
- Sector-specific technical capabilities should be required on the evaluation team when needed to assess key intermediate outcomes.
- Evaluation questions are based on the program logic and must be designed carefully from the beginning to understand the scope and limitations of the evaluation.