Cost Benefit Analysis Guidelines

List of Acronyms

• BAU – Business As Usual
• CBA – Cost Benefit Analysis
• DCLG – Department for Communities and Local Government
• EA – Economic Analysis
• EIF – Entry Into Force
• ERR – Economic Rate of Return
• HSPH – Harvard School of Public Health
• IMC – Investment Management Committee
• IO – Input-Output
• LCU – Local Currency Units
• MBO – Management by Objectives
• MCA – Millennium Challenge Account
• MCA – Multi-Criteria Analysis
• MCC – Millennium Challenge Corporation
• M&E – Monitoring and Evaluation
• NPV – Net Present Value
• O&M – Operations and Maintenance
• PIR – Policy and Institutional Reform
• PV – Present Value
• RCA – Root Cause Analysis
• SCDPs – Sector Consistent Design Patterns documents
• UA – Uncertainty Assessment
• USD – United States Dollars
• VA – Value Added
• VSL – Value of a Statistical Life
• VSLY –Value of a Statistical Life Year
• WASH – Water, Sanitation and Hygiene
• WTP – Willingness To Pay

Introduction

The Millennium Challenge Corporation (MCC) was established in January 2004 to “(1) … provide United States assistance for global development … and (2) to provide such assistance in a manner that promotes economic growth and the elimination of extreme poverty and strengthens good governance, economic freedom and investments in people.” (Millennium Challenge Act of 2003 as amended). MCC has interpreted its legislative mandate in light of the strong empirical relationship between economic growth, economic inclusion, and income gains of the poor, adopting the mission statement: Poverty Reduction through Growth.[[In embracing ‘Poverty reduction through growth’ MCC recognizes the critical role of economic growth in poverty reduction, particularly over the long term. It also recognizes the need to orient growth toward those productive activities that are most inclusive, including for women, the poor, and socially excluded minorities.]]

MCC seeks to work with partner governments to promote growth and poverty reduction by focusing its efforts on alleviating constraints to economic growth, carefully examining the drivers of these constraints, and designing appropriate, sustainable, and cost-effective interventions to alleviate them. Prior to signing any compact, MCC is required to provide to Congress an economic justification for funded projects, which includes the expected economic rate of return (ERR) from a cost benefit analysis (CBA). MCC’s investment criteria further require that compact projects be funded only if the estimated project ERR does not fall below MCC’s ERR threshold.[[See MCC’s Investment Criteria (internal document) and discussion below.]]

This document, the Cost Benefit Analysis Guidelines (hereafter: the CBA Guidelines), provides guidance to help MCC economists conduct cost benefit analysis as part of the compact development process, which includes a series of analytic products that collectively work to ensure investments are consistent with MCC’s mission statement. These CBA Guidelines are periodically updated and should be considered a ‘living document’.[[Updates will take place as needed and no more frequently than once a year. Some changes in the current 2021 update are: (i) the introduction of Decision Notes—Internal short notes that are used to introduce methodological changes; (ii) changes aimed at greater methodological consistency with standard practice in CBA as well as with sector-specific CBA guidance; (iii) designating CBA Guidelines as a ‘living document’ which are to be periodically updated; and (iv) introducing CBA topics with new guidelines in production for future inclusion. Specific new topics covered in the current 2021 update are: (i) labor income benefits; (ii) an introduction to CBAs for grant facilities and to the treatment of uncertainty in CBA; (iii) value of statistical life; (iv) salvage value treatment; (v) which ERR to use; and (vi) a policy and institutional reform section.]]

In addition to the CBA Guidelines, MCC also develops, publishes and periodically updates Sector CBA Design Principle documents (hereafter: SCDPs).[[The CBA Guidance for Land was finalized in June 2019. The CBA Guidance for Water, Sanitation and Hygiene was finalized in March 2019. Internal publication of the Guidance for the Transportation Sector is scheduled for Summer 2021. The Guidance document for the power sector is in draft form. In addition to sector guidance, EA issued in 2018 an internal draft of Economic Analysis of Policy and Institutional Reform Programs: “Position Paper and Recommendations for MCC management”.]] The SCDPs are intended to be the primary point of reference for economists designing CBAs in a given sector. Decision Notes are produced as an additional resource, mainly as a means to document methodological or procedural changes which may have not yet been integrated into in to SCDPs or this document. The CBA Guidelines, in turn, collect and summarize recommendations from SCDPs and from the Decision Notes in addition to offering general overarching guidance when appropriate. 

CBA Guidelines do not provide a history of CBA practice at MCC, nor do they attempt to analyze or draw lessons from historical CBA practice at MCC or other institutions.  The CBA Guidelines are not exhaustive but rather provide a listing and summary of guidance accumulated as methodological decisions are made and topical notes are developed in response to ongoing discussions and emerging issues.

The remainder of this section will discuss the context of economic analysis at MCC under which Cost Benefit Analysis takes place. Section II describes how to conduct Cost Benefits Analysis starting with the theory of change, followed by how to identify benefit streams and ending with guidelines for cost estimation. Section II also includes a special section on CBA for grant facilities and on Policy and Institutional Reform.  Section III covers reporting and publishing CBA results.

Cost Benefit Analysis in the Context of Economic Analysis at MCC. In 2015, MCC Investment Criteria formalized the practice, dating back to 2010, that MCC programs aim to alleviate a root cause of a binding constraint to economic growth in the respective partner country.[[MCC’s Investment Criteria do not currently contemplate multilateral compacts. In 2018 MCC received congressional approval to engage in concurrent compacts (Public Law No. 115-167). Since then, when working with multiple country partners on regional programs, MCC has sought to address a root cause of an economically significant obstacle to growth through regional integration (see the “Preliminary Guide to Developing Regional Integration Compacts”, MCC Internal Document).]] To operationalize this objective, MCC collaborates with host country representatives to conduct a Constraints Analysis and a Root Cause Analysis. The Constraints Analysis (CA) is a data-driven, diagnostic exercise which helps to identify the binding constraints to economic growth in a given country.[[Detailed guidance on Constraints Analyses can be found in the MCC document, Guidelines for Conducting a Constraints Analysis, a chapter of the larger Compact Development Guidance document. The Constraints Analysis also seeks to identify barriers to the inclusion of key groups, such as women and girls, in receiving the benefits of growth or potential MCC programs and is informed by an analysis of high potential and inclusive sectors of growth within an economy. (See https://www.mcc.gov/our-impact/constraints-analysis).]] The Root Cause Analysis (RCA) refines understanding of the nature and underlying factors that cause and perpetuate the binding constraint, including those which lead to exclusion/inclusion of women and social minorities.[[The root cause analysis is used to inform a “problem statement” for compact and threshold programs to address. Country Teams discuss alternative ways to address the identified problem during this stage. The “Guidance on Root Cause Analysis will be updated at the end of 2021.]] As the root causes of the binding constraints are identified, projects aimed at helping alleviate the binding constraints are proposed. A Cost Benefit Analysis (CBA) then models the economic logic of a proposed project in quantitative terms. Specifically, the CBA estimates economic benefits that can be directly attributed to the proposed project and compares them to the project’s costs over a set period of time. 

A CBA is typically summarized with metrics that facilitate comparison across projects. MCC uses the economic rate of return (ERR) as an investment criterion that can be used to compare projects that may vary significantly in size.[[Another metric is the Net Present Value (NPV).  When examining projects of similar size, or considering design changes to a single project, NPV gives a more relevant indicator, highlighting differences in value that may not be apparent from ERR changes alone (e.g. single digit changes in ERR can correspond to tens of even hundreds of million worth of benefits).]] The ERR captures the rate at which economic resources invested in a project deliver economic benefits to society. In 2015, MCC Investment Criteria formalized an ERR threshold of ten percent, requiring funded projects to have ERRs equal to or surpassing this rate.[[Transparency requires that all calculated ERRs are reported in MCC’s investment decision document, the Investment Memo. The section on “Reporting and Documenting” in these guidelines list ERRs that are reported at different points in MCC programs. In MCC’s early years, the ERR threshold was country-specific and related to the country’s recent average economic growth rate. MCC currently has a 10 percent standard hurdle rate to simplify decision-making, support program cost-effectiveness and increase accountability for tax-payer funds. This historically was in broad conformity with other development institutions. MCC is currently discussing whether this hurdle rate should be adjusted going forward. A lower hurdle rate is advisable if the analyst takes a conservative estimate of future benefits or the project has plausible benefits beyond an end period. The period over which benefits are typically estimated is 20 years, long enough to accommodate most MCC investments, and broadly in line with time horizons used by other development institutions. However, it is up to the economist to adjust the time beyond the default time horizon if the nature of the investment warrants it.]] This threshold is also referred to as the “hurdle rate.” MCC Investment Criteria states that “ERRs must be calculated at the lowest possible level of disaggregation in accordance with the program logic when is feasible and cost effective (for MCC resources) to do so”.[[In “MCC Investment Criteria, Framework and Definitions”, p. 7.]]

Definition of Projects. The program logic determines what constitutes an analytical project. The term “project” in the context of CBA discussions typically denotes the item or items being analyzed – i.e., the lowest level at which it makes sense to analyze a group of elements given their level of complementarity or joint necessity.[[See Grouping Complementary Elements for Analysis in the following section.]] Economists will report ERRs at the lowest level of disaggregation that is feasible and logical, with the understanding that the ERR threshold refers to the project level ERR.  Hence, the project for this purpose may be an aggregation of logically separable program elements.  Such aggregation for the purposes of applying the ERR threshold is allowable if elements are either:

• dissimilar but jointly necessary to deliver the benefits intended (logically one “project”); for example, a road-building project would include a maintenance policy;
• a group of small items for which the available evidence of impact pertains to or a similar grouping of items which share a targeted outcome; or
• are logically separable but share the same program logic; for example, a set of electrical distribution lines in a given peri-urban zone, technologically similar irrigation systems for separable land perimeters, construction of multiple schools, or investments in multiple roads may all be analyzed separately but then aggregated to the “project” level.

This definition of “project” is distinct from that intended by a group of activities MCC designates as a “Project” for program management and communication purposes. MCC Compacts’ administrative hierarchy organize components in the following order (decreasing in size): Compact, Project, Activity and Sub-Activity.[[This packaging may not correspond to the projects defined for the purposes of calculating economic criterion for investment and as used in this Guidance. MCC has not reconciled the administrative and economic terms and the levels of ERR reporting.]] For example, an MCC Tourism Project may contain differentiated activities to improve marketing efforts, improve the experience of wildlife viewing, provide access routes, and foster greater competition among hoteliers.  A power sector “access” project may contain distribution lines in peri-urban areas, solar panels in other rural areas, and support for financing connections in the entire network. In both cases, each general element (whether called an “Activity” or a “Sub-Activity”) should be separately assessed against the ERR threshold.[[In the case of activities or sub-activities that do not pass the hurdle rate but are considered of critical importance for compact or project outcomes, the Investment Criteria require that a full and transparent record of the rationale and factors leading to this exceptional treatment be included as part of the public record.  For example, high sunk costs and a need to modify project design could lead to a re-optimized allocation of funds across activities.  Another example may derive from a critical need to include an exceptionally poor or vulnerable population group within the project scope for the purpose of social equity. Households may be unable to, for example, pay full cost recovery tariffs for utility services (typically in rural or low-income peri-urban areas).  The funding of a sustainable and equitable expansion of basic services may require subsidies that may increase fiscal sustainability risks. These cases, when there are no other reasonable options, require management approval insofar as “Management, through the IMC, is responsible for holding MCC Country Teams accountable to these Required Criteria.” The IMC provides recommendation to the CEO who takes the final decision before submission to the Board for final approval.]]

All compact projects are subject to the economic analysis described in this section and, under MCC Investment Criteria, should have an estimated ex-ante ERR above the ten percent threshold rate to be considered for investment approval by the MCC’s Investment Management Committee (IMC). In addition to the ERR required in advance of investment decisions (the “ex-ante” ERR), MCC also estimates ERRs with updated project information at closeout. These are called “closeout” ERRs.[[MCC is committed to publishing both ex-ante CBAs and closeout CBAs. In addition, MCC has published three reports on closeout ERRs. See also the corresponding blog article.]] A closeout ERR is updated to cover actual costs at closeout, but still covers mainly projected benefits. Sometimes benefits are also updated in closeout ERRs. For example, changes in costs due to a change in scale or scope of a project may have caused associated changes in projected benefits. In addition to closeout ERRs, MCC is increasingly also conducting or commissioning “evaluation-based” ERRs that reflect post-compact evaluation evidence.[[These ERRs are estimated by external evaluators, while the original and closeout ERRs are estimated by MCC staff.]] These evaluation-based ERRs allow for the incorporation of evidence on benefits gathered after project completion.

MCC’s Investment Criteria require that the ex-ante ERR of a funded compact project does not fall below the ten-percent threshold. However, this criterion does not require that the project with the highest ERR is funded. Rather, the criteria allow selection of investments with lower ERRs than alternative proposed projects when such investments achieve other goals reflective of MCC’s mission and congressional mandate, as reflected in the Investment Criteria. Such could be the case for projects that may improve the distribution of benefits to more effectively reduce poverty, or projects that are expected to have significant inclusive growth enhancing effects, but which are not captured in the benefit streams producing the ERR.

In contrast to the requirements for Compact Programs, the calculation and reporting of ERRs for projects under Threshold Programs is not currently required but is optional. The rationale for this is based on the preponderance of support in these programs being directed towards supporting policy and institutional reform (PIR), with the expectation that these programs will help the countries to become eligible for future compact support. PIR interventions have historically been more difficult to model and apply robust CBA practices to. However, for Compact Programs, MCC is now committed to modeling all interventions with distinct program logic, including PIR interventions.

Conducting Cost Benefit Analysis and Calculating ERRs

Conducting a CBA requires a number of steps and analytical considerations. These are summarized in this section and explored in more detail in the SCDPs. This section provides guidance to economists on: (i) establishing the theory of change and grouping program elements to conform with the logic(s); (ii) identifying the expected benefit streams; (iii) estimating costs; (iv) establishing outcome trends both with and without the project (i.e., the counterfactual);[[Both trends are of course uncertain, and analysts are encouraged to consider those uncertainties, while bearing in mind that the primary effect of interest is the difference between with and without project outcomes, under otherwise harmonized assumptions. The economist should note how the space of feasible project alternatives was defined and how the specific project was selected. MCC’s current practice, due to resource limitations, does not fully identify alternative scenarios beyond the counterfactual. Only in very specific circumstances have alternative projects been considered and costed. The economist should consider including alternative parameters that are considered and costed in the counterfactual exercise.]] (v) important analytical considerations including the use of evidence to quantify impacts, the time horizon and discount rate and the treatment of uncertainty; and finally, (vi) special guidance on the use of CBA for grant facilities and political and institutional reform (PIR) projects.

Establishing the Theory of Change

The estimation of the economic impacts of any intervention requires formulating a theory of change, also referred to as a program logic. The theory of change should agree—insofar as possible—with the state of theory and evidence on how economies work and how institutions, firms, and people behave. Many projects rely on behavior change by individuals or firms. Examples of this might include investment in irrigated agriculture, procurement actions, utilization of community rangeland, or teachers’ absenteeism. In such cases, an economic analysis should ideally be performed from these agents’ perspectives to confirm that they have an incentive to change their behavior in a sustained manner and this analysis should account for the opportunity costs of time and effort for these agents. Without confirming this incentive compatibility, the analysis should not assume that the behavior change occurs.[[Human behavior responds to a multitude of factors including financial factors, cultural norms, individual values and psychological rewards. If the economic incentives are severely misaligned with desired behavior change, it is unlikely that change will occur, at least in a lasting manner. The economist together with the Country Team should confirm whether the incentive compatibility exists for the behavior change to occur and inform program design accordingly.]]

The program logic should emanate from a clear and robust root cause analysis (RCA), which includes examination and identification of whether market failures and government failures are at work, how they affect the without-project scenario, and whether the proposed solutions would likely be appropriate and effective. The proposal of solutions requires a decision regarding which activities, sub-activities, and/or their elements will be combined for analytical purposes, and therefore considered a logically coherent project, i.e., a project determined by a program logic.

Grouping Complementary Elements for Analysis. Each MCC CBA requires a decision regarding which activities, sub-activities, and/or elements thereof will be combined for analytical purposes. Although program elements are often designed to jointly produce a result, a contention that all elements are necessary must be critically assessed. In principle many investment decisions are separable, so analysts should disaggregate the analysis by element(s) insofar as feasible and suggested by the program logic. In addition, the risk of having a single CBA covering all activities in a Project is that activities expected to perform poorly can be hidden behind an aggregated ERR for the entire Project.[[As noted elsewhere, economists should report all calculated ERRs in MCC’s investment decision document, the Investment Memo (see Footnote 9).]]

When program elements are highly complementary, they should be combined for the purposes of the analysis, i.e., if benefits may not accrue from one project component alone without the other investment(s). The economist ascertains relevant complementarities and provides the rationale for combining program elements. For example, a greenfield power transmission line, absent accompanying investments in generation capacity and distribution, may provide no benefits. In this case, the costs of required complementary investments should be included in any CBA which includes those joint benefits, regardless of whether MCC funds cover those costs. A comparison of accrued benefits with and without complementary investments may help to assess the degree of complementarity between the program elements.

In other cases, elements may be theoretically complementary on some level, but their combination is not strictly necessary to achieve the hypothesized benefits. For example, rural roads and irrigation systems have incremental benefits on their own with little or no complementarity, and their ERRs should be reported separately. In conducting separate analyses, the analyst may assume that the other complementary elements are implemented, if that is considered likely, or compute ERRs under a variety of scenarios with regards to complementary interventions potentially undertaken by MCC and others. In situations where elements are logically separable, but the existing evidence of impact pertains only to a similar bundle of interventions, it may only be feasible to conduct a combined analysis. In this case, while the ERR may need to be computed for this typical bundle of interventions, MCC should consider evaluating the project elements separately to add to the evidence base on which elements are most cost-effective.

Market and Government Failures. Interventions and investment to support growth are often designed to correct or remove market and government failures. Developing a clear project logic to do so requires a clear understanding of the nature of these failures and how they distort outcomes.

Market failures exist in situations where the free market fails to allocate resources efficiently in an economy – that is, there is an alternative feasible allocation that improves welfare for one person without lowering it for anyone else (a Pareto improvement). A government or policy failure occurs when there is an alternative government policy that could lead to a Pareto improvement. Governments may set out to correct market failures and/or achieve social objectives (e.g. equity, democratic representation, national security, cultural preservation, etc.) but a government intervention may be or become a policy failure.

Understanding which failure is present is fundamental to designing an effective intervention and to conducting economic analysis of proposed interventions.

MCC economists assess market and government failures identified through the Constraints Analysis or the Root Cause Analysis in order to evaluate whether the proposed activities address the failures. An understanding of relevant market and government failures is critical to improving the overall impacts and distributional impacts of MCC investments affected by these issues. Program design should be informed by an evidence-based assessment that identifies failures at play as well as their impact and potential means for addressing the failures.[[The economist should attempt to bring evidence to bear on distributional impacts to support the beneficiary analysis (e.g., growth or fiscal incidence curves, administrative data).]]

There are several types of market failures worth distinguishing when assessing an intervention’s economic justification and identifying appropriate solutions:

Market Power: Market power involves firms’ ability to affect the price of their output or inputs (including labor) by changing the quantity they supply or the input levels they demand. When firms utilize market power, market outcomes are not efficient. There is growing appreciation of the negative impact of excessive market power on growth in developing countries, as it can impede the replacement of inefficient firms by more productive ones. It can also drive up the costs of inputs in ways that are detrimental to an economy’s international competitiveness. Natural monopolies (wherein economies of scale are high relative to market size) also exist in some markets, potentially requiring public intervention to ensure that service providers in those markets set the supply and price of services to be more consistent with the social optimum.

Externalities / spillovers: Externalities exist in situations where the effect of production or consumption of goods and services imposes costs or benefits on others which are not reflected in the prices charged for the goods and services being provided. Consequently, with a positive externality, the production and consumption of a good or service is below the optimal social level; with a negative externality, production is above the optimal level—reducing social welfare in either case. Public interventions to expand/reduce production can thus serve to improve social outcomes.

Public Goods: A public good is differentiated from a private good on the basis of two characteristics:

• Non-rivalry:This means that when a good is consumed, it does not reduce the amount available for others to consume. For example, benefiting from a radio station or national defense does not reduce the radio waves or security available for others. By contrast, consuming scarce water services at a given point in time and space and reduces the stock of water available for others to consume in that time and place.
• Non-excludability: This occurs when it is not possible to provide a good without it being possible for others to enjoy it as well. An example of a local public good is the construction of a dam to stop flooding: The dam provides protection for everyone in the area, not just those who contributed to its financing.

In some cases, a good may have some but not all of these characteristics. Important examples include common pool resources and club goods. Public goods are generally under-provided because these characteristics of non-rivalry and non-excludability mean that consumers of such goods can consume them without paying (the “free rider” problem), and firms providing the goods have difficulty charging for their use to recover costs.[[This supports an argument for government intervention to protect public resources (the ‘commons’) such as air quality, land use, water supplies, and underpins the need for supranational authority to protect global commons (law of the sea, cross boarder watershed management, carbon emissions, etc.).]]

Imperfect information / information asymmetry: An informational asymmetry occur when one party to an economic transaction possesses greater material knowledge of the quality, attributes, or precise terms of the good or service or other value exchanged than the other party. Information asymmetries particularly impede transactions in credit and insurance markets, but also in labor, health care, and some product markets.

Coordination Failures: These market failures occur when parties are unable to coordinate complementary actions or investments, resulting in a socially inefficient or otherwise suboptimal outcome. For example, there may be two complementary investments that are each only profitable if the other takes place, but because investors do not coordinate, neither takes place. Coordination failures can impede private investment in large (non-public good) infrastructure assets such as a transmission or a rail line.[[Coordination failures can also produce socially inferior equilibria in arenas such as maintaining peace, arms control, trading rules, and/or regional cooperation agreements.]]

Government failures can entail omissions of key underpinnings of an efficient market economy; for example, incomplete protection of property rights or insufficient macroeconomic stability. Areas of government failure include excessive or inappropriate regulatory oversight, market distortions through tariffs and subsidies and the granting of economic monopolies. For example, the government may enact restrictions on market entry that reduce competition and productivity. Similarly, a government’s funding and creation of state-owned enterprises may reduce market efficiency if these enterprises become rent-seeking entities, operate under differential rules or become corrupt.

While MCC investments may be able to directly address or compensate for identified market failures during compact implementation, policy and institutional reforms (PIR) are generally required for impacts to last beyond this five-year period; further, other types of market failures may only be addressable with PIR. Modeling the costs and benefits of PIR reforms typically requires greater reliance on rough estimates of parameters governing program benefits and these benefits are more subject to uncertainty with regard to completion and sustainability risks. These issues are discussed further later in this section in the special guidance on CBA and Policy and Institutional Reform.

Identifying an Expected Benefit Stream

Once there is an understanding of the root causes of the constraint a project proposes to address and a coherent program logic, economists model the linkages through which the project activities are hypothesized to increase welfare in the host country, estimate the parameters necessary to quantify benefit streams, and characterize the likely distribution to beneficiaries of the project.[[MCC’s CBA practice has been to count benefits accruing in the specific partner country for which the compact is being developed, to preserve a focus on poverty reduction and growth in that country. Simultaneously, in the interest of economic efficiency, it attempts to count all costs borne, regardless of what entity or country bears them. Draft principles currently under review (“Principles for treatment of environmental externalities in MCC cost-benefit analysis, with attention to greenhouse gases”, March 2021) clarify the implication of these positions in more detail with respect to environmental externalities and lay out additional practical considerations. These considerations focus on balancing an emphasis on partner country benefits with interest in opportunities to cost-effectively improve regional or global public goods related to climate regulation that also significantly impact those living in low-income countries (via, e.g., greenhouse gas mitigation). In many instances, other guidance and criteria relevant to project development should prevent significant externalities, but as of writing EA does not have confidence that this will always be the case, due to degrees of enforcement, conceptions of significance, and conceptions of counterfactuals.]] MCC’s approach can include the value of longer, more productive lives, the value of home production, such as child care and domestic services, and changes in future welfare associated with a country’s natural assets.[[EA adopted a value of statistical life approach to measure lives saved in a March 2020 VSL Decision Note. The VSL approach replaced the national income measure, the disability adjusted life years (DALYs) valuation. EA’s practice with respect to increased consumption of leisure is varied. While time savings is measured as a benefit, EA has opted to account for time savings only insofar as there is a link to a productive use of the time saved, i.e. the time saved is substituted for a marketable activity or to increase human capital.]] Local benefits of improved use or preservation of natural capital should be included if credible evidence on their impact is available.

This section discusses: techniques used for benefit measurement; treatment of cost savings; treatment of direct, indirect and induced benefits; and the characterization of beneficiaries.[[There is additional and more detailed guidance of relevance to identifying benefits in the SCDPs and in the first two annexes to this document. See Annex 1 for the discussion on the Value of a Statistical Life discussion. See Annex 2 for the discussion on Labor Income Benefits.]]

Benefit Measurement. MCC conforms with standard CBA techniques by using a welfare-based approach. The focus of welfare-based techniques is on approximating the measurement of equivalent variation, including willingness-to-pay under partial equilibrium (i.e. changes in consumer surplus).[[In standard welfare theory, increases in money-metric utility from improved and expanded consumption choices are measured as equivalent real income gains (i.e., using compensating or equivalent variation). A consumer surplus approach and a value-added approach can approximate these real income gains, albeit under different assumptions—the former restricts itself to behavioral changes in or two markets in partial equilibrium while the latter tries to measure real income gains directly while largely ignoring behavioral responses. Thus, these techniques are both valid under appropriate conditions.]] The consumer surplus approach is standard in CBA because it is simpler than other approaches and, in contrast to other approaches, the sources of error are bounded, typically smaller, and well understood.[[The May 2020 Technical Note on Consumer Surplus in Cost Benefit Analysis expands on the advantages of using consumer surplus, including to measure behavioral response. Consumer surplus in a given market is the total difference between consumers’ willingness to pay and the price consumers pay. A producer surplus approach is equivalent to a consumer surplus approach, except that the profit function is fully linear. EA affirmed consumer surplus as the preferred method in a July 2020 Consumer Surplus and Value Added in CBA Decision Note.]] For a development project that improves provision of a good or service, consumer surplus provides a lower bound on the aggregate benefit from increased consumption of that good or service while accounting implicitly for related changes in consumption of complements or substitutes.[[The forthcoming transportation and power SCDPs provide more specific examples. The transportation SCDP includes the roads sector as well as other modes of transportation.]]

EA publishes Sector CBA Design Principles documents (SCDPs) that are updated periodically. These guidance documents distill in some detail the key benefit streams for projects within each given sector and are intended to be the primary guidance for designing CBAs in that sector.

Cost Savings. In both Consumer Surplus and Value-Added approaches, benefits accruing as cost savings to the economy’s producers and consumers (including governments) would be included as a benefit stream. In particular, under the Value-Added approach, cost savings for consumers are included at without-project levels of consumption. A consumer surplus approach would include these cost savings, along with the consumer surplus benefit of substitution towards the good provided (and away from consumption of other goods) at the same level of nominal income.

Direct, Indirect and Induced Benefits. When a project is believed to generate multiple benefits, it is important to differentiate between direct, indirect and induced benefits. This is especially critical in a value-added approach, but these considerations are also relevant for a consumer surplus approach in some cases.[[For example, in a consumer surplus approach, some induced benefits are already included, but there may be additional induced benefits that could be appropriate to include—however, doing so requires an understanding of why these benefits are not being internalized by decision-makers. This may have to do with market failures or behavioral phenomena.]]

Direct effects arise directly in sectors or markets MCC invests in (e.g. jobs and salaries paid by the receiving sector). Typically, the direct benefit to users or consumers of the expanded supply (e.g. water, power, port services etc.) is greater domestic consumption at lower cost, or surplus available for export. Firms eventually expand their use of goods and services until profit increases commensurate with their cost savings are exhausted. Such benefits are included as part of standard cost benefit analysis.

Indirect effects arise through a value chain effect as the project stimulates greater demand for input of intermediary products and services, with concomitant benefits to incomes in the supplying firms or industries (and potentially further up the chain to those firms’ suppliers). Scale economies and coordination may also be considered. Note that for these effects to be additional to those already captured through measurement of direct benefits above, there needs to be a distortion in the supplying market, which could include the labor distortions discussed above (See Boardman et al. 2018). The economist will assess the strength of the evidence supporting the existence and extent of a distortion that would merit the inclusion of additional benefits—not already included in the standard cost benefit analysis.[[The EA division is considering the use of a more structured approach to the estimation of additional indirect and induced benefits in cases where the evidence strength supports their inclusion. One example is the macro-simulation approach currently undertaken by the IFC.]]

Induced effects arise from expenditures of households enabled by the increasing incomes generated through the direct and indirect effects. For example, a transport or electricity project could cause an expansion of investment, trade, and innovation; more firms could enter, existing firms could invest more or in different activities, and this increased activity will shift demand for transport services. Such general equilibrium effects can be modeled (including for innovation and diversification), depending upon the context and likely magnitude of impact, as long as the evidence and economic logic are sound. These need to be separated out due to concerns about potential “double-counting.” Generally, such impacts may be difficult to measure and indistinguishable between projects.

Policy and institutional reforms to reduce significant distortions in the national economy (trade liberalization, labor market reforms) may be an exception, where induced effects on can be large and important to capture. In such cases it is may be valuable to model macroeconomic effects in the CBA, relying on credible empirical evidence for parameters producing the supply response (for example, from cross-country or panel estimates of the impacts of deregulation on GDP). Inclusion of such induced benefits or multipliers should be used cautiously and critically.

Characterizing Beneficiaries.[[Guidelines for Economic and Beneficiary Analysis (MCC’s internal document).]] MCC has historically considered beneficiaries of projects to be residents of the partner country who experience higher real long-term income as a result of the project.[[Or higher equivalent income, in the case of benefits with missing or incomplete markets.]] SCDPs and Decision Notes may provide more specific guidance and anyone who receives a benefit defined in a CBA model in accordance with such guidance is considered a beneficiary.

Estimating Costs

Estimating the costs of a proposed project involves capturing the monetized opportunity costs of all inputs required to deliver the associated benefits. Included are costs paid by MCC, costs paid by all other actors – donors, governments, and private entities, whether in the form of cash, in-kind contributions, time costs, and any natural asset use or depletion, regardless of whether the sources of funds is—or burdens are borne—inside or outside the compact country.[[Note that MCC does not pay taxes to partner country governments.]] Also included are a pro rata share of the cost of implementation (including expenditures associated with MCA and other administrative operations), as well as monitoring costs.[[Evaluation costs, which reflect institutional learning priorities beyond the scope of any single project, should not be considered in CBAs. In addition, EA’s practice is to exclude 609g funds and to include Compact Facilitation Funding (previously known as Compact Implementation Funding).]] When CBAs are being revised but investment designs are not sufficiently different (say, because of changes due to costs and scale only), economists should construct two CBAs: an Updated (or projected closeout) CBA, which includes “sunk” (non-recoverable) costs, and a CBA for investment decision making which excludes sunk costs (producing a “marginal” ERR).[[EA adopted this approach in a March 2020 Decision Note.]] To clarify the basis of MCC decision making for stakeholders while maintaining accountability for overall use of funds, both CBAs and their results are reported and explained in STAR Reports. When project development results in more fundamental changes to initial plans (e.g. requiring new designs), the marginal ERR should still be used to guide decision-making.[[Expenditures incurred on discarded investments but that are not directly related to the project for which the CBA is being calculated should not be included in that project’s CBA. The Country Team should exercise judgment, however, on what costs are attributable to the project.] All compact costs that are directly attributable to the projects should be accounted for across the existing set of CBAs, including costs associated with investments which are separable from all others but for which a CBA was never constructed.

When calculating the opportunity costs of using productive resources, such as labor, land and natural capital, these resources should be valued according to their most likely or feasible alternative use.[[Economists’ default practice is to use the prevailing market rate.]] Where market distortions are significant or markets are missing, the economic analysis should attempt to use and estimate shadow prices. For example, water or forest users may not fully internalize the economic value of scarce water or other resources, in which case the shadow value is higher than the market price.[[See sector guidelines for estimation of prices in the absence of market distortions.]]

Some of the more difficult costs to estimate may be those related to policy and institutional reforms, discussed above. In addition to the budgeted costs to support reforms, there will also be labor costs from the reform efforts, adjustment costs as resources are reallocated, and/or compliance costs. Expert views can be used to inform ex ante estimates of costs that MCC would not finance, and if possible, these costs should be tracked (using the Monitoring & Evaluation Plan) during implementation for closeout ERR and learning purposes.

Cost impact of real exchange fluctuations. MCC partner countries may experience large exchange rate changes during compact implementation which can affect the relative price of imported commodities or equipment and the grant value of MCC assistance in local currency terms. Imported irrigation equipment, for example, will be relatively more costly compared to local incomes after a currency devaluation. The “Denominating Costs and Benefits” topic in the Reporting and Publication section will review the denomination of costs under exchange rate fluctuations in more detail.

Establishing the Counterfactual

Establishing the appropriate counterfactual for a project (i.e., the “without project scenario”) requires an understanding of the country and sector context, trends, the market or government failure(s) involved, as well as any existing or planned efforts to resolve the constraint. Both the with-project and the counterfactual scenarios should be as realistic as possible; in the latter case, capturing what would happen in the absence of the proposed program elements.For example, without a proposed investment in road maintenance, the condition of the road would decline as modeled using engineering and usage parameters. Without a successful power sector reform, operational and maintenance performance and system expansion would likely remain similar to current levels. Similarly, without conservation measures, groundwater supplies would be depleted according to current per capita usage levels, projected population growth, and climate change.

In addition, the appropriate counterfactual depends upon the nature of the intervention and whether it responds to a market or a government failure. For example, if a proposed MCC project is designed to finance private good(s) – i.e., goods and services that would promise some prospect of profitability for their supplier – then it is likely that some measure of private investment would occur in the counterfactual, and a continuation of recent investment trends might be a reasonable assumption. If provision of the given private good involves unmitigated risk, appreciable technology spillovers, information asymmetries, or likely retaliation by a dominant incumbent, there may be some investment in the counterfactual, but not at the socially optimal level. Conversely, if the item being provided is a pure public good, such as an efficient judicial system, physical security, or clean air, it is generally safe to assume very low provision in the counterfactual.

Fungibility and alternative project financing means. In the absence of MCC financing, it is possible that another public entity, or another donor, would finance the project being assessed. Resources are fungible to some degree and a given project financed by MCC may serve to create fiscal space for other public projects. While the possibility of alternative public financing can have real world implications, it is not relevant for a cost benefit analysis. CBA ascertains whether a proposed public expenditure, action, or policy represents an efficient use of public resources or a net improvement in social welfare. MCC conducts CBA of the items to be financed, and the counterfactual that MCC uses is that the items or inputs provided under the project will not otherwise be provided by the public sector.

Similarly, in the case of investments in processes to spur or implement policy and institutional reform (PIR), the main question is not: in the absence of MCC would this reform have taken place? Rather, the question is, in the absence of the items financed (including convening authority, conditions precedent, leverage, monitoring of outputs, and other more intangible inputs such as technical assistance, systems, and related assets), what would the policy and institutional outcomes be? This formulation maintains a logically consistent counterfactual to use across project types, whether projects involve tangible assets or more intangible investments. It also renders it unnecessary for the analyst to quantify the likelihood of each intervention’s obtaining alternative public support in the absence of MCC’s investment, a task which is difficult to do robustly.

Analytical Considerations

Evidence to Quantify Impacts

To the extent possible, MCC bases its estimation of benefits on the strength of the program logic and the available evidence supporting the assumptions the program logic is based on. This requires technical inputs from sector experts and studies (including feasibility studies where appropriate), a review of any existing evidence of impact, data on current outcomes, and valuation data that MCC may collect as needed.[[Economists need to be integrally involved in drafting scopes of work for consultants involved in due diligence and design to ensure that the requisite information is available for economic analysis.]] Preferably, evidence of a project’s expected impact is drawn from evaluations of similar completed interventions and projects in the relevant country, provided the evaluations have been validated. When such evidence is not available, as is often the case, results from other settings or another country with similar economic conditions may be applicable if these results are validated externally, supported by credible arguments and adjusted for contextual differences. In the absence of precise evidence, if the program logic is strong and there is adequate baseline data (e.g., at least validated qualitative results from similar interventions of impact on the targeted outcome(s)), MCC may construct a model that utilizes defensible assumptions to produce an expected ERR and then test the sensitivity of results to key but uncertain parameters. Such an exercise should clearly signal the risks associated with increased uncertainty due to less precise evidence. However, MCC will not approve proposals without sufficient supporting evidence that the project will have a significant impact on economic growth and poverty reduction.

Time Horizon and Discount Rate

MCC’s current usage of the ERR and NPV decision metrics implies the use of a fixed discount rate to discount future benefit flows over time. This places a higher value on nearer term benefits at a constant ratio from year to year. The rational for the specific hurdle rate of ten percent—which implies a fixed discount rate of ten percent—is largely historical and may have arisen from a belief that the opportunity cost of capital is higher in lower income countries. More practically, it reflects past practice in other agencies and development institutions, such as the World Bank. To achieve a given ERR, this requires that the benefits from projects that are delivered further in the future are larger than projects with short term benefit profiles. EA continues to monitor new research and evidence with regards to time discounting and future revisions of the CBA Guidelines may include further guidance on best practices, particularly with respect to public goods, externalities and application to climate change interventions.

The time horizon to analyze projects is flexible, although MCC’s standard practice has been to choose a 20-year time horizon, plus the period required to put the investment(s) or activities in place. While the discount rate captures the preference for near-term benefits over long-term benefits, the choice of time horizon captures a mix of practical modeling concerns and also serves as a proxy for the fact that benefits (and costs) that occur sooner after an investment are more likely to be realized than those that occur very far in the future, due to the potential for inevitable confounding events. Therefore, in all cases, analysts need to study the sustainability of investments and benefit streams over such time periods, considering whether divergence from the counterfactual is likely to be maintained (e.g. considering the likelihood that necessary maintenance will be completed both in the with- and the without- project scenarios). Regardless of the reference time horizon utilized, the analysis should vary the time period over which the ERR is calculated to determine the sensitivity of the estimated returns to the time horizon. When the magnitude of the economic returns is highly sensitive to the time horizon (especially when plausible extensions or contractions cause it to cross the hurdle rate), this should be noted explicitly. Cases of high sensitivity to the time horizon may merit additional evidence gathering and vetting of assumptions about institutional requirements for achieving long-term benefits, with an eventual extension (or not) of the time horizon or incorporation of salvage values. Credibly measuring the value scrap equipment and other goods at the end of the time horizon is important, particularly because of the impact on the final ERR if this salvage value varies significantly across different time horizon options.[[April 2020 “Salvage Value Technical Note”.]] While high sensitivity to time horizon means that net benefits may be higher if it is extended, it also implies the converse: The economic viability of the project is more vulnerable to failures to ensure sustainability over the assumed project life, increasing uncertainty.

Treatment of Uncertainty

A cost-benefit analysis, like any modeling study, is conducted under conditions of uncertainty and as such will never perfectly capture anticipated project impacts. The uncertainties faced by the CBA analyst stem from many sources. For example, parameters drawn from existing studies may not have sufficient external validity to directly transfer to context of the project being appraised, and even internally valid confidence intervals on key parameters may have wide bounds. Valuing investments that increase productivity typically requires price trajectories over the project lifetime which in turn depend on difficult to predict market conditions. Even near-term costs of project implementation typically evolve, sometimes significantly, between the investment decision and compact completion.[[An analysis of ERRs (Ospina. S. “Why Do ERRs Change”, MCC May 2018) found that project costs increased 35% on average when comparing original ERRs and closeout ERRs.]] At a broader level, complementary investments may not take place on schedule (or at all), competing investments may depreciate the benefits of projects funded by MCC, and exogenous (e.g. political or climate) shocks may dramatically affect the differential impact of the MCC investment. Given the inevitability of such uncertainties, it is important to build an understanding of their relevance with an eye toward informing project design and selection. While terms are used differently in different contexts and disciplines, hereafter we refer to this process as “uncertainty assessment” (UA), which we consider an overarching term that encompasses a range of quantitative and qualitative techniques[[E.g., sensitivity analysis, scenario analysis, vulnerability assessment, Monte Carlo-based uncertainty propagation.]] for identifying and characterizing uncertainties and assessing their impact on project outcomes. While the set of techniques to be drawn upon will vary by context and require judgment by the analyst,[[A brief overview of uncertainty types and UA techniques is provided in this guidance update. The EA division expects to produce additional practical guidance on selecting between approaches and presenting results from this analysis, which will be incorporated into this and other sector-specific guidance documents.]] MCC expects that any CBA will address the economic implications of significant uncertainties to the extent feasible, with key risks discussed in the investment memorandum. These should be characterized quantitatively where possible and qualitatively where not.

Characterizing the uncertainty associated with a CBA can serve multiple purposes depending on the stage of the analysis and stage of the decision process. As examples:

• In choosing between projects, UA can help clarify whether there are risk/return trade-offs between potential elements of a compact (e.g., one project may have a higher median or mean net present value (NPV), but also a higher chance of a negative NPV).
• At earlier stages of project development, conducting sensitivity analysis can guide where additional data collection would have most value in refining the estimates of cost and benefit streams. This can also help guide monitoring and evaluation design to target learning toward areas of greater uncertainty.
• Earlier stage analysis can also help identify structural assumptions or future conditions under which the project design is vulnerable (or positively responsive), thereby facilitating iterations to project design that enhance benefits and mitigate vulnerabilities.
• Lastly, because uncertainty assessment typically involves thorough exploration of model behavior, it can also help validate and verify model structure. That is, as part of such explorations, the analyst may reveal unexpected responses to changing inputs that, upon investigation, may turn out to arise for a variety of informative reasons:
• They could be valid, in which case the UA helped reveal an unexpected consequence of accepted and embodied assumptions.
• They may be indicative of erroneous relationships encoded in the model (e.g., failure to account for an important interaction between project elements).
• Or they may be indicative of erroneous coding of valid relationships (e.g., bugs in the spreadsheet).

The process of characterizing uncertainty to treat it effectively in a particularly analytic and decision context is well-studied,[[Though language and default practice can vary significantly across disciplines. Economists should maintain awareness of this when attempting to integrate multiple sources of evidence and uncertainties into a single CBA.]] but also requires judgments that balance practical considerations, analytic rigor, and audience needs. Future revisions of this guidance are expected to offer more explicit recommendations on choosing between practical techniques; for the moment it is useful to briefly consider different sources of uncertainty and how they can be treated. Boundaries are fuzzy and multiple taxonomies exist, but a simple taxonomy involves considering three types of uncertainty:

• Parametric uncertainty refers to uncertainty regarding the appropriate value of numerical inputs that propagate through the relationships embodied in the model. Examples here would include the change in efficiency of services following an urban planning reform; the morbidity or mortality impact of improved water and sanitation on community health; or the increased efficiency value of public expenditures as the result of public financial management interventions.
• Structural uncertainty is related to whether and how key relationships (including benefit streams) are captured in the model, and how they interact. E.g., whether it is reasonable to assume induced investment from lowered input prices (e.g., from improved roads), the existence of non-negligible climate feedbacks on crop yields, or whether labor supply decisions can be abstracted, versus requiring a model that treats household production, labor supply, and consumption as a joint decision.[[Often, a structural assumption can be expressed as specific settings or limiting cases of a parametric assumption, and how to classify it is essentially a practical choice.]]
• Scenario uncertainty is generally related to major variation in exogenous factors. These can often be risks that are impossible to control and difficult to predict ex ante, but which can have a major impact on project implementation or outcomes. Examples include political crises, disruptions to trading relationships, price or exchange rate instability, labor unrest, disease outbreak, and deeply uncertain climate trends or extreme weather events. In general, when considering large exogenous changes, it is important to consistently consider the impact of the shock in the with-project and counterfactual cases, since it is the differential between the two that determines the economic merit of the project. In some cases, a shock may widen the gap between with-project and counterfactual benefits even if the shock reduces both benefit streams in an absolute sense. For example, extreme drought might reduce the magnitude of benefits flowing from an investment in a multi-purpose reservoir and irrigation system (since there is less water to be distributed), but the lack of even limited irrigation capability during the drought may be catastrophic.[[This dynamic is a route through which resilience-enhancing aspects of investments would potentially be evaluated, though the economics of resilience is a nascent field. Future versions of this guidance may revisit this topic in more detail.]] Conversely, shocks can also narrow the gap between with-project and counterfactual by stranding assets, e.g., a road built in a border region where geopolitical tensions reduce or eliminate traffic.

A given source or type of uncertainty can be treated with more than one technique, but often particular sources of uncertainty lend themselves to specific techniques. For example, uncertain qualitative issues (e.g., whether a complementary investment gets made) are often most conveniently treated with scenarios. (Numerical) parametric uncertainties that do not have a credible agreed upon probability distribution are often treated with simple sensitivity analysis, bounding values, or “break-even” approaches where the value required to cause the project to fall below the hurdle rate is identified and then assessed for plausibility.[[More complex approaches examine how uncertainties interact to create conditions of failure.]] In the most straightforward case, parameters with well-characterized statistical distributions allow for Bayesian or Monte Carlo propagation from input uncertainty to output uncertainty. Finally, it should be noted that scenario and probabilistic approaches are not mutually exclusive, as probabilities can be analyzed conditional on specific scenarios, and even the probability distributions used in a Monte Carlo process can be varied.[[I.e., typically, not only is the point estimate for an input parameter uncertain but the distribution of key variables parameters will also be subject to uncertainty. Therefore, the analyst will need to use (and perhaps elicit) judgment when developing parametric distributions, and also consider multiple distributions where relevant – e.g., an exponential family with narrower confidence windows and a uniform or triangular distribution.]]

For analysis conducted in support of the Investment Memo, MCC attempts to treat its reference ERR and NPV estimates in a probabilistic context using Monte Carlo approaches, which also include variance decomposition as a sensitivity analysis by-product.[[See the “What ERR to Report?” section in these guidelines.]] Future guidance will describe this process in more detail as well as offer guidance on practical approaches to uncertainty assessment at earlier project stages. In general, even when a primary analysis is conducted in a Monte Carlo framework, it can be valuable to examine economic costs and benefits under a variety of scenarios and sensitivity explorations to identify key risks to project benefits, while recognizing some exogenous shocks may be difficult to credibly model and are therefore better considered qualitatively. Regardless of the exact approach taken, such critical examination allows a focus on mitigation strategies for those parameters or assumptions for which the evidence is weakest and those which have the largest impact on economic returns.

Special Guidance

CBA for Grant Facilities

The key for any economic analysis is a clear logic and rationale. For grant facilities, the rational should follow from the analysis of root causes of binding constraints.[[A Grant Facility is a management structure for disbursing MCC compact or threshold funds through a competitive process of soliciting proposals, appraising them against defined criteria, selecting the best proposals, and issuing multiple awards.]] Country teams interested in pursuing a grant facility should articulate how it will address root causes and why a grant facility is better suited to doing so than other modalities for disbursing funds. If the rationale at this stage is satisfactory, the country team can begin further due diligence.

To inform due diligence, the economist should identify the key benefit streams that will underpin the economic rate of return as well as any critical modeling assumptions. Benefit streams will vary by the sector and purpose of the grant facility. In sectors where EA guidance is available (e.g. WASH), such documents provide a reference for modelling projects. However, a rational used for grants facilities is to leverage private sector investment and/or achieve additionality, for which there is no current internal guidance.[[Some rationale for this practice is provided in Warner, Andrew, A Framework for Efficient Government Investment (February 2013). IMF Working Paper No. 13/58, Available at SSRN: https://ssrn.com/abstract=2233761]]

One of the key tasks of grant facility due diligence is to collect information on the pool of potential compact projects. This information will allow the economist to understand the “typical” projects that could be funded, calculate the ERR of specific projects, and examine the distribution of the overall pool. The economist can then calculate an overall ERR for the grant facility to present to the Investment Management Committee.

The economic model and analysis prepared for the investment memo will provide a foundation for economic analysis during grant facility implementation. The MCC economist will work with their MCA counterpart, an MBO consultant, and/or an external grants program manager to develop a specific analysis model template that is based on the rationale and model developed for the IM. The template will outline the information necessary to make an ERR calculation and be applied to each individual grant proposal during the formal appraisal and selection process. This specific template will outline the information necessary to make an ERR calculation. A lesser standard of rigor than a full ERR calculation may be permissible in the case of very small grants.

CBA and Policy and Institutional Reform (PIR)

The primary guidance for conducting CBA on PIR elements was updated in 2018 and should be the primary source of guidance for these analyses.[[Economic Analysis of Policy and Institutional Reform Programs: Position Paper and Recommendations for MCC Management.]] The discussion in this section reflects main points from this guidance.

Estimating economic returns for policy and institutional reforms is difficult given the greater complexity of program logic, uncertainties, and data requirements. However, a clear program logic, defined benefit streams, and the requisite data should in most cases render CBA possible. ERRs must be estimated separately for activities that have a separable program logic linking inputs to benefit streams, including PIR activities. Consistent with MCC’s investment criteria, ERRs should be reported at the lowest level of disaggregation that is feasible and logical. When PIR activities are highly complementary to other physical investments in the compact, a combined ERR should be reported (i.e. when PIR elements are strictly necessary for achieving benefits from non-PIR elements). For example, land titling and transferability may require complementary investment in a functioning procurement audit system or infrastructure for an effective contract enforcement system in order to achieve project objectives. The decision to conduct separate analysis will depend on the resources required for the analysis, project size, and the degree of jointness between components in the program logic.

Some specific issues considered in the guidance include:

• The current guidance for operations and maintenance (O&M) is to analyze O&M investments under “business as usual” (BAU) for operations, maintenance, and other dimensions of performance. The guidance is to proceed, in collaboration with sector experts, with quantifying benefits of PIR interventions as deviations from the BAU scenario. This requires realism on the time path of reform, sustainability, and associated risks.
• Where there are benefits of PIR that would extend beyond enhancing the performance of MCC’s investments—as assessed by the economists together with sector experts—economists will attempt to include these benefits in the economic analysis of PIR investments. However, as with more traditional CBA, when expanding the set of benefits considered, economists should be careful to ensure consideration of additional costs from, e.g., the complementary investments required to realize those benefits.
• Where there are multiple small components of a broader PIR intervention that are functionally non-separable—as determined by the extent to which various elements are required in combination to produce identified benefit streams—these should be combined into a single PIR analysis.
• It is widely viewed that uncertainty regarding the outcomes of PIR projects may be greater than for other projects, particularly since the benefits of many PIR projects are often explicitly indirect or induced (e.g., improving the investment climate). MCC expects to review the policy response to this greater uncertainty to better inform decision making.

Reporting and Publication

MCC officially reports final ERRs at potentially five key points in time, and the ERR at each of these points is given a specific title and definition. The first four are normally completed by MCC Economists and considered ex ante CBA models, while the fifth is completed by an independent entity and considered an ex post model.

1. Original ERR: This refers to the baseline economic analysis that serves as an input to the Investment Memo and is part of the investment criteria considered for MCC management decision making. For indicators linked to CBA benefit streams, the M&E Plan’s indicator definitions and targets and the evaluation’s planned effect size and exposure period should match the Original ERR, unless a compact modification with a Revised ERR occurs.
2. EIF ERR: After a program enters-into-force (EIF), the first CBA model is published on MCC’s website. If economists significantly revise the Original ERR from MCC Board approval to EIF, then MCC will publish an EIF ERR instead of the Original ERR. The published EIF ERR model will document the ERR estimate from the Original ERR CBA model. Revisions typically occur when projects are further defined or new data on key parameters becomes available. These revisions have no implications for the indicators in the M&E Plan and for the evaluation. The necessary definitions, targets and exposure periods still come from the Original ERR.
3. Revised ERR: If a CBA model undergoes substantive changes after EIF (i.e., during implementation) due to compact modifications or new data that changes key assumptions or parameters, then the resulting Revised ERR will be published as the most recent model on MCC’s website, replacing the previous model. The ERR estimates from all previous CBA models will be documented within the most current version. It is possible to have more than one Revised ERR for a program. For indicators linked to CBA benefit streams, the M&E Plan’s indicator definitions and targets and the evaluation’s effect size and planned exposure period should match the Revised ERR only if a compact modification occurs. If there is a Revised ERR but no compact modification occurs, the Original ERR’s definitions, targets and exposure periods apply.
4. Closeout ERR: The Closeout CBA model typically incorporates updated investment cost and output data, but not updated outcome measurements, since it is unlikely that a sufficient exposure period has elapsed for outcomes to have occurred and been measured. The previous ERR estimates should be documented in this published CBA model.
5. Evaluation-Based ERR: An entity independent from MCC produces an evaluation-based CBA after the project’s final, independent evaluation has been completed (typically 2-7 years after compact closure). This CBA model allows updating of costs, output and outcome measurements.

For simplicity and ease in communicating results, the ERR at each stage is ultimately reported as a point estimate to key stakeholders (e.g. Congress). However, MCC publishes full Excel-based CBA models as well as supplementary information to provide context to the reported ERR. These documents explain the underlying economic rationale for the project and major modeling decisions, present the uncertainty and potential risks that underlie the ERR estimate, and provide supporting evidence, such as citations of studies that informed key parameters. To support consistency in reporting practices across economists and to provide stakeholders with consistent messaging on risks and benefits for final ERRs, EA provides templates for these key documents. These templates are updated periodically to improve the communication of any additional information.

Before publication on MCC’s website, and, in the case of the original ERR, before IM clearance, all CBAs must be peer reviewed according to the peer review guidelines and complete EA’s established clearance and publication process.[[See “Guidelines for Country Team and Peer Review of Cost Benefit Analysis” and “ERR Web Publication Process”..]]

Which ERR to Report?

Official communications of final ERRs must include the mean value of the ERR as the point estimate and the probability that the ERR is above MCC’s threshold, both obtained through a comprehensive sensitivity analysis. The key aspects of ERR reporting are outlined below.[[In a July 2020 Decision memo “What ERR to Report? Guidance for determining what ERR economists should report in official communications” EA adopted a standardized statistic for all official communications reporting an ERR. The Decision memo outlines guidance that allows for comparable measures of benefits and ERRs across programs.]]

There is no “correct” statistic that consistently balances all relevant considerations across projects. However, the mean ERR value from a comprehensive sensitivity analysis provides a relatively stable estimate of the average outcome of the CBA model, after accounting for known risk factors. A risk in adopting this statistic is that the mean ERR, as calculated from standard software packages, can be greatly affected by extreme shocks.[[That is, even a small number of cases with ERRs approaching negative infinity can cause the mean ERR to become unbounded as well.]]Fortunately, such outcomes are often considered uneconomic and automatically removed from the sensitivity analysis by standard software.[[If the economist approximates the parameter distribution using a gaussian, or other unbounded distribution, there will always be sufficiently extreme outcomes to cause the ERR to be undefined.]] However, removing these extreme outcomes may sometimes be a mistake.⁵⁶ In this later case, the economist may report the median ERR value obtained through a comprehensive sensitivity analysis and explain why a deviation from standard practice was needed.

The probability that an estimated ERR is above the threshold should be reported in official communications, along with the ERR point estimate. This metric is a result of the sensitivity analysis derived from a Monte Carlo simulation and aims to summarize the risk of the project falling below MCC’s investment criteria hurdle rate. The estimate is relatively more stable to changes in the model’s parameters and is expected to be easily understood by key stakeholders, supporting their interpretation of the ERR point estimate. However, both the mean ERR and the probability that it is above threshold are a function of certain assumptions about the domain and specification of the Monte Carlo analysis (e.g., certain risks may not have been probabilistically incorporated). These should also be reported.

Separate from the distinction between the two metrics discussed above, is the decision on whether to report point estimates from a base case or from a comprehensive sensitivity analysis.[[Because most investments require “up-front” capital costs, the benefit streams of a CBA model must reach a threshold of positivity for the ERR function to be defined, so the ERR can be undefined even for bounded parameter distributions.]] The base case ERR is a point estimate typically used during project development to communicate CBA model iterations that inform project design and facilitate decision making among Country Team members. This estimate typically reflects the most likely outcome of the project, but it is generally unstable to parameter realizations or other model updates, therefore exaggerating the effect of such changes. The official ERR used for MCC management decision making and communications is the result of the comprehensive sensitivity analysis as it aims to incorporate known uncertainty and risk, and is more robust against large fluctuations in the ERR point estimate when there are small changes to CBA model inputs.

Economists still have discretion to determine the approach and methods used for valuing each parameter within the CBA model to the best of their ability based on the information available, quality of data, overall level of uncertainty, type of parameter (e.g., log, percent, binary), etc., provided that the approach is based on evidence, in line with MCC guidance and properly sourced and explained. Peer review and transparency in reporting details on the parameter estimates used for modeling aim to ensure high-quality analysis.

Supporting documents reporting a final ERR (e.g., CBA models published on MCC’s website, M&E Plan) should include at least the following standardized information to comply with MCC’s full disclosure requirements and be aligned with the practices noted above: the ERR (mean), the probability that the ERR is above MCC’s threshold, the NPV (mean), the present value (PV) of all benefits, an outline of critical parameters (i.e., those that are most influential to the ERR estimate, including the sources and ranges for these parameters), and an indication of when project components with separate logics do not have separate ERRs including a note on why and if an ERR is expected in the future. This additional information aims to provide insights into the uncertainty and distribution of summary statistics to support greater understanding of an investment’s potential risks and benefits. Additional specifics for each supporting document are included within the templates developed and approved by EA management. Templates will be updated as needed or new templates will be created for new documents.

Denominating Costs and Benefits

Both costs and benefits should be consistently expressed in terms of either local currency units (LCU) or U.S. dollar values--using the same base year in either case (e.g., “2018 dollars”). Since foreign exchange rates are difficult to predict with any accuracy, the current exchange rate at the time of the analysis should be used in an ex ante analysis, with recognition of current exchange rate management practices (peg, managed float, etc.). MCC plans to review best practices and incorporate these for reference in a future revision of the CBA Guidelines. The purpose of this review is to achieve consistency with respect to exchange rate assumptions and modeling exchange rate risks. Pending the update of these guidelines based on this planned review, post-project ERR’s should consider carefully the effects of the currency choice, as exchange rate fluctuations can affect the ERR. For example, if project costs are incurred in dollars in year 1, but in year 5 the rate of exchange of LCU/USD increases, the ERR in USD terms will fall, but not in LCU terms.[[Currency conversions for compact transactions occur in real time, which means that only purchases of imported equipment or materials that occur after an exchange rate change will be affected. For example, in the Moldova THVA closeout ERR, a large devaluation occurred over a period of several months at the same time that a large percentage of compact funds were being spent on imported irrigation equipment. The purchase price and date of irrigation equipment were tracked by transaction to accurately reflect the increase in the real price of the imported equipment relative to farmer incomes.]] In such cases, the economist should utilize the local currency values to estimate the closeout ERR. When reporting the net present value of benefits in US dollars, the exchange rate obtained at the beginning of the project (EIF stage) should be used.

Annex 1: Value of a Statistical Life

MCC applies a VSL[[See the March 2020 VSL Decision Note (Bowen and Osborne 2020) for additional background.]] concept to value increased longevity resulting from MCC investments, usually in the health or water and sanitation sectors.[[This approach is consistent with current CBA practice outside MCC and is recommended by the Harvard School of Public Health CBA Guidelines (2019).]] The VSL for a target population can be defined as the average willingness to pay (WTP) for a small reduction in own mortality risk over the next year divided by this risk change. For example, if the average WTP for a 1/10,000 reduction in own mortality risk is $20, then the VSL is:

$\$20⁄1⁄10,0000\; =\; \$200,000.$

The VSL reflects target population individuals’ own “rate of tradeoff between small changes in own mortality risk and own spending on other goods and services, within a defined time period.” Note that this is “not the value that the individual, the government, the society, or the analyst places on saving a life with certainty.”[[Robinson et al. 2019.]]

VSLs are typically estimated using revealed or stated preference studies.[[See Robinson et al. 2019 for a detailed discussion of the methodology and literature.]] In contexts for which multiple high quality VSL studies are available for the target population of interest, these studies should be used to establish the VSL for modeling purposes. A detailed review of the literature, however, found that only 26 VSL studies were conducted in low or middle-income countries over the past 20 years and that many of these have significant methodological limitations.[[Robinson, Hammitt and O’Keeffe 2019.]] Given the limited availability of high quality VSL studies in most low- and middle-income contexts and substantial time and expense required to conduct high-quality VSL studies, the Harvard School of Public Health (HSPH) CBA Guidelines (2019) recommend that, over the near-term, CBA practitioners working in low- and middle-income contexts extrapolate VSL parameters from VSL estimates from high-income country contexts, where numerous high-quality VSL studies are available.

Given that demand for mortality risk reduction has been shown to decrease as income decreases (i.e. it is a normal good), it is necessary to correspondingly adjust VSL estimates transferred from high-income contexts. Based on the above mentioned literature review of VSL studies conducted in low and middle-income countries, the HSPH Guidelines recommend applying an income elasticity of 1.5 to the US VSL estimate to do so.[[Excluding studies with implausibly high or low ratios of VSL to GNI per capita, the authors find that the income elasticities implied by the VSL values found by the remaining 15 studies and the US Department of Human and Health Services VSL value range from 1.1 to 2.6 with a mean of 1.5 and a median of 1.4.]] In particular, the analyst should apply the formula:

$VSL$_target = VSL_reference × (Income_target⁄Income_reference)^elasticity

MCC CBA will follow the HSPH Guidelines base case recommendation where the United States VSL[[Following the HSPH Guidance, the U.S. Department of Health and Human Services (USDHHS) VSL will be applied. Specifically, a $9.4 million VSL corresponding to a U.S. GNI per capita of $57,900 in 2015 dollars (although these reference figures may be periodically updated).]] is the reference, the elasticity is 1.5, and income is measured in PPP GNI per Capita.[[While an elasticity of 1.5 applied to the US VSL and PPP GNI per capita income will be applied for purposes of testing investment ERRs against MCC’s 10% threshold, MCC analysts are welcome to follow the HSPH Guidelines’ recommendation to estimate a variety of specifications for purposes of sensitivity analysis.]] The VSL will be allowed to adjust over time with income growth. Following typical practice for policy applications, however, MCC will not adjust the VSL for income differences within countries.[[ Analysts have typically adopted this practice due to an authorizing environment perception that it is fair: applying a population-average treats everyone similarly.]]

The HSPH Guidelines recommend use of value of a statistical life year (VSLY) calculations for age related sensitivity analysis, but do not provide guidance for incorporating the age of target beneficiaries into base case analysis for purpose of assessing against a cost-effectiveness threshold.[[HSPH Guidelines note that “this recommendation is explicitly designed for use in uncertainty analysis; these guidelines do not address the use of VSLY estimates for other purposes such as deriving cost-effectiveness thresholds” (page 45). It is also noted that “assuming VSLY is constant provides a rough proxy for the effects of age and life expectancy but is not supported by theory or the available empirical research.” Note that the VSLY approach discussed in the guidelines relies on the assumption that the benefit of an increased probability of being alive in a future life year should be discounted at 0%. As noted in Robinson, Hammitt and O’Keefe 2019, this is inconsistent with typical past practice and is justified in part based on the fact that the approach is recommended only for sensitivity analysis.]] Robinson, Raich, Hammitt and O’Keeffe (2019) take a different approach, assessing the available empirical evidence on the ratio of child to adult VSLs in high-income countries to inform US policy decisions, but again suggest that age adjustments should be applied only for sensitivity analysis until more precise estimates of variation in values by age are available.[[Note that child VSLs necessarily rely on parents’ revealed and stated preference as a proxy.]] Nonetheless, consistent with the practice of transferring VSL estimates from high-income contexts, MCC will transfer child to adult VSL ratio estimates from high-income contexts for base case analysis over the near term.[[This practice should be periodically revisited at the discretion of the chief economist. Little is known at this time about the VSL child to adult ratio in low and lower-middle income contexts, which may differ substantially from high-income contexts.]] In particular, MCC will apply a ratio of 1.75 for beneficiaries 0-17 years of age, which is the midpoint of the range of values found by the Robinson, Raich, Hammitt and O’Keeffe (2019) literature review.[[Note that US VSL estimates are based on the revealed preference of adults (i.e. the studies exclude children) so that applying a higher value for children does not imply that the adult value should be adjusted downward (to reflect adult values being at the lower-end of a VSL distribution from which higher child values have been removed).]] A value of 1.0 will be applied for ages 18 and above.[[High-income country evidence suggests that working adult VSLs may follow an inverse-u pattern peaking in middle age. Values for older adults may remain the same or decline. Little is known about low or middle-income countries, though. Given the limited empirical evidence and problematic elements of the VSLY approach, we propose applying an adult value of 1.0 for base case analysis, consistent with the HSPH Guidelines.]]

If the value of mortality risk reduction benefits is not allowed to vary by within-country income differences, but the value of other benefit streams is allowed to vary by within-country income, then distributional analysis will imply that low-income households assign a greater value to their own mortality risk reduction relative to other benefits than they actually do. For this reason, the HSPH Guidelines recommend that VSL values be allowed to vary by within-country income differences for purposes of distributional analysis (but not for purpose of assessment versus a cost-effectiveness threshold, such as MCC’s 10% hurdle rate). Following the HSPH guidance, MCC will experiment with adjusting VSL values for within-country income-differences for purposes of beneficiary analysis even where these values are not adjusted for purposes of the base case assessment against the hurdle rate. Complications that arise will be addressed in a separate guidance note.

The benefit of increased probability of future life years, reflected in individuals’ revealed or stated preference for mortality risk reduction, is not fully experienced in the year in which mortality risk is reduced, but rather over the course of the expected (average) increase in longevity. This is based on the argument that the benefit of expected future years of life is certain to occur (i.e. be sustainable) without any further expense beyond the period in which mortality risk is averted.

Annex 2: Labor Income Benefits

Labor Income Benefits. MCC projects can impact employment and labor incomes. Two standard cases are where labor is hired for a project, e.g., for road construction and maintenance—in which case the salaries paid enter as a cost, not a benefit[[See Harberger (2008), ‘Labor Market Issues” http://www.econ.ucla.edu/harberger/introCBpart2.pdf. Note that the opportunity cost of labor needs to be appropriately estimated, and may be less than the wage paid. These issues of valuation apply equally to estimating employment benefits and are discussed below.]]--and when the project is a training project for which raising labor incomes of participants is the explicit objective. Addressing these cases is relatively straightforward. This Annex considers labor benefits that are less direct but that may be potentially significant for some projects, and which standard CBA practice does not always consider. For example, an investment in an industry may lead to deepening of supply chain linkages, thereby increasing output and hiring in upstream or downstream sectors. In a region where there is significant unemployment or underemployment, an MCC investment may lead to net job creation across multiple sectors. Further, in many contexts, the benefits may address objectives beyond income growth. Job creation, or jobs for specific groups such as women, youth, or populations in disadvantaged regions, may be valued by policymakers or society for reasons of equity or social stability.

Within the CBA framework, employment-related benefits arise when distortions in labor markets are addressed. First, there is often pervasive unemployment and underemployment in developing countries. When there is substantial unemployment, the market wage exceeds the opportunity cost of labor; each new job filled by an unemployed person thus raises national income by the difference in the hourly wage paid (or the marginal product of labor) and the opportunity cost times hours worked. This gain is not accounted for in the hiring decisions of firms, which only hire to the point where the wage and marginal product are equated, hence it is a form of externality (Robalino and Walker 2017). An equally if not more important source of allocative inefficiency in developing countries are the structural disparities in labor productivity across sectors—whether across economic sectors like agriculture vs. industry, or informal vs. formal sectors within an industry--that may reflect legal, mobility or information barriers to entry of labor into different sectors (one common example is a wage floor in the formal sector). The process of economic development can be cast in part as one of structural shifts in labor from low to high productivity uses (Rodrik, McMillan, and Sepulveda 2017). Any investment that effects such a shift serves to raise labor net incomes, hence value added.[[In contrast, where there are no distortions, a new project merely reallocates labor from existing uses where it earns the equilibrium wage, so there is no net gain in labor incomes.]]

While the above logic based on market failures is compelling, incorporating employment outcomes into CBA may be difficult. Two key challenges are (1) determining the potential employment impacts of investments, and (2) estimating the social cost of labor, which will determine the net gain in income added caused by the employment creation or labor adjustment. The analyst should start by considering the nature of the project and the employment conditions in the environment, i.e., the overall labor market. Is a significant amount of hiring expected, either in industries directly affected (direct employment effects) or upstream or downstream sectors (indirect effects)? Can the project be expected to, directly or indirectly, draw labor from lower to higher productivity sectors? For example, agricultural processing expansion will likely draw a workforce from relatively low productivity rural farm labor. With regards to employment conditions, is there substantial un- or under employment? Where there is substantial overall slack in the economy, whether nationally or locally, investments may lead to net job creation through the income multiplier (induced employment effects).

If initial consideration of the project and the labor market suggest that net increases of one or more of the types just described in employment are potentially significant, the analyst must determine how to predict these outcomes for the CBA. Unfortunately, there is a relative dearth of examples in the literature. A straightforward approach is to obtain parameters from evaluations of similar interventions in comparable environments. An example is investments in rural roads for which a number of mostly quasi-experimental impact evaluations have measured impacts on household incomes and employment. Otherwise, the analyst can consider using existing data or collecting new data to capture potential employment effects.

Where indirect employment creation via upstream linkages are likely to be important pathways to job and income creation, an Input-Output matrix for the economy may be applicable. However, most IO tables are broad brush, so may fail to capture localized or sub-industry linkages.  Further, they assume completely elastic supplies of all factors, including labor, which can lead to substantial overestimation of induced employment effects. This limitation can be overcome by using computable general equilibrium models. CGE models in principle capture direct and indirect (interindustry) effects as well as consumption multiplier (induced) effects on employment. This method may be beyond the scope and resources available for most work (and requires that a recent Social Accounting Matrix is available), and the advisability of taking a conservative stance with regards to multiplier effects was noted in the text. Still, these impacts may be important and may differ across potential projects especially when considering investment directed at specific locations. An ongoing partnership of MCC and IFPRI is exploring the use of LEWIE (Local Economy-Wide Impact Evaluation) models, which are a type of CGE models applied to local (usually rural) economies, where consumption multipliers as well as agricultural supply chain effects are typically important pathways to development.[[Multiplier effects as well as direct and indirect effects are also captured (in a non-structural way) in macro regression models of the impact of specific types of investments or reforms on GDP (and less commonly, on employment).  The IFC is currently developing macro-simulation approaches that use such estimates as a basis for imputing expected employment gains by broad economic sector.]]

When impacts on employment are expected to be largely contained to one or a few related sectors—so that partial equilibrium approaches suffice--collecting new survey data may be a feasible means of learning about potential employment impacts. EA often collects surveys to inform its CBAs, most often in the form of Willingness-to Pay surveys for infrastructure like power and water. However, even when firms are interviewed, these surveys as currently designed are not usually informative about employment effects. Managers are often asked what they would be willing to pay for say, more reliable power, not how their output and hiring would respond.[[Such stated preferences approaches do not capture employment or labor income externalities, nor would revealed preferences based on firm demand functions do so. A firm’s willingness to pay for a service measures the marginal benefit to the firm—that is, profits, not to factors of production. Estimating the labor externality requires information on potential hiring and wages and on the opportunity cost of labor.]] WTP surveys could easily be expanded to ask about current and potential employment and wages. Still, these surveys focus on customers in the primary market (e.g., for power, water); they do not capture linkages with upstream or downstream enterprises, where much of the employment impact may be felt. A promising alternative is sector-tailored value chain surveys like those recently developed by the World Bank. Both the directly affected industries and those linked to them are sampled.  In addition to current use of labor and capital labor ratios, information is elicited on potential use of labor and other factors if demand and output were to expand. Using this information, employment multipliers can be generated for different investments (e.g. Farole and Konishi’s, 2017).

Once the potential employment impacts (direct, and if possible, indirect) are ascertained, determining the economic cost of labor must be addressed. Prior initial examination will presumably have uncovered distortions in the labor market in the form either of substantial unemployment or labor productivity gaps between sectors (since if the labor market was relatively efficient and equilibrating there would be little point to assessing employment outcomes). When the project induces transitions from lower to higher productivity jobs, the opportunity cost of labor is the marginal productivity of labor (usually proxied by average productivity) in the initial employment. The net gain in labor income, hence in value added, will be determined by the difference in productivity or simply wages in the new uses and the older ones.

If workers are drawn from the ranks of the unemployed the opportunity cost of their labor is not zero because the leisure and home production activities of the unemployed bring utility and must be given up or reduced when they take a job. Estimating this opportunity cost requires knowledge of the labor supply curve for unemployed individuals (estimating the gain from working is akin to estimating a producer surplus). Note however, that MCC’s CBA practice traditionally has not assigned a value to leisure lost (or gained) as a result of a project, in keeping with the focus on changes in income, although forgone home production should be counted.

It is difficult to predict whether the new jobs will draw from the ranks of the unemployed or from the alternatively employed, and if the latter, whether from low or high productivity work. Unless the program specifically targets the unemployed, it may be reasonable to assume that the new jobs draw from the unemployed and currently employed in proportion to their share of the labor force (by skill level). Then the unemployed take only a fraction of the new jobs even in high unemployment settings, so most workers will come from other jobs. A conservative approach to estimate the opportunity cost of time is then to use a weighted average of the wage rates in the formal and informal sectors, with the weights based on the expected skill composition of labor demanded. The net value added gain per worker would be calculated as the difference between the expected (presumably formal) wage in the expanding sector and the opportunity costs so calculated.

Employment benefits related to social objectives of equity, poverty reduction, or stability are externalities since the hiring decisions of firms do not incorporate these benefits (Robalino and Walker label them ‘social externalities’). In principle these benefits can be monetized and included directly in CBA calculations, using discrete choice experiments to elicit what people would be willing to pay to achieve additional jobs for specified groups, as in Ricaldi and Mousley’s (2019) study of a solar power project in the West Bank and Gaza. This revealed preference, willing to pay approach has a close analog in methods to determine the value of environmental improvements and public amenities. However, there is little evidence yet on the reliability of applying WTP to valuing jobs and other equity related externalities. While the WTP approach is promising, more research is needed. One way forward would be to conduct such a WTP study while also sounding out policy makers and other stakeholders on the issue.

The difficulties of monetizing employment benefits—whether overall job creation, jobs for specific groups, or ‘higher quality’ jobs such as formal employment--can be avoided by using separate non-monetary criteria to assess or rank projects. For example, projects with significant job benefits, or benefits to targeted groups, may be favored over other projects with similar ERRs. MCC’s Beneficiary Analysis should in principle allow for this, although it has not been used as such to date.[[See the Guidelines for Economic and Beneficiary Analysis.]] More systematic or formal methods, labeled ‘multi-criteria analysis’ (MCA), have been in use for some time by various government agencies in the U.S and elsewhere, particularly for ranking public investments in transportation.[[A common framework for MCA (DLCG 2009) encompasses the following steps, once the main objectives or criteria and the expected changes in them have been identified by the investing authority alternative projects: (1) develop a scoring approach for each criteria to evaluate each project on a scale of 1-100; (2) create a performance of each project’s scores on each criteria; determine weights for each criteria; (3) calculate a single weighted score for each project over the criteria and compare project scores. In the present case, the standard CBA benefit estimates for the projects would constitute one criterion (and easily scaled as they are monetized values) while the number or type of created jobs for different groups would constitute other criteria. While this allows the combination of standard CBA estimates with other outcome measures, it obviously entails a series of choices regarding scoring systems for each criterion and most subjectively, how to weigh the different criteria. This may be difficult, since weights may differ for different stakeholders, the government, and MCC’s own objectives. More simply and perhaps most useful for MCC, the process could stop at the performance matrix stage and not formally score and compare alternatives. Decision makers at MCC and in country governments would still have a systematic presentation of expected benefits of projects in different dimensions, helping guide choices and keeping the process relatively transparent.]] With MCA, monetized benefits—that is, the project’s ERR—is one but not the sole criterion for selecting investments. Note that, while the multi-criteria approach avoids the need to assign a monetary value to employment benefits, which a formal integration into CBA would require, it is still necessary to predict the net change in employment using the methods above.

References

Boardman, Anthony E., et al. Cost-Benefit Analysis: Concepts and Practice. Cambridge University Press, 2018.

Bowen, Derick H, & Ngeleza, Guyslain K. Land Sector Cost-Benefit Analysis Guidance. MCC Internal Document, June 2019.

Bowen, Derick H, & Osborne, Stefan. VSL Decision Memo: Guidance for Valuation of Mortality Risk Reduction. MCC Internal Document, March 2020.

Epley, Brian. Consumer Surplus for Cost-Benefit Analysis. MCC Internal Document, May 2020.

Farole, Thomas; Konishi, Yasuo. 2017. Jobs in North Lebanon: Assessment of the Potato Value Chain. Jobs Working Paper; No. 9. World Bank, Washington, DC.

Harberger, Arnold. 2008. “Introduction to Cost Benefit Analysis, Part II: Labor Market Issues” http://www.econ.ucla.edu/harberger/introCBpart2.pdfhttp://www.econ.ucla.edu/harberger/introCBpart2.pdf.

Lee, D.B., Klein, L.A., Camus, G. Induced Traffic and Induced Demand. Transportation Research Record. 1999;1659(1):68-75.

McMillan, Margaret S.; Rodrik, Dani; Sepúlveda, Claudia. 2017. Structural Change, Fundamentals, and Growth: A Framework and Case Studies. Washington, DC: International Food Policy Research Institute and the World Bank.

The Millennium Challenge Corporation. Economic Analysis of Policy and Institutional Reform Programs: Position Paper and Recommendations for MCC Management. MCC Internal Document, August 2018.

The Millennium Challenge Corporation. Guidelines for Country Team and Peer Review. MCC Internal Document, May 2014.

The Millennium Challenge Corporation. RCA Practice Note. MCC Draft Internal Document, November 2018.

Motamed, Mesbah. Salvage Value. MCC Internal Document, July 2020.

Osborne, Stefan. Water Supply and Sanitation Sector Cost-Benefit Analysis Guidance. MCC Internal Document, July 2019.

Ospina, Sandra. A Commitment to Data-Driven Investments. The Millennium Challenge Corporation. May 4, 2016. https://www.mcc.gov/blog/entry/blog-050416-data-driven-investments

Ospina, Sandra. Why do ERRs Change? MCC Internal Document, May 2018.

Ostry, J., Loungani P, and Berg, A. 2019. Confronting Inequality: How Societies Can Choose Inclusive Growth. Columbia University Press.

Ricaldi, Federica; Mousley, Peter. 2019. Measuring Jobs-Linked Externalities in Private Investment Projects: A Fragility, Conflict, and Violence Perspective. Jobs Working Paper; No. 38. World Bank, Washington, DC.

Robinson, L., Hammitt, J. K., & O’Keefe, L. Valuing mortality risk reductions in global benefit-cost analysis. Journal of Benefit-Cost Analysis, 10(S1), 15–50. 2019.

Robinson, L.A., W.J. Raich, J.K. Hammitt, and L. O’Keeffe. “Valuing Children’s Fatality Risk Reductions,” Journal of Benefit-Cost Analysis, 10(2): 156-177. 2019.

Robalino, David A.; Walker, David Ian. 2017. Economic Analysis of Jobs Investment Projects. Jobs Working Paper; No. 7. Washington, DC: World Bank.

Szott, Aaron. Sunk Costs Decision Memo. MCC Internal Document, March 2020.

Warner, Andrew. A Framework for Efficient Government Investment. IMF Working Paper No. 13/58. February 2013.

Wolf Jr., Charles. Markets or Governments: Choosing between imperfect alternatives, Cambridge, MIT Press, 1988.