Design

TREDD practices begin in study design, when MCC staff and contractors define what data needs to be collected for what outputs and outcomes, how, and why. MCC staff and contractors must determine (i) if the data activity requires collection and handling of PII and/or sensitive data, and (ii) if disclosure of this data may pose any risk of harm to the data provider(s).

If PII does not need to be collected, then it should not be. If data that is being collected is already publicly available and not sensitive, then the necessity of promises of confidentiality should be considered carefully. These decisions should be discussed and agreed between the MCC staff, contractor, and country partners prior to data collection to ensure the research protocol and corresponding informed consent statement(s) align with the requirements of the study.

The following sections describe the TREDD practices to consider during the design phase of the data activity.

Training

The objective of training in the protection of human subjects is for data handlers to understand: (i) key ethical principles in research (beneficence, respect for persons, and justice), (ii) data provider vulnerabilities, and (iii) the risks to data providers, data handlers, MCC, and country partners of improper data sharing, and the corresponding risk mitigation measures. For the data handlers:

MCC M&E staff, other MCC staff (as applicable) – Training (with certification from the training provider) required every 4 years, or sooner in the event of a major change to the Common Rule.
Contractor Key Personnel – Training (with certification from the training provider) required every 4 years[[MCC is aligning its requirements with the social science research community by requesting renewal every 4 years, or in the event of a major change to the regulations. See for example the requirements of Harvard University, Georgetown University, and Stanford University, each of which requires renewal every three years.]], or sooner in the event of a major change to the Common Rule.
Data Collection/Field Staff – Training is required for data collection staff on the informed consent, survey instrument(s), and field protocols established to adhere to ethical principles.
Other data handlers – Training is strongly recommended for MCA staff, other contractor staff who collect, store, analyze, and/or share data.

Understanding Laws and Regulations

Data handlers should identify and understand all relevant local laws for proper data stewardship. Applicable laws include data privacy and protection laws, as well as any national regulations on research and protection of human subjects. There are several resources available to consult and identify relevant laws, including:

International Compilation of Human Research Standards by HHS is a listing of over 1,000 laws, regulations, and guidelines on human subjects’ protections in 130 countries and from many international organizations.
Data Protection Laws of the World by DLA Piper Law Group and Data Protection around the World by Commission Nationale de l'Informatique et des Libertés (CNIL) allow users to compare laws and regulations between countries.

The applicable laws and regulations may evolve over time. MCC and MCA Office of General Counsel (OGC) staff can support data handlers in understanding these issues as needed.

Contractors are also required to exclude United States (US) and European Union (EU) citizens from MCC-funded surveys. This is to mitigate additional requirements for managing data in accordance with US and EU data privacy laws.

Identifiable Data - Handlers and Data Flow

Early in the Design stage, MCC, country partners, and contractors should identify the necessary data handlers over the course of the data activity life cycle and clearly document who needs access to identifiable data and when. This information should be determined prior to submission of the research protocol to the Institutional Review Board (Section 3.4). Depending on the data activity and/or procurement mechanisms, there can be the multiple data handlers, including MCC, MCA, evaluation firms, and data collection firms.

Figure 1 outlines the two common data workflows across data handlers for an independent evaluation to consider when building the research protocol and informed consent. However, MCC notes that data handlers and data flow are context specific and may be adapted to the needs of the study and the requirements of the Institutional Review Board.

Figure 1.

While it is not MCC’s practice to routinely hold raw, identifiable data after the completion of a data activity, it may do so in a limited number of circumstances. For example, MCC may hold identifiable data: (i) to facilitate the transfer of an evaluation task from one contractor to another (where contractor 1 submits raw data to MCC to transfer to contractor 2), (ii) if there is a specific and known need to revisit same data providers for future data activities (such as to study sustainability issues), and (iii) if other business requirements are identified by the DRB (see Section 9.5). For this reason, contractors should ensure informed consent and research protocols allow and facilitate MCC’s holding of identifiable data, or if that is not feasible, establish protocols for how the contractor will manage transfers to another contractor or other MCC designated agent should the need arise. Where MCC does have reason to hold identifiable data, contractors are requested to submit the identifiable data to MCC only (not for access outside of MCC). In addition, as the holder of identifiable data, MCC will continue to protect the confidentiality of such data and withhold data where the disclosure of such is prohibited by law or MCC reasonably determines that the disclosure of such would harm an interest protected by an exemption under the Freedom of Information Act (FOIA)[[The Freedom of Information Act (5 U.S.C. 552 (1996)) was first passed in 1967 and gives the public a statutory right of access to federal agency records with the aim of encouraging government accountability through transparency. Like all federal agencies of the US Government, MCC may withhold information pursuant to the exemptions and exclusions to disclosure contained in the statute. See MCC’s FOIA Regulation (available at https://www.mcc.gov/resources/doc/mcc-foia-regulation-2018) for a detailed description of the rules MCC follows in processing requests for records under the Freedom of Information Act.]].

Institutional Review Board (IRB)

According to US regulations, IRBs “assure, both in advance and by periodic review, which appropriate steps are taken to protect the rights and welfare of humans participating as subjects in the research. To accomplish this purpose, IRBs use a group process to review research protocols and related materials (e.g., informed consent documents and investigator brochures) to ensure protection of the rights and welfare of human subjects of research.[[https://www.fda.gov/RegulatoryInformation/Guidances/ucm126420.htm]]”

All MCC-funded independent evaluations that require human subjects – whether quantitative, qualitative or both - are required to undergo IRB review, even if they are exempt under HHS definitions of human subjects’ research. This requirement is built into the Standard Evaluation Firm Scope of Work. If an IRB initially classifies the evaluation as “exempt” from full review, the contractor must discuss with the MCC Project Manager (PM) and Contracting Officers Representative (COR) how to proceed given MCC’s preference for all independent evaluation protocols to be reviewed by an IRB. If the contractor believes the independent evaluation should not undergo IRB review, a justification must be submitted to MCC and must be cleared by the MCC PM, COR, and M&E Managing Director.

For MCC data activities, there are three main types of IRBs to consider:

National IRB – This is a centralized IRB established within a country to review and govern research in that country.
Academic Institution IRBs – IRBs that are based within universities to govern the research produced by university staff. This is typically required if one or more staff members of the contractor are based in an academic institution.
Independent IRB firms – There are independent IRBs that may be contracted for academic and non-academic research.

Working with MCC staff and MCA staff, contractors will lead the IRB process for relevant data activities, which consists of the following steps:

Identify IRB(s) –MCC requires its contractors to submit to an HHS-registered IRB AND adhere to any National IRB requirements (as applicable). Depending on the context, the contractor may need to submit a protocol to multiple IRBs (for example, if a National IRB is required, but it is not HHS-registered, the contractor will also need to submit to an HHS-registered IRB). Staff and contractors can reference the International Compilation of Human Research Standards and Office for Human Research Protections (OHRP) Database to identify appropriate contacts and IRBs to ensure the protocol is reviewed by at least one IRB that is HHS-registered and local requirements are followed.
Timing – The contractor should identify in advance the schedule(s) for the IRB(s) and the time requirements for submission and review. These may vary significantly across contexts and type of IRB. Given this process can take 1-3 months or more, depending on the IRB process, contractors should build this into their projected timeline as early as possible.
Cost – The costs of initial and periodic IRB reviews vary by country and by how many years the protocol must be in place. This is because standard IRB review may require both an initial, larger submission and review fee, as well as an annual review fee to maintain the IRB coverage over the course of the research life cycle. For reference, costs for initial reviews by Independent IRB firms can range from $1300-$1500 USD, with annual review fees between $800-$1500 USD. This can be reduced if the study is determined to be exempt or not required to fall under annual review.
Representation – Depending on IRB requirements, contractors may need to submit and/or present the research protocol and documents to the IRB in person. This should be built into the work plan and budget accordingly.

Preliminary Findings

Early in the design stage when preparing the IRB review package, contractors should determine whether a timely feedback loop is required to share data activity findings. MCC has identified cases in which it was critical to share results of the data activity (even preliminary results) with data providers and/or other stakeholders in a timely way because the data collected directly affected the health and well-being of the population. MCC and contractors should assess upfront what data is being collected and, if applicable, whether data that directly affects the health and well-being of the population can be reported back in a timely manner, and through what mechanism(s). MCC views such ethical responsibilities as superseding any methodological concerns about contaminating the study sample. When such data is being collected – for example water quality testing – a process for ensuring an appropriate feedback loop should be built between MCC, the contractor, and relevant local stakeholders before the results are available. This issue should be fully discussed in the research protocol and agreed with the IRB before proceeding to data collection. In addition, this information sharing may require additional financial resources in the evaluation budget.

Informed Consent

The informed consent should be context-specific and furnish data providers with the following information:

Statement on data activity purpose (i.e. program evaluation) and voluntary nature of their participation.
Duration and description of specific procedures, reasonable expected risks of providing data, and reasonable expected benefits of providing data.
Promises of confidentiality and data sharing. The contractor should first determine if (i) PII data needs to be collected (for specific study purposes) AND (ii) if confidentiality promises are required. If PII data is not needed – it should not be collected, thereby limiting specific risks to confidentiality. If the data collected is public – directly observable and not sensitive – promises of confidentiality should be carefully considered as they may be unnecessary. The following statements in the informed consent lay the (possible) foundation for proper data sharing in the future:
1. Statement of whether or not the data will be shared, and if shared, with whom and to what extent. In particular, it may be necessary to clarify who will have access to the identifiable dataset and who will have access to a de-identified version of the dataset. If computational reproducibility requires access to identifiable data, then a statement on who will have access to identifiable data for the purpose of reproducibility should be included. If de-identified data will be made public or otherwise shared, then the statement should state this[[This is an explicit requirement in the Revised Common Rule (45 CFR 46.116(b)(9), subpart A).]];
2. Statement on how data will be de-identified (as applicable); and
3. Broad consent – The contractor may consider obtaining broad consent for identifiable data to be shared with other researchers for unknown learning purposes[[This is an option under the Revised Common Rule (45 CFR 46.116(b)(9), subpart A).]]. However, even if broad consent is obtained, MCC anticipates minimizing holding and/or sharing identifiable data to mitigate unauthorized disclosures or misuse of identifiable data that may cause harm to data providers and/or handlers.

MCC provides contractors with a generic informed consent statement template (Annex 1). The MCC PM and contractor should review the draft informed consent language (preferably based on the template) to ensure agreement prior to submission to the IRB. However, MCC recognizes the final informed consent statement must be reviewed and cleared by the contractor’s IRB(s). In instances where the IRB requires changes to the informed consent which may limit MCC’s ability to collect, store, and/or disseminate relevant data, contractors must notify MCC of these changes by providing a copy of the IRB approved informed consent marked to show changes from the originally agreed language. MCC staff will determine whether the IRB’s required changes should be discussed with the MCC Disclosure Review Board (DRB).

Future Data Sharing

Data handlers should carefully consider how elements of the design may inform (or prevent) future efforts for data de-identification and data sharing and take appropriate actions, including but not limited to:

Monitor knowledge of sample frame. What is the source for the sample frame, how available is this source to others, what is the size of the sample frame, and what percentage of the sample frame will be selected for the study? These are questions that will inform data de-identification efforts, specifically focused on understanding the extent to which outliers in the study sample may be outliers in the population, and therefore potentially useful for re-identification of individuals, households, communities, etc.
Monitor availability of linkage documentation. For future de-identification efforts, the data handlers will need to be aware of linkage documentation that may support re-identification efforts or mitigate de-identification efforts. For example, when preparing a public-use data set, the contractor may determine village names need to be de-identified and removed from the data. But if the names of the villages in the sample are disseminated elsewhere, such as in an Evaluation Design Report, this information could be used to re-identify the village names in the dataset and increase household/individual re-identification risk. For this reason, MCC staff and contractors should carefully consider what information is and will be available about the study sample that may pose a re-identification risk and manage appropriately.
Monitor knowledge of treatment. For program evaluations, how well-known is “treatment” status? For example, will random selection of communities/villages/schools/facilities/etc. receiving the treatment be publicized? This should be carefully considered as it is a form of linkage that may support re-identification efforts if the treatment status, or other information about the treatment group, is known and can help to re-identify data providers. MCC staff and contractors should therefore consider carefully how treatment status, program beneficiary lists, etc., may be managed to mitigate future re-identification risk.
Consider de-identification strategy early. De-identification efforts often require data permutations - such as suppression of specific variables’ values, including top and bottom coding, conversion of continuous variables to categorical or removal of any identifiable variation. Even if data does not need to be submitted to MCC until all data rounds are completed, MCC requires contractors to begin documenting their de-identification strategy in the De-Identification Worksheet (discussed below) at the completion of each round of data collection, as per the Standard SOW.
Flag identifying and sensitive data. Beginning with questionnaire design and data entry, the contractor may consider creating flags – such as a specific suffix in the variable number or name – to create an easy reference in data analysis, de-identification, and dissemination for variables which should be carefully considered. These variables may then be removed or permutated for proper data sharing in adherence with promises of confidentiality and risk mitigation.