Terms of Reference for BORESHA Impact study and BORESHA III project End line Evaluation
- Overview
Project Title: Building Opportunities for Resilience in the Horn of Africa (BORESHA I, BORESHAII, BORESHA III)
Background and Introduction
Funded by the European Union Trust Fund for Africa (EUTF), Building Opportunities for Resilience in the Horn of Africa (BORESHA) is a 5-year cross-border project (December 2017- December 2022) whose overall objective is to promote economic development and greater resilience, particularly among vulnerable groups, in the Mandera Triangle (a geographical region where the countries of Ethiopia, Kenya and Somalia meet). In BORESHA II and BORESHA III, DRC, World Vision and CARE International continued with resilience work, building on the first phase of the project. The life of the BORESHA II project was 10 months, from 1st March to end of December 2021, while the BORESHA III project is a one-year project from 1st January 2022 to 31st December 2022.
The project follows a community-driven approach to address the shared nature of the risks and opportunities facing vulnerable persons and communities in the Mandera Triangle border area. It is part of the EU's programme for Collaboration in the Cross-Border areas of the Horn of Africa, providing close to 70 million euros of investment to prevent and mitigate the impact of local conflict and to promote economic development and greater resilience in four different cross-border regions. The consortium was led by DRC and in the initial phase (BORESHA I) was implemented in partnership with WYG/Tetratech, World Vision and CARE International. BORESHA II and BORESHA III has three partners, namely DRC (the lead partner), CARE International and World Vision.
Overall objective of the project:
The overall goal of the project is to promote economic development and greater resilience, particularly among vulnerable groups, including youth, women, displaced persons and persons living with disabilities. This contributes to the overall EUTF objective to address the root causes of instability, forced displacement and irregular migration and to contribute to better migration management.
Expected results of the project:
1. Communities in Mandera Triangle are more resilient and better prepared to withstand and respond more effectively to shocks;
2. Individuals and communities become more self-reliant through increased skills and opportunities for cross-border employment, diversified enterprise and livelihoods;
3. Cross-border rangeland and other shared natural resources are more equitably and sustainably managed.
Target Beneficiaries:
Overall the project targeted 350, 000 individual beneļ¬ciaries. The programme sought to work with local and national government authorities, community leaders, young people, women, youth groups, women’s associations, local communities, private sector and local business leadership in Mandera, Gedo and Dolo Ado. Specifically, the project targeted vulnerable communities and individuals, or those at risk of migration or displacement.
BORESHA Theory of change
If we strengthen cross-border communities’ capacities to identify their own priorities, plan and advocate for measures to help them withstand shocks;
If we promote the development of inclusive cross-border environment for livestock and non-livestock trade and business, and foster private sector opportunities for women and young people;
If we support the equitable and common management of natural resources in the cross-border area;
Then communities will become more resilient and self-reliant; individuals, including women and young men, will have the skills and opportunity to take up a more diverse range of employment and livelihoods options; natural resources will be used more rationally, and with less conflict; local governments will be more accountable to their constituencies (outcomes); and fewer people will be displaced within, or migrate out from, the cross-border region (impact).
Since November 2017, the project has been implemented in three phases in the target areas of Mandera county in Kenya, Dollow and Beled Hawa district in Somalia, and Dolo Ado and Dolo Bay in Ethiopia: BORESHA I was implemented over 39 months (from December 2017 to February 2021); BORESHA II from March to December 2021, and; BORESHA III from January to December 2022. Project objectives remained the same throughout the three phases, with BORESHA II and BORESHA III building on the achievements of the previous phases.
With the project closure approaching, DRC intends to conduct a comprehensive impact study covering all three phases of BORESHA over the period from October 2022 to January 2023, and in addition, to complete an end of project evaluation of BORESHA III covering the period November 2022 to January 2023. The impact study will assess and document impacts of the three BORESHA project phases in specific thematic intervention areas, showing what worked and/or didn’t work and why, and make recommendations for future resilience programming/work that provides strategic direction to DRC and partners on future cross border resilience building. The BORESHA III evaluation will help document BORESHA III key achievements, gaps, challenges, lessons, and recommendations for future programming in resilience in volatile contexts.
Read more on BORESHA here https://boreshahoa.org/about-boresha/
Anticipated timing of the Impact Assessment Study and the Final Year Evaluation:
On/around October 2022 to 15th March 2023. Key timelines/workplan activities are set out in the table below:
Activity
Dates
Responsible
Procurement and contracting process - Mid-July to mid-September 2022 - PMU and supply chain
Impact study implementation and reporting - October to January 2023 - Consultancy firm and management team
BORESHA III Evaluation - November 2022 to January 2023. - Consultants and Evaluation management team
End of project dissemination workshop to close project and share findings of impact study and BORESHA III final year evaluation - February to March 2023 - BORESHA PMU and partners
Evaluation Owner: Nicoletta Buono; Evaluation Manager: Peterson Mucheke
Evaluation Team: PMU, Partner leads (CARE, WV, DRC), DRC HQ and Regional evaluation advisory team.
Type of evaluation: BORESHA Impact Study and BORESHA III End of Project Evaluation
Evaluation Trigger: This evaluation is triggered by: 1) Donor requirement and project commitment, as set out in the project contract document; 2) DRC evaluation policy (MELMOP) on projects with budget threshold > 2 million dollars, and; 3) A need to document the impacts of the three BORESHA phases and to document recommendations for future programming.
- Objectives of the impact study and the BORESHA III Evaluation
This TOR covers two elements – An Impact Study of the three phases of BORESHA, and an end of project evaluation of the final phase of BORESHA (BORESHA III).
- Impact Study
Objectives of the Impact Study
This study will analyse and document the impacts of the three BORESHA project phases through the lens of specific thematic intervention areas, showing what worked and/or didn’t work and why, and to make recommendations for future resilience programming/work that will provide future strategic direction on cross border resilience building, thinking, and planning, for DRC, consortium partners, donors and other stakeholders. The study will also interrogate and challenge assumptions -what changed (in terms of context (internal and external factors) and project design - in the course of project implementation and how it impacted the project), and contextualize project outcomes and results for all to understand.
Scope of the Impact Study
Topics/study areas to be covered in the impact study include: -
- The extent and depth of resilience achieved by project
interventions - comparing resilience outcomes (in targeted groups,
communities or individual entities) as a result of project supported
interventions, and also those not supported by project interventions,
focusing on:
- Climate resilience, including successes and failures related to project interventions.
- Conflict and resilience relationships
- Economic resilience, including successes and failures related to project interventions.
- Capturing outcomes and development results from the three phases of
BORESHA that demonstrate improved economic resilience at the individual,
household and community levels including:
- The benefits and sustainability of project interventions that targeted rural communities and the agriculture sector (crops and livestock); and,
- The sustainability of project interventions intended to strengthen market systems.
- Impacts resulting from project interventions, and progress made with the development of cross border restrictions on trade and resilience building.
- Progress made to improve/develop market systems and key value chains to provide vulnerable persons and communities a means to participate in economic activity.
- The impact of unexpected shocks/external factors, such asCOVID-19, drought etc.
- Adaptive programming that occurred in cross border work, and the evidence thereof in line with context complexity. How flexible were donors in response to this? Any challenges with budgeting and/or length of projects?
- The value (social and economic) of interventions and how they
contribute to peaceful interactions and co-existence between
communities.
- Community structures
- Infrastructure development etc
Key questions for the Impact Study
The impact study questions will follow the thematic areas set out under the scope above. The consultant is required to share the proposed questions, which will be reviewed and refined by the taskforce during the inception phase.
Impact Study methodology
The study methodology will be defined and set out in detail in liaison with the taskforce. Key methodology requirements are as follows:
- Demonstrate how the consultant(s) will measure and document key changes that happened during the course of the project (process evaluation) and how these changes impacted the expected outcomes/impacts. This will involve interrogating the assumptions set at the beginning of the project.
- Compare and contrast project outcomes among actual beneficiaries and non-beneficiaries to capture evidence of change or otherwise.
- Be flexible in line with conditions on the ground and ethical considerations.
- Incorporate mixed methods (such as reviewing project documents and other studies, as well as quantitative and qualitative approaches).
Practical implementation aspects of the Impact Study
The bid submitted by the consultant(s) should include the following information:
- A clear outline and explanation of the Impact Study protocol/plan, including the proposed approach, methodology, days covered (approximately 45 days in 3 months)
- A detailed budget.
- Skills and experience of the consultant team members.
- Language proficiency of consultant team members.
- A workplan which incorporates a schedule of activities and timelines, including on the following tasks/processes:
- Desk review
- Start-up meeting
- Field data collection including use of digital tools
- Consultation with beneficiaries
- Draft reports
- Debrief meeting – reflect on findings and recommendations
- Acceptance of Final Reports
- Follow up on findings and recommendations
Impact assessment deliverables
- An Inception Report in response to the TOR, detailing methodologies to be used and representative sample size calculations, a detailed execution plan, data-collection tools, clearly defined research/assessment questions and corresponding sub-questions (incorporating a summary of introductory meetings). The report should also clearly explain how the proposed approach and methodology will ensure adherence to laid down COVID 19 protocols in the three countries.
- Draft Study Report incorporating a draft summary of findings and recommendations.
- Impact study datasets.
- A presentation of key findings in a validation workshop (in person or online).
- A comprehensive Final Impact Study Report that highlights the findings, lessons learnt, best practices and practical recommendations for future programming.
The Final Impact Study Report will be due on or before 31st of December 2022. The findings, recommendations and conclusions should go beyond the project level and provide higher level learning for DRC and partners after the project closes.
- A two-pager stand-alone Impact Study Summary Brief that is sharable for dissemination.
- A DRC Lessons Learned Note.
- A presentation of the findings made at the dissemination workshop.
- BORESHA III end of project Evaluation
End of project evaluations were conducted for BORESHA I and for BORESHA II. Therefore, this particular evaluation focuses only on BORESHA III. It will help to document BORESHA III key achievements, gaps, challenges, lessons learned, and recommendations for future programming on resilience outcomes in volatile contexts.
Objectives of the Evaluation
The key objective is to document the overall performance and impacts of the BORESHA III project against set goals, objectives/outcomes, and outputs as a result of project interventions, as defined in the project theory of change and results framework. The evaluation will also seek to document any lessons learned, expected as well as unexpected results or changes within and outside the project that impacted project delivery and impact.
Scope of the Evaluation
- Will cover only the BORESHA III phase.
- The main focus of the evaluation will be to assess the processes followed and implementation of project activities, contextual conditions, internal and external changes during the project life cycle and their impact on implementation, and on achieving project results and outcomes, including the sustainability of benefits.
- The evaluation will also review and assess the documented project results/reports, the roles of the three implementing partners (DRC, World Vision International and CARE International), and reflect on the impact of the project on beneficiaries in the three countries (Ethiopia, Kenya and Somalia).
- The evaluation will build on the recently completed BORESHA phase I and BORESHA phase II evaluations, the BORESHA project studies and ROM mission reports that are available for review. The evaluation will be light on methodology. The study will also refer to other interventions implemented within the project location for comparison.
- Thematic areas will include: Resilience, Disaster Risk Reduction (DRR), Index Based Livestock Insurance (IBLI), Livelihoods, the Private Sector, TVET, VLSA, Natural resources management, drought, conflict/peace building, cross border, gender, WASH, COVID 19. For each thematic area, the evaluation should focus on impacts, challenges, lessons learnt, sustainability, replication and scaling-up opportunities (expand each of the thematic areas to give a sense of what was done under each eg. technical approaches used, expected results, extent to which the approaches worked).
- Geographical areas include: Dollow Somalia, Mandera Kenya and Dolo Ado and Dolo Bay in Ethiopia, and Nairobi for coordination activities.
Evaluation methodology
The evaluation methodology will be further developed by the consultant / evaluator(s) and be well articulated in the response to the ToR. The methodology should demonstrate robustness and rigour in addressing the evaluation objectives and questions. Interested consultants will be expected to propose the design and methodology to be used in this evaluation as part of their proposals. The methodology proposed by the successful consultant shall be discussed, finalized and approved by the taskforce before commencing data collection as part of the quality assurance process. It is however expected that the methodology will use a mixed-methods approach including both qualitative and quantitative components. The consultant should demonstrate how they will conduct the evaluation. The consultant will aim to provide a clear analysis plan in line with the BORESHA III end of project evaluation objectives. Moreover, the Evaluation team will assess the accountability of implementing partners to beneficiaries, as well as integration of crosscutting issues, in particular gender and conflict/peace building.
Practical Implementation of the BORESHA III Evaluation
The bid submitted by the consultant(s) should include the following information:
- A clear outline and explanation of the Evaluation Plan, including days covered.
- A detailed budget.
- Skills and experience of the consultant team members.
- Language proficiency of consultant team members.
- A workplan which incorporates a schedule of activities and timelines, including on the following tasks/processes:
- Preparation of draft ToR
- Selection of consultants
- Agreement on ToR
- Desk review
- Start-up meeting
- Field data collection including use of digital tools
- Consultation with beneficiaries
- Draft reports
- Debrief meeting – reflect on findings and recommendations
- Acceptance of Final Reports
- Follow up on findings and recommendations
Evaluation Questions
The questions that the evaluation will seek to answer are highlighted under the different OECD criteria below. The evaluation will use the OECD DAC evaluation criteria adapted to humanitarian action by ALNAP.
The relevant OECD DAC evaluation criteria for the project are:
The evaluation questions to be answered under each evaluation criteria are listed below.
- Effectiveness
- To what extent did the project achieve its intended objectives and outcomes?
- To what extent did the project build the resilience of communities and individuals to respond to climate related shocks?
- What internal and external factors affected (both positively and negatively) the effectiveness and coverage of the project?
- What were the design, implementation, coordination, integration, collaboration and overall programming strengths and weaknesses?
- To what extent was the project gender sensitive and conflict sensitive in its approaches?
- Efficiency
- How well did the project utilise the available resources to maximise benefits?
- Coherence
- Linkages between BORESHA III and other existing processes and actors. In what ways did adopted approaches in BORESHA III contribute to resilience in line with existing interventions, structures and public sector strategies and plans?
- Coordination
- Consortium Management. How did project ensure effective feedback mechanisms between stakeholder institutions and implementing partners as well with other projects in the cross – border consortium environment?
- How well did the project partners connect and coordinate with beneficiaries?
- Impact
- What has changed (within the BORESHA III context – external and internal factors - positive or negative as well as intended/unintended as well as project design) that can be associated with to the BORESHA III project contribution (in different areas and target populations) in line with project theory of change? What has been the major changes and contributions to tackling cross border challenges/needs?
- Did impact vary for different targeted areas, groups, households, or individuals (refugee, Host, men and women)? If so, how and why?
- How has BORESHA III helped to build on the impacts of the prior phases of the project.
- Sustainability. At the design phase what elements of sustainability were identified? What is their status now?
- Are the project benefits and impacts sustainable? Which ones and in what ways?
- What opportunities for scaling up/replication have been identified?
- What lessons can we pick from the project (design, implementation, assumptions…)?
- Relevance.[1]
Evaluation deliverables
The following are the evaluation deliverables:
- An Inception Report in response to the ToR presenting the consultant’s understanding of the TOR, detailing methodologies to be used and representative sample size calculations, a detailed execution plan, data-collection tools, clearly defined research/assessment questions and corresponding or feeding sub-questions (incorporating the summary of the introductory meetings). Clearly demonstrate how proposed approach and methodology ensures adherence to laid down COVID 19 protocols in the 3 countries.
- Draft Report incorporating a draft summary of findings and recommendations by partners.
- Evaluation datasets.
- A presentation of key findings in a validation workshop (in person or online)
- A comprehensive Final Evaluation Report that highlights the findings, lessons learnt, best practices and practical recommendations.
The Final BORESHA III Evaluation Report will be due on or before 31st January 2023. The findings, recommendations and conclusions should go beyond the project level and provide higher level learning for DRC and partners after the project.
- A two-pager stand-alone BORESHA III Evaluation Summary Brief that is sharable for dissemination.
- A DRC Lessons Learned Note.
- A presentation of the draft report made at a dissemination workshop – in person or online.
The following sections relate to both the impact study and the BORESHA III end of project evaluation.
- Recommended documentation for review for the Impact Study and the BORESHA III end of project Evaluation.
As a minimum, the selected consultants will be made aware of and have access to the following documentation:
- Project proposal, project document, logical framework and/or theory of change, publications and relevant monitoring and donor reports and data.
- Project studies.
- Reports from evaluations conducted (BORESHA I and BORESHA II as key evidence of the history and implementation of recommendations from previous phases) on similar or related projects.
- Evaluation report template – attached.
- Intended use of the findings and recommendations Impact Study and the BORESHA III end of project Evaluation.
The intended users of both the Impact Study and end of project Evaluation shall include the European Union Trust Fund for Africa (EUTF), DRC programmes, BORESHA III consortium partners and other stakeholders who have an interest in resilience work in the cross-border areas.
- Findings will inform future programming on resilience in cross-border settings, specifically in the Horn of Africa.
- The findings will be presented to donors and other stakeholders at an end of the project conference. The findings will also be shared at other post project partner events, as appropriate (e.g. with Altai), and disseminated to a wider audience. DRC Lessons Learned Note will be used and sent to MEL@drc.dk for input to the annual meta-evaluation process in DRC.
- Post assignment Follow Up
- A signed management response will be shared with the consultants and Head of Desk and GSL on Monitoring and Evaluation in OPSU
- The Lessons Learned Note, which is sent to HQ mel@drc.ngo will be done by the evaluator
- Dissemination meeting
The consultants will together with the taskforce present the findings of both pieces of this assignment in the end of BORESHA project dissemination meeting in February/March 2023.
- Managing the task- Roles and responsibilities
The key stakeholders and their specific roles in the process are:
No
Stakeholder - Role in the evaluation
1 Project PMU lead - Commission the impact study and evaluation
2 Study/Evaluation Manager - Study/Evaluation planning and coordination
3 Study/Evaluation Team of Advisors / Committee, PMU, Partner leads - Provide technical support and guidance to the study/Evaluation Manager and team. Review key study/evaluation documents to shape the quality of the study/evaluation processes and products
4 Study/Evaluation team PMU, Partner leads, consultants - Implementation of the study/evaluation plan
5 Consultant TBD - Implement evaluation and deliver agreed deliverables within schedule
6 Other key stakeholders: Partners, Local government officials and beneficiaries’ communities. Project staff. - Provide feedback and experiences on the project. Validation of findings.
- Expected profile of consultants
The Impact Study/Evaluation team/organization shall include one local consultant from each of the 3 project countries. The desired specification and qualities of the consultants are:
- Demonstrated experience conducting impact studies for large and complex interventions.
- Demonstrated experience conducting final evaluations for multi-country resilience projects.
- Firm/team with personnel holding advanced degree in any of development, agriculture, natural resources management, disaster management, rural development or related fields.
- Demonstrated experience in similar assignments, in particular in the cross-border areas of the Horn of Africa and ASALs.
- Advanced knowledge and/or experience in resilience, climate change, disaster management, livelihoods, grants, training and rural development or related field as well as project implementation and monitoring.
- IRB certification or NACOSTI or equivalent certification.
- Team working experience and/or knowledge of livestock production, marketing, processing, animal health service delivery, training institutions or formation of national laws, policies and strategies.
- Proven experience of using participatory methods for data collection and analysis in cross-border multi-partner and multi-country programme/project studies/evaluations.
- Experience of effective interaction with local, national and regional government institutions and marginalized communities.
- Good spoken and written communication skills in English.
- Team must consist of some members fluent in Somali language.
- Excellent skills and ability to articulate ideas in a clear and accurate manner including the ability to prepare reports.
- Strong interpersonal skills, analytical skills and ability to establish and maintain effective working relations.
- Operational requirements – the consultants will be required to manage data collection for both tasks (the Impact Study and the BORESHA III Evaluation) concurrently. A single bid will be prepared and submitted to fully incorporate both assignments. Only one contract will be issued for both assignments.
- Data Protection and Confidentiality
While executing this assignment, the consultant and all the parties involved shall ensure effective protection of confidential and sensitive data and information in conformity with the humanitarian and protection principles and to applicable legal data protection standards[2]. All data collection and processing activities shall be executed in accordance with the following principles, that is:
- Safeguarding individuals’ Personal Data is a crucial part of humanitarian mission to protect the lives, integrity and dignity of beneficiaries and participants and is fundamental in the provision of protection response and humanitarian aid.
- People-centred and inclusive. Evaluation activities will respect the interests and well-being of the population, in all relevant phases of the evaluation and which activities must be sensitive to age, gender, and other issues of diversity.
- Do No Harm - evaluation activities must include a risk assessment and take steps, if necessary, to mitigate identified risks. The risk assessment must look at negative consequences that may result from data collection and subsequent actions or service delivery.
- Defined purpose and proportionality. The purpose must be clearly defined and explained to the participants in the data collection process
- Informed consent and confidentiality. Personal information may be collected only after informed consent has been provided by the individual in question and that individual must be aware of the purpose of the collection. Further, confidentiality must be clearly explained to the individual before the information may be collected. Consent must be genuine, based on the data subject’s free and informed decision.
- Data protection and security. The evaluation process must adhere to international standards of data protection and data security.
- The handling of data for this evaluation process must comply with GPDR regulations (General Data Protection Regulation (GDPR) Compliance Guidelines) and national data protection policies
- Competency and capacity. Actors engaging in this evaluation activities are accountable for ensuring that evaluation activities are carried out by competent team who have been trained appropriately.
- Impartiality – all steps of the evaluation cycle must be undertaken in an objective, impartial, and transparent manner while identifying and minimizing bias.
1. The relevance part of the project has been assessed in the recently conducted evaluation thus it is not included in this assignment
[2] ICRC Handbook for Data Protection in Humanitarian Action, UNHCR’s Policy on the Protection of
Personal Data of Persons of Concern, DRC’s Operational Handbook regarding data security and
management of hardware and the European Union General Data Protection Regulation (GDPR).
How to apply
Please find complete RFQ documents in the following link: RFQ 203110 - Boresha Evalaution
Submission requirement is attached separately. Interested firms that meet requirements should send their proposal and other required documents to email address tender.ken@drc.ngo on or before 15th August 2022 5pm EAT.
Please indicate “PROVISION OF BORESHA final Impact study and BORESHA III Project Endline Evaluation Consultancy – “RFQ NO.KEN-203110” in the subject line of your email application.