Terms of Reference (ToR) for the COMIC Relief Project end - Evaluation in Korogocho Informal Setlements, Nairobi
1.0 About Terre des hommes
Terre des hommes (Tdh) is Switzerland's largest children's aid organization. For more than half a century, the Foundation has been working alongside children in distress. It struggles to enforce their rights, without political, racial or denominational concerns. In more than 30 countries, Tdh protects children against exploitation and violence, improves the health of children and their mothers, and provides psychological support and material assistance in times of humanitarian crisis.
Tdh has been present in Kenya since 2011 and is currently supporting access to basic services for populations affected by displacement, natural crises and poverty in Dadaab refugee camps, in Garissa County and in Korogocho slum in Nairobi.
The intervention in Dadaab refugee camps is focused on prevention and response to child protection needs while strengthening protection community-based mechanisms in the refugee operation and its host communities.
In Korogocho informal settlements, Tdh implements early child development activities and child protection activities.
2.0 PROGRAM BACKGROUND
Korogocho is the fourth largest slum in Nairobi and one of the most congested and poorest urban areas in Kenya. It has an estimated population of 150,000 inhabitants (including 60,000 children) who live in a mere 1.5 km2, coping with exceedingly cramped, unsanitary and unsafe housing conditions and high levels of poverty. Most Vulnerable Children (MVCs) are identified by the Kenyan government and UNICEF as those in the poorest 40% of households and living in informal urban settlements. High economic insecurity, violation of rights, endemic violence and lack of social protection services are undermining the capacity of the communities to enjoy their rights and develop opportunities in equality.Tdh` long-standing child protection programme in Korogocho has identified that these children are especially vulnerable to abuse. There are high levels of sexual and gender-based violence (6 % of 2018 caseloads) and child neglect (32%). Girls in Korogocho are further exposed to specific risks, both as young children and then later in life (sexual violence, sexual activity as a means of livelihoods, single parenting, early pregnancy). The factors which contribute to and exacerbate child protection concerns include endemic unemployment and severe household financial insecurity; high crime rates including domestic and intimate partner abuse.
Tdh in partnership with Comic Relief has been implementing the project ““Kuishi na Kustawi”-Transforming early childhood development outcomes, a 3-year Early Childhood Development project promoting optimal physical, emotional and secure development and provide early learning opportunities to 1800 most vulnerable children aged 0-6 in Korogocho slum.
The project`s objective is to provide holistic and nurturing care for young children (especially girls) through the following inter-linked outcomes;
1.Centre-based development: Improving the ability of ECD centres and their staff to respond to the development needs of children aged 0 to 6.
2.Support parents/community stakeholders to promote optimal and nurturing ECD: Strengthen parents ‘and community stakeholders ‘capacities to actively nurture the physical, social, emotional, and cognitive growth of young children
3.Policy: Influence policy choices of the local and national child development services to prioritise early childhood development.
The project will end in December 30, 2022, and an external final evaluation is planned to take place in between October and November 2022.
The project`s theory of change identifies centre-based development as a key outcome towards achieving the envisioned objective. This is to be achieved by building of capacities in existing ECD centres, improving access to and infrastructure at centres, early childhood development methodologies in ECD centres (and for parenting) drawing on the existing curriculum and by introducing a child-centred approach to promote skills including: creativity, problem-solving, communication and collaboration in children 0 to 6. Children’s individual learning style and individual learning goals will be noted, with learning trajectory recorded by ECD workers and parents. ECD personnel will better understand how to promote cognitive, physical, and socio-emotional development, and be able to apply this knowledge in their work. Towards this end, Tdh has been working with 30 ECD centres in Korogocho.
In addition to centre-based development, community ownership and demand creation of ECD is pivotal towards having a holistic approach positive towards ECD outcomes. Parental training, peer to peer parenting groups, supporting access to services and guided child to child interactions with older children are among the interventions in place, while facilitating social inclusion specifically for children with disabilities and girls.
Consistent and intentional technical and financial support of ECD is key to spur and sustain positive ECD outcomes and it`s is for this reason that the results achieved under outcome 1 and 2 be complemented by advocacy at Sub-County, County and national levels to influence the national ECD frameworks and inform community of practice in improving ECD interventions amongst peers.
3.0 EVALUATION PURPOSE
The evaluation is a Tdh final evaluation to be conducted through an external firm or consultant intending to inform Tdh and Comic relief on the effectiveness, efficiency, relevance, and overall performance of the project. The main objective is to evaluate the program's impact against the desired results as articulated in the project's result framework in the 3 years of implementation. Recommendations will be used at a strategic level to improve learning for future interventions.
The evaluations specific objectives aim to;
- Assess the progress of the project's indicators against the targets, as articulated in the result-framework.
- To assess the effectiveness of centre based ECD approaches towards improving capacities to respond to developmental needs of children between 0-6 years as well as improving access and quality.
- To assess the effectiveness of the project in changing social and community attitudes and perceptions on Early childhood development.
- To appraise and document the degree of the project`s influence on the community of practice for ECD interventions as well as local and national policy implementation and/or formulation.
- Provide input to similar programs or scale ups and learning on assumptions and pathways for Tdh` developing theory of change on promoting and sustaining positive ECD outcomes.
- To document and make recommendations backed by solid evidence on what worked (or didn’t work) as well as sustainability of the results in each of the 3 outcomes identified during project design.
4.0 EVALUATION CRITERIA
The project evaluation should be organized taking into consideration of OECD evaluation criteria, and program quality standards. The evaluation will also consider key sector and sub-sector indicators for assessing the performance. The following provides a guide to the questions to be addressed by this evaluation, under each of the criteria below:
Relevance and Appropriateness:
- To what extent were the project activities responding to recipients/target beneficiaries and identified needs and priorities defined by local actors.
- Analyse whether and how the project activities responded to any changes that might have happened during the project duration
- How did the project activities complement and align to similar activities and influence efforts implemented by local and national actors?
- Analyse the relevance of the designed project activities to the project overall objective and outcomes. Furthermore, were they relevant to the project theory of change and the identified outcomes and outputs.
- How relevant was the geographic location to the project activities either at the beneficiary level or for influencing?
- Was the project design appropriate to the risks and challenges identified through the project design or might have emerged during the project duration. (An analysis of challenges and risks within the internal and external environment needs to be provided)
Impact:
- To what extent has the project generated or is expected to generate significant positive or negative, intended, or unintended changes in the lives of recipients and in their environment?
- To what extent has the project contributed or is expected to contribute to: Statements, initiatives, policies or programs by national or local government officials supportive of the ECD whether technically or financially to improve ECD access, quality as well capacity and efficacy of the ECD workforce to respond to children`s developmental needs.
- What are the broader effects of the project on individuals, gender, and age groups, host-communities and institutions?
- What were the significant factors influencing the achievement of the project's outputs and outcomes?
Effectiveness:
- To what extent has the project achieved its objectives, and its results, including any differential results across different groups?
- What factors have contributed to achieving or not achieving intended project outcomes and set objectives?
- Have the M&E system delivered robust and useful information that could be used to assess progress towards outcomes and contribute to learning?
- Has the accountability system ensured participation, regular feedback/complaint from the community and provided a timely response?
- Assess how the partner portfolio and the mix of different types of partners have contributed to the objectives of the strategy.
Efficiency:
- To what extent has the project delivered results in an economic and timely way?
- Was the project designed and/or amended throughout the implementation period to provide the best value for money?
- Was this project the most cost-effective as compared to similar projects?
- Were the resources for running all the activities available, adequate and was this the best use of resources to achieve results?
- To what extent has effective coordination and collaboration with existing interventions and partners been addressed and achieved?
Sustainability:
- To what extent are there financial, institutional, socio-economic, and/or environmental mechanisms built into the design of the project for sustaining project results after end of external support?
- To what extent can the activities of the project continue after donor funding ceased? Is there an exit/sustainability strategy in place or planned?
- Is it likely that the benefits of the project (capacities developed; linkages, mutual learning and knowledge and experiences shared) would be sustainable or is there any action required at the endpoint?
- To what extent do project recipients and stakeholders have ownership, capacity, and resources to maintain the activity results after external funding ceases?
- What were/are the major factors that influenced the achievement or non-achievement of sustainability of the project?
5.0 EVALUATION APPROACH AND METHODOLOGY
Tdh recommends an inclusive mixed methods approach, a combination of both quantitative and qualitative research methods that are relevant to the nature of the project and the evaluation objective. It should be utilization-focused, gender-responsive, and explore the possibility of utilizing participatory methods for developing case studies and/or examples to support the finding of the evaluation process. Data should be disaggregated by sex and according to other relevant project-specific parameters.
Therefore, the data collection for this evaluation will contain various tools and methodologies (both quantitative and qualitative approaches). The consultant is expected to develop a detailed methodology (to be endorsed by Tdh) for this evaluation. We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders including children.
Some of the data collection tools expected to be used during the evaluation are:
- Desk review: The consultant(s) will also conduct a desk study/ literature review of all available documentation in preparation for the evaluation, including review of different programme documents, such as monitoring, quarterly, annual, and study reports, progress reports, mid-term evaluation report, technical training reports, project proposals, MEAL framework, etc. and analyzing secondary data. The evaluation methodology should make use of different data already generated by Tdh program and MEAL team. The information and findings of the desk review should be integrated with the data and findings from primary data collection and analysis, and not placed in a separate section.
- Interviews with Key Informants: The team are expected to conduct a range of interviews with key informants and stakeholders and will visit and interview relevant project staff, partner staff, governmental actors who were included in the project activities, Tdh project staff in Korogocho ..etc. a number of meetings will need to be conducted Tdh focal points to guide the consultant to prepare the inception report and identity actors for the data collection.
- Focus group discussions: The team will conduct focus group discussions with direct and indirect beneficiaries of the project including project beneficiaries, partner staff, and project staff.
- Beneficiaries’ Surveys: The team is expected to conduct surveys to the beneficiaries of the project. The consultant will devise an appropriate sampling method to select sample beneficiaries from the different target beneficiary groups and they will conduct the survey with sampled beneficiaries. We anticipate that this evaluation will be a participatory review and learning exercise. Thus, it requires the consultant(s)/firms to be experienced in participatory approaches to learning and inquiry, and especially in seeking the views and perceptions of key stakeholders that include:
- Targeted beneficiaries
- Partners and actors directly involved in the project at different levels:
- The local implementing partners
- Community leaders and gatekeepers (if applicable) and representative bodies of the target population (with due consideration of the possible lack of women’s representation among leaders and bodies).
- Relevant local and County authorities and other relevant duty bearers.
Data collection tools and appropriate sampling methodology should be prepared by the consultant and shared with the Tdh focal point for comment and approval. The data collection tools proposed by the consultant should be able to respond to the key evaluation questions mentioned above.
Samples for beneficiary surveys should be selected based on a statistically relevant sample per the survey system, that is, a 95% confidence level and a confidence interval of 5) with the total beneficiary count per intervention as the sampling frame.
The evaluators/external consultants should take into account the following methodological considerations;
- The evaluator’s understanding of the evaluation questions
- Description of the phases related to the evaluation approach proposed.
- Sources of information for primary and secondary data collections.
- Sampling strategies, including area and population group represented in the sample, sampling procedures, and sample size.
- Instruments to be used for data collection.
- Different types of data analysis will be carried out.
- Reference indicators and benchmarks for each evaluation question (if relevant).
6.0 EVALUATION TIMEFRAME AND DELIVERABLES.
The evaluator should produce:
- Inception report as per the firms/ consultant’s template including the tools/materials/templates. The inception report MUST include the detailed evaluation work-plan and evaluation matrix.
- Sharing detailed data collection plans and tools in time for Tdh to be able to conduct quality assurance.
- Sharing ALL the cleaned qualitative and quantitative data collected from the different sources. This should include code books.
- Briefing and debriefing meetings in addition to the routine meetings and discussions with the Tdh team.
- The final evaluation report (based on the firms/consultant`s template, incorporating comments and technical inputs from the reference group) with a summary of the evaluation brief (2-pager at maximum) and a power point presentation detailing the purpose, process, findings, and recommendations of the evaluation.
- Updated results framework that clearly shows the status/value of Project indicators visa-vis the targets
To facilitate the process, Tdh will ensure;
- Timely review and validation of the inception report.
- Timely review and validation of data collection tools and data collection approaches.
- Beneficiary lists to facilitate sampling for the beneficiary surveys are provided.
- Provision of requisite documents for the desk reviews i.e., project reports, evaluation reports, project proposal and theoretical frameworks for the project.
- Provision of research assistants/ enumerators (to be paid by the consultant) in case they will be required.
The timeline provided for this is 6 weeks from awarding of the tender.
REQUIREMENTS
A consultancy firm with valid registrations, and it should have legal registration paperwork in Kenya and formal access to the locations of the programme implementation.
The selected firm/ consultant should possess the following minimum qualifications as follows:
Qualifications and experience
Academic skills
Master's degree in advanced applied research/evaluation methods/ economics, social sciences, or any related academic discipline or an affiliation with a research institution or a university, holding a PhD, or being in pursuit of a PhD in a relevant field is an advantage.
Previous Experience
- 5 of years evaluating development programmes
- At least two evaluation contracts of similar value, nature and complexity implemented over the last three years or more.
- Strong background in monitoring and evaluation techniques and ideal experience child protection programming in development contexts.
- Conversant with the context in Nairobi`s informal settlements will be an added advantage.
- Experience in developing and implementing Evaluations with the UN, International NGOs or donors.
- Familiarity with the OECD/DAC evaluation framework.
- Excellent knowledge and experience in survey design, implementation of surveys and statistical data analysis.
- Excellent analytical, communication, writing and presentation skills in English and Swahili.
- Ability to analyse complex interventions.
Accountability
- Creates a respectful office environment free of harassment and retaliation and promotes the prevention of sexual exploitation and abuse (PSEA).
- Accepts and gives constructive criticism.
- Follows all relevant procedures, processes, and policies related to the organisational principles.
- Meets deadline, cost, and quality requirements for outputs.
- Monitors own work to correct errors or incorporate inputs.
- Takes responsibility for meeting commitments and for any shortcomings.
Orientation
- Identifies the immediate and peripheral programme staff of own work.
- Establishes and maintains productive working relationships with staff.
- Identifies and monitors changes in the needs of evaluation, including donors, governments and project beneficiaries.
- Keeps staff/managers informed of developments and setbacks related to the evaluation.
How to apply
SUBMISSION OF APPLICATION/EXPRESSION OF INTEREST
The interested firm/ consultant should submit a technical proposal with a detailed evaluation methodology, indicative work plan, and the overall approach to the evaluation and an all-inclusive budget proposal no later than October 27th 2022. The submission of proposals (technical and financial) and/or related questions should be directed via procurement email to: ken.tenders@tdh.ch
The submission should include the followings:
- Company/ Consultant profile including a history of similar projects (if applicable);
- A cover letter;
- CV and biographies of independent consultant/consulting firm and key assessment team members (if any);
- At least two proof of previous experiences for contracts of similar value, nature and complexity implemented over the last three years or more.
- References for each evaluation team member or the firm;
- An example of a recent evaluation report.
- Proof of registration of the consultant/ consultancy firm