TOR for an External Evaluation of EA31/2016
These Terms of Reference should be adapted to the context. These can also be used as a resource to guide internal reviews.
Catholic Caritas Foundation of Nigeria EA 31/2016/Nigeria/Lake Chad Basin Crisis Final External Evaluation
This section should include a brief description of:
-the history of the programme, including its objectives, main activities, budgetary information, duration;
The Emergency Appeal Programme as launched by Caritas Internalis (EA31/2016) was borne out of the huge humanitarian crisis in the Lake Chad Basin of Nigeria, occassioned the massive scale of hostilities orchestrated by some terrorist groups in the North-East of Nigeria, popularly known as Boko Haram. UN OCHA in 2015/2016 reported 2.2 million persons as being displaced by this crisis, with a slight improvement to about 1.7 million persons are still internally displaced as at 2017, there are still large scale displacements in Borno and Admawa States with reports of forced returns of some IDPs to areas that are still not very safe and secure. There have been continuous reports of massive violations of human rights on a daily basis. The food security situation has been extremely concerning, as conflict continues to limit the amount of land under cultivation and 3.7 million persons are expected to face critical levels of food insecurity during the upcoming lean season (June through September 2018).
Project Objective: Improved access to Nutrition, Livelihoods, WASH, and Health of 9,000 vulnerable IDPs, returnees and host community members in Hawul and Askira Uba local government areas of Borno State, in North Eastern Nigeria.
– The relationship between the programme and past/future interventions;
-The main stakeholders of the programme (e.g. CI MOs; Diocesan Caritas, local Church, local government, the UN, peer agencies etc.); the partnerships and frameworks of collaboration;
The Project is run with support from CI-MOs. Caritas Nigeria is the coordinating body with direct interface with Caritas Internationalis, while JDPC Maiduguri is the direct implementing body. The project is implemented using a vast array of Community mobilizers, Volunteers, Field officers and Extension Agents, who work to ensure overall project success.
-The rationale for the evaluation.
Amongst many reasons, the rationale for the proposed Evaluation includes the following:
- The reason activities have or have not been implemented as planned or were adapted?
- Did the intervention have an impact? Why or why not? How and for whom did it have an impact?
- To what extent can the measured or observed changes be attributed to the intervention?
- Did the intervention have any unintended consequences?
- Is the intervention cost effective? Can the cost be compared with alternatives to investment, in other words, could the results have been reached with less input?
If the intervention was successful, can it be replicated to other settings and if so, in which settings? Can it be adapted, replicated or built on to increase its reach or scope (for a larger population or a different region)?
The final evaluation will focus on the following key objectives:
- Assess the relevance, appropriateness, effectiveness, accountability and impact/sustainability of the programme;
- Assess the effectiveness of the leadership of Caritas Nigeria in coordinating the dioceses and CI MOs within the EA31 2016
- Identify lessons learned, best practices and recommendations to inform future programme design.
- Was programme design based on an impartial assessment of needs? Are needs assessments disaggregated by age, sex and disability? Do they include people’s needs, vulnerabilities and capacities?
- Did the assistance provided by Caritas Nigeria meet the needs of the affected population? Were the persons most in need identified, selected, and supported by the programme?
- Which parts of the assistance were the most appropriate and why? Which were least appropriate and why? Were activities aligned with the affected population’s needs and priorities?
- Were recommendations and learning from past reviews and evaluations applied to the response?
- Was the response timely?
- What internal and external factors affected the speed of the response?
- Was the internal organizational and managerial structure of the project effective?
- Were there appropriate systems in place to monitor activities, outputs and outcomes of the programme? Did monitoring outcomes inform programme adjustments/revisions?
- Did the project activities lead towards the achievement of the expected results/indicators as set in the Results Framework?
- To what extent has the affected population been involved in the design or implementation of the programme?
- Were appropriate systems of downwards accountability (participation, information sharing and feedback/complaints), put in place and used by project participants? Were project participants aware of the feedback/complaints mechanism?
- Were project participants and communities aware of selection criteria?
- Were project participants and communities aware of the assistance they should receive?
- How effective was Caritas Nigeria in coordinating internally?
- How effective was Caritas Nigeria in coordinating the dioceses and CI MOs?
- How effective was Caritas Nigeria in coordinating with external stakeholders such as other agencies, organisations, the local and national government?
- What aspects of coordination could be improved in the future and how?
- Has Caritas Nigeria’s response strengthened local capacities?
- What are the intended and unintended, positive and negative effects of the project?
- What, if any, aspects of the programme will have a longer-term impact?
The final evaluation will rely on two main evaluation stages (1) design phase (2) field phase
- The evaluator(s) will undertake a desk review of programme documentation including planning documents, project proposals, situation reports and quarterly reports;
- The evaluator(s) will also review other relevant documentation such as minutes of decision-making meetings, 3W mappings of Caritas Nigeria’s response, consolidated situation reports of Caritas Nigeria’s response (if applicable);
- The evaluator(s) will review other monitoring and reporting documents from secondary sources (i.e UNHCR reports).
- If appropriate, the evaluator(s) will obtain feedback from the CI HD on the most relevant sites to visit.
- After the design phase, the evaluator(s) will conduct fieldwork to collect and analyse data in order to answer the evaluation questions;
- Data collection methods should be inclusive and utilise a range of methods, including focus group discussions and key informant interviews and with key project stakeholders. The use of surveys and other remote data collection tools should also be explored by the evaluators to maximise data collection;
- The evaluator(s) should ensure a systematic triangulation of data sources and data collection methods and tools, and seek to validate data through regular exchanges with programme staff where appropriate.
- EXPECTED OUTPUTS
The evaluator(s) should produce the following key deliverables:
- Draft Evaluation Report to be submitted to Caritas Nigeria/ CI support mechanism (if applicable) and the CI Humanitarian Department
- Final Evaluation Report inclusive of:
- Executive Summary
- Description of Methodology
- Main findings
- Conclusions inclusive of best practices and lessons learned
USE OF THE EVALUATION RESULTS
The intended audience for the evaluation are Caritas Nigeria key staff, including senior management, CI MOs who have supported the programme, the Caritas Internationalis Humanitarian Department and the Caritas Confederation. Evaluation findings will be shared with programme participants as appropriate.
The competencies required from the External Evaluator are:
- Advanced degree in social sciences, political sciences, economics, development or related fields;
- Experience in leading evaluations, especially in the field of humanitarian response;
- Ability to use participatory approaches to evaluation;
- Experience of operational management of humanitarian/development programmes;
- Good knowledge of the local context;
- Good analytical skills;
- Excellent writing skills in a CI Confederation language (EN/FR/ES);
- Any other appropriate language skills;
- Understanding of the Catholic Church and Caritas structure and mission.
Proposals should include:
- Proposed evaluation methodology (if different from above);
- Description of deliverables and a timeline;
- A financial proposal including the cost implication for other evaluators if any;
- CV(s) of evaluator(s).
Add as many lines as needed to prepare Time Frame plan
|Evaluation phase||Activities||Dates||# of Days|
|Preparatory||Initial Rapid Assessments||11-06-2018||1|
|2.||Set-up of project control mechanisms||12-06-2018||1|
|3.||Selection of priority sectors and locations||12-06-2018||1|
|Field phase||Project Start-up Sensitization||13-06-2018||1|
|2.||Formation and engagement of Community Implementation committee.||13-06-2018||1|
|4.||Monitoring and Evaluation||13-06-2018||1|
|Reporting||1st interim report||14-06-2018||1|
|2.||2nd interim report||14-06-2018||1|
|3.||3rd interim report||14-06-2018||1|
|4.||Consolidated Situation Reports||14-06-2018||1|