Evaluation Management
Introduction
This indicator takes information from the ERC (Evaluation Resource Center) to measure the performance in managing evaluation, including the timely conduct of evaluations, the completion and implementation of management responses, and the quality of decentralised evaluations.
This is both a Global, Regional, and Country Office level indicator.
Organisational Objective
The organisational objective can be broken down into three parts:
Planned evaluations completed on time.
Completed evaluations have management responses with no overdue key actions.
Evaluations are of acceptable quality.
Notes
We exclude UNV, UNCDF, UNOSSC
Calculation of Scoring
The Evaluation Management Indicator is divided into three distinct components, each aligned with a specific organizational objective. These components are as follows:
Evaluation Planning: This part assesses the efficiency and timeliness of the evaluation scheduling process, ensuring that evaluations are completed as planned.
Evaluation Management Responses: This section measures the promptness and effectiveness of management in responding to the evaluations conducted, emphasizing the importance of timely action on feedback and recommendations.
Evaluation Quality: The final component evaluates the overall quality of the evaluations themselves, focusing on the depth, accuracy, and usefulness of the findings and conclusions drawn.
Evaluation Planning
30%
Evaluation Management Responses
40%
Evaluation Quality
30%
These components provide a holistic view of an organization's evaluation management capabilities, reflecting its commitment to continuous improvement and accountability.
Part 1: Evaluation Planning (30% weight)
This focuses on the punctuality and scheduling discipline of the evaluation process, specifically measuring the percentage of evaluations completed within their designated timeframes. This assesses the effectiveness of evaluation planning and execution at the Country Office level.
The target for each year is typically set in ERC by December 31 of the previous year. However, it can be updated daily until March 31 of the same year; updates made after that date are not considered for reporting.
Until January 31, the score is based on the previous year’s achievement of the target. From February 1 to January 31 of the following year, we score based on the current year’s achievement of the target.
Completed evaluations, even if submitted late, are not condered overdue and do not penalise the score.
The evaluations considered for the current year are those whose planned end dates fall within that calendar year.
Statuses
Planned: Not started yet
In Progress: Evaluation Team onboard
Initiated ToR: The ToR is now being written
Completed: Evaluation has been completed
Overdue: Passed planned evaluation end date.
An evaluation can only have one status at a time.
The calculation of this indicator is:
100 * Number of current‑year evaluations that have been completed and whose planned end date was before today / Total number of current‑year evaluations with a planned end date before today = % of Completed Evaluations
100 * Number of current‑year evaluations that have been not completed and whose planned end date was before today / Total number of current‑year evaluations with a planned end date before today = % of Overdue Evaluations
So the score is 100 - (% of Overdue Evaluations), minimum at 0.
The global score is not the average of the CO scores; it is calculated using all evaluations that meet the given conditions.
The Bureau/Country Office score is calculated using all evaluations relevant to the specific bureau/country office.
Part 2: Evaluation Management Responses (40% weight)
A) % Evaluations without a management response 6+ weeks after evaluation (50% of Part 2)
This component evaluates the proportion of evaluations lacking a management response more than six weeks after completion. It is a key indicator of the efficiency and timeliness with which management acts on evaluation findings.
The formula used is:
Global / Bureau
Percentage of Overdue Evaluations Management Responses = 100 * (Number of current-year evaluations with management responses overdue as of today but not yet completed / Total number of current-year evaluations with management responses overdue as of today) Sore = * 100 - Percentage of Overdue Evaluations Management Responses
Country Office
Country offices are scored based on the number of overdue management responses (6+ weeks after evaluation completion):
0
100
1
75
2
50
3
25
4+
0
B) Key actions timings (50% of Part 2)
For part 2B, We consider all evaluations from the past five years, including the current year-end, and include all planned actions except those marked as ‘No longer applicable.’
% Overdue key actions = Number of key actions that were planned to be due by today but are still in Overdue or Not Initiated status / Total number of key actions that were planned to be due by today exlcuding No longer applicable.
For Global/Bureau: For each 1% below 100% completed, we deduct 1% from the score.
For Country:For each 1% below 100% completed, we deduct 5% from the score.
Part 3: Evaluation Quality (40% Weight)
This section determines the overall quality of evaluations based on a comprehensive scoring system. Evaluation quality is critical, reflecting the evaluation's depth, accuracy, and usefulness.
Six distinct scores are possible, each reflecting a specific level of evaluation quality.
High Satisfactory
Satisfactory
Moderately Satisfactory
Moderately Unsatisfactory
Unsatisfactory
Highly Unsatisfactory
The calculation of the scoring is that for the last calendar year (so 2024 when it is 2025): confirm this with IEO
Excluding evaluation without QA
Total Highly Satisfactory and Satisfactory / Total Evaluation Reports QA
50% or above is 100 points, below 50% is 2 points per 1% below 50%.
Overall Calculation
Part 1: Evaluation Planning (30% of final score)
Part 2: Evaluation Management Responses (40% of final score)
Part 2A: Management Response Timeliness (20% of final score)
Part 2B: Key Actions Completion (20% of final score)
The Part 2 score is calculated as: (Part 2A + Part 2B) / 2
Part 3: Evaluation Quality (30% of final score)
Final Score Calculation
The final score combines these three parts with their respective weights:
Final Score = (Part 1 × 0.3) + (Part 2 × 0.4) + (Part 3 × 0.3)
This calculation method ensures that each aspect of the evaluation process—planning, management response, and quality—is appropriately considered in the overall performance score.
The traffic light indicator methodology for this indicator is:
Green = 85+
Yellow = 70+
Red = <70
Note: this applies for all subindicator color scoring.
Useful Links
Last updated