Evaluation Management
Introduction
This indicator takes information from the ERC (Evaluation Resource Center) to measure the performance in managing evaluation, including the timely conduct of evaluations, the completion and implementation of management responses, and the quality of decentralised evaluations.
This is both a Global, Regional, and Country Office level indicator.
Organisational Objective
The organisational objective can be broken down into three parts:
Planned evaluations completed on time.
Completed evaluations have management responses with no overdue key actions.
Evaluations are of acceptable quality.
Notes
We exclude UNV, UNCDF, UNOSSC
Calculation of Scoring
The Evaluation Management Indicator is divided into three distinct components, each aligned with a specific organizational objective. These components are as follows:
Evaluation Planning: This part assesses the efficiency and timeliness of the evaluation scheduling process, ensuring that evaluations are completed as planned.
Evaluation Management Responses: This section measures the promptness and effectiveness of management in responding to the evaluations conducted, emphasizing the importance of timely action on feedback and recommendations.
Evaluation Quality: The final component evaluates the overall quality of the evaluations themselves, focusing on the depth, accuracy, and usefulness of the findings and conclusions drawn.
Evaluation Planning
30%
Evaluation Management Responses
40%
Evaluation Quality
30%
These components provide a holistic view of an organization's evaluation management capabilities, reflecting its commitment to continuous improvement and accountability.
Part 1: Evaluation Planning (30% weight)
This focuses on the punctuality and scheduling discipline of the evaluation process, specifically measuring the percentage of evaluations completed within their designated timeframes. This assesses the effectiveness of evaluation planning and execution at the Country Office level.
The target for the year is typically set in ERC by 31st December of that year. So from January 1st to March 31st, we score based on the previous year's achievement of the target. From April 1st to March 31st of the following year, we score based on the current year's achievement of the target.
Statuses
Planned: Not started yet
Initiated ToR: The ToR is now being written
Team Onboard: Consultants have been hired
Completed: Evaluation has been completed
An evaluation can only have one status at a time.
The calculation of this indicator is:
Completed Evaluations / Evaluation Planned for the year = % of Completed Evaluations
This is based on a linear % of the target from April 1st to March 31st of the following year and we count the % of completed evaluations against this linear trendline. Above the trendline, we give 100 points for this component, below the trendline, we deduct 1 point for every 1% below the trendline.
Part 2: Evaluation Management Responses (40% weight)
A) % Evaluations without a management response 6+ weeks after evaluation (50% of Part 2)
This component evaluates the proportion of evaluations lacking a management response more than six weeks after completion. It is a key indicator of the efficiency and timeliness with which management acts on evaluation findings.
The formula used is:
1 - (Number of Evaluations Overdue by More Than 6 Weeks / Total Completed Evaluations in the Programme Period) = Percentage of Evaluations with a Management Response
For Global/Bureau: We take the % of evaluations without a management response more than six weeks after evaluation. For each 1% below 100%, we deduct 4% from the score.
For Country: Country offices are scored based on the number of overdue management responses (6+ weeks after evaluation completion):
0
100
1
75
2
50
3
25
4+
0
B) Key actions timings (50% of Part 2)
For part 2B, we count the % of key actions that are not completed within 18 months of the evaluation.
For Global/Bureau: We take the % of key actions that are not completed within 18 months of the evaluation.
For Country:For each 1% below 100% completed, we deduct 5% from the score.
Part 3: Evaluation Quality (40% Weight)
This section determines the overall quality of evaluations based on a comprehensive scoring system. Evaluation quality is critical, reflecting the evaluation's depth, accuracy, and usefulness.
Six distinct scores are possible, each reflecting a specific level of evaluation quality.
High Satisfactory
Satisfactory
Moderately Satisfactory
Moderately Unsatisfactory
Unsatisfactory
Highly Unsatisfactory
The calculation of the scoring is that for the last calendar year (so 2024 when it is 2025): confirm this with IEO
Total Highly Satisfactory and Satisfactory / Total Evaluation Reports
50% or above is 100 points, below 50% is 2 points per 1% below 50%.
Overall Calculation
Part 1: Evaluation Planning (30% of final score)
Evaluations are scored against a linear trendline based on the annual target from April 1st to March 31st:
Performance at or above the trendline = 100 points
Performance below the trendline = 100 minus 1 point for each 1% below the trendline
Part 2: Evaluation Management Responses (40% of final score)
This section comprises two equal components (Part 2A and Part 2B), each contributing 50% to Part 2:
Part 2A: Management Response Timeliness (20% of final score)
Global/Bureau: For each 1% below 100% compliance, deduct 4 points from the score
Country: Scored based on number of overdue management responses using discrete bands (0 = 100 points, 1 = 75 points, 2 = 50 points, 3 = 25 points, 4+ = 0 points)
Part 2B: Key Actions Completion (20% of final score) Measures the percentage of key actions completed within 18 months of the evaluation:
Global/Bureau: Score based on % of key actions completed within 18 months
Country: For each 1% below 100% completed, deduct 5 points from the score
The Part 2 score is calculated as: (Part 2A + Part 2B) / 2
Part 3: Evaluation Quality (30% of final score)
Based on the proportion of evaluations rated as Highly Satisfactory or Satisfactory in the last calendar year:
50% or above = 100 points
Below 50% = 100 minus 2 points for each 1% below 50%
Final Score Calculation
The final score combines these three parts with their respective weights:
Final Score = (Part 1 × 0.3) + (Part 2 × 0.4) + (Part 3 × 0.3)
This calculation method ensures that each aspect of the evaluation process—planning, management response, and quality—is appropriately considered in the overall performance score.
The traffic light indicator methodology for this indicator is:
Green = 90+
Yellow = 70+
Red = <70
Useful Links
Last updated