Country Office Performance Audits
Data Owner | Narayanan Chettiar (narayanan.chettiar@undp.org), OAI |
Availability in Data Warehouse | Available |
Data Refresh Rate | Daily |
Accountability Weighted Scoring | 30% |
Introduction
Country Office Audits are audits performed by OAI (Office of Audit and Investigations). They measure compliance with corporate rules and regulations. The Performance App takes data from 2018 onwards for all the Country Office audits captured in CARDS and published on the Audit Public Disclosure website. We showcase the rating from the latest country office audit, as some country offices may have had multiple audits during that period.
There are four main possible audit ratings:
Fully Satisfactory
Minor Improvements
Major Improvements
Unsatisfactory
The underlying data has more categories, but we have mapped it as follows:
Old Categories | New Categories |
---|---|
Fully Satisfactory | Fully Satisfactory |
Good | Fully Satisfactory |
Satisfactory | Fully Satisfactory |
Partially Satisfactory/SI | Satisfactory/Some Improvement Needed |
Partially Satisfactory | Satisfactory/Some Improvement Needed |
Satisfactory/Some Improvement Needed | Satisfactory/Some Improvement Needed |
Marginally Deficient | Partially Satisfactory/Major Improvement Needed |
Partially Satisfactory/Major Improvement Needed | Partially Satisfactory/Major Improvement Needed |
Partially Satisfactory/MI | Partially Satisfactory/Major Improvement Needed |
Seriously Deficient | Unsatisfactory |
Unsatisfactory | Unsatisfactory |
Deficient | Unsatisfactory |
An audit will result in a set of recommendations that are tracked in a system called CARDS, managed by OAI. The Country Office is responsible for implementing these recommendations and providing status updates via CARDS within specific timeframes. Once the updates are provided, OAI evaluates them to confirm whether the recommendations have been implemented.
The recommendations are flagged according to the following traffic-light methodology based on OAI's assessment of the implementation status:
Time Frame | Status | Indicator Color |
---|---|---|
0-12 months | Great | Green |
12-18 months | Acceptable | Yellow |
18+ months | Unacceptable | Red |
Organisational Objective
The organisational objective is the best audit performance possible, with no unsatisfactory audits and maximising entirely satisfactory audits. Additionally, the fastest and comprehensive implementation of the audit recommendations.
Data
Check that we are not missing the field here for the actual main result of the audit
Field | Description |
---|---|
as_of_date | Date of data extraction or last update. |
audit_categ | Category or type of audit (e.g., OAI - Office of Audit and Investigations). |
audited_bureau | Bureau or regional office audited (e.g., RBAS - Regional Bureau for Arab States). |
audited_org_id_c | Unique identifier for the audited organization. |
audited_cty_code | ISO 3166-1 alpha-3 country code (e.g., MAR - Morocco). |
audited_unit_desc | Description of the audited unit. |
action_bureau | Bureau responsible for audit action. |
action_org_id_c | Identifier for the action organization. |
action_cty_code | Country code for the action location. |
action_unit_desc | Description of the action unit. |
action_responsible_person | Name or title of the responsible person. |
audit_id | Unique audit identifier. |
audit_title | Title of the audit. |
audit_period_start | Start date of the audit period. |
audit_period_end | End date of the audit period. |
audit_issue_date | Date when the audit report was issued. |
audit_type | Type of audit conducted (e.g., COA - Country Office Audit). |
reco_no | Number of the recommendation. |
recom_id | Unique recommendation identifier. |
reco_title | Title or summary of the recommendation. |
reco_descr | Detailed recommendation description. |
reco_planned_imnpl_d | Planned implementation date for the recommendation. |
reco_priority | Priority level (e.g., High, Medium, Low). |
reco_oai_status | Status of the recommendation (e.g., Implemented, In Progress, Not Implemented, Withdrawn). |
reco_open | Binary indicator of recommendation status (0 = closed, 1 = open). |
sub_domain | Specific organizational domain (e.g., SRF - Strategic Results Framework). |
Data Export
Requires definition
Calculation of Scoring
We score only using each country's latest CO audit reports. Previous historical audit ratings are shown in the indicator drill down but are not considered in the scoring methodology.
For each audit result, we assign the following scores:
Audit Result | Score |
---|---|
Fully Satisfactory | 100 |
Satisfactory / Some Improvement Needed | 80 |
Partially Satisfactory / Major Improvement Needed | 60 |
Unsatisfactory | 0 |
Unsatisfactory is rated as zero. We want to significantly reduce the aggregate scores if even a small number of country offices in a region have Unsatisfactory audit ratings because these are so important to the organisation.
For Regional Bureau aggregation, we take an average of all the scoring, and the global average is calculated as an average of all country scores, not as an average of the Regional Bureau aggregation. This is to avoid skewed results, as different Regional Bureaus have different numbers of country offices. No weight is attached to country population size, amount of programming, or office size/budget.
Traffic Light System
Traffic Light | Score |
---|---|
Green | 85+ |
Yellow | 70+ |
Red | <70 |
Limitations & Future Improvements
Not incorporating country size, budget, programming amount, and other key details into weightings could fail to account for larger country offices' naturally higher complexity and risk exposure. Weightings may better reflect performance. An office with many projects and budgets receiving an Unsatisfactory audit rating is far more important to the organization than a smaller office receiving an Unsatisfactory audit rating — but this is currently not reflected in the methodology.
Looking only at the latest audit per country loses historical context. Incorporating a rolling 3-5-year average could smooth out anomalies and show improvement/regression over time.
No qualitative context is provided on why audits scored the way they did or what actions are being taken. Incorporating audit insights could make the data more meaningful.
We do not take into account the management responses in the scoring methodology, but we do show it in the drill-down
Resources
CARDS (Requires invitation from OAI)
Last updated