Digital Standards Compliance

This section is in progress.

The UNDP Digital Standards provide guidance for UNDP teams on best practices when creating digital solutions for development.

The Performance App has been reviewed against each of these standards to ensure it is in compliance.

Out of the 90 "dos" from each of the ten Digital Standards, the Performance App is in compliance with xx

1. Start with the Need.

Building something that is not needed is the most significant risk in digital. Most technology projects fail. This is not because the technology doesn’t work. It’s because something has been built that people don’t need or can’t use.

The Performance App has followed the guidance for this standard.

• Interview at least 5 people who directly experience the problem. Make sure your questions are not biased.

The team has interviewed over 100 UNDP team members at every level, including the Executive Office, Regional Bureaus, and Country Offices. This includes both the leadership and operational levels

• Experience the problem first-hand — where possible.

CPU (Corporate Performance Unit), which is the unit behind the Performance App, has been making manual reports on various different datasets that are now available in the Performance App. Previously, this was an extremely manual process that was slow, cumbersome, and did not provide real-time data.

• Write a clear Problem Statement to frame the problem.

The problem is that UNDP uses varied dashboards across different offices, leading to inconsistent performance monitoring, lack of real-time data, and decision-making challenges.

• Use quantitative research, where possible, to measure the magnitude of a problem. This type of research can help you determine whether the problem you are trying to solve impacts enough people. If it doesn't, then you might need to rethink your approach.

Quantitative research was employed to determine the impact and scope of the problem. It confirmed that the inconsistencies in performance monitoring affect all levels of UNDP globally, which justifies the development of the Performance App. For instance, there is a wide discrepancy between the KPIs and metrics that are monitored across each of the five Regional Bureaus.

• Understand users’ technology constraints (smartphone use, cost, availability of internet, digital literacy, network infrastructure, etc.)

Because the Performance App is an internal UNDP application, and all offices and bureaus have internet connectivity and staff that have digital literacy, there is not a significant amount of technology constraint on the project

• Create User Journeys that show how things are currently done.

Dozens of interviews were carried out, and detailed pain points and existing state analyses were drawn out.

• List and prioritize the current pain points.

We have reviewed the current pain points across the different operational levels at UNDP, from the Executive Office, Bureau, and Country Office. The decision was made to order the deployment from Global -> Country -> Bureau based on the urgency of the pain points and the overall scale.

• Review previously attempted solutions.

Previously attempted solutions, such as Throughline, did not go live for various reasons. These have been reviewed and the current approach has been tailored based on the lessons learned

• When working with the government, ensure solutions are co-created.

While this is not directly applicable to the Performance App, as this is an internal UNDP application, we have ensured that we are co-creating solutions with end users by hosting workshops, showcasing and testing design prototypes, and focusing strongly on the vision while being flexible on the details. We have updated numerous indicators based on specific user feedback with additional data points, filters, and views.

2. Bridge the Digital Divide

• Speak to people from groups more likely to be excluded

Not applicable to the Performance App as the target audience is only UNDP team members.

• Build for low literacy when possible by using visual signals and illustrative content instead of heavy text

Overall, the Performance App relies mostly of visuals, with some accompanying explanation text where it has been deemed required.

• Observe real people using an early version of the solution to make sure it’s accessible

The Performance App has gone through extensive testing from September 2023 to April 2024.

• Consider barriers and pain points where users might be excluded and work out how to address them

Not applicable to the Performance App as the target audience is only UNDP team members.

• Look at examples of when similar digital solutions have excluded people — and why

Not applicable to the Performance App.

• Research the digital divide, and prioritize which barriers are most important in your context

Not applicable to the Performance App as the target audience is only UNDP team members.

• Create a plan to tackle the digital divide and set measurable targets proactively

Not applicable to the Performance App as the target audience is only UNDP team members.

• Regularly review if your solution is unintentionally excluding people, and make sure someone (usually the product owner) is accountable for addressing them

Not applicable to the Performance App as the target audience is only UNDP team members.

• Localize your application by translating into local languages

This has not yet been done for the Performance App, but the future vision is to ensure that it is available in the main UN languages.

• When working with the government, encourage a whole-of-government approach (vs. building for one sector/ministry only).

While not directly applicable for the Performance App, we have considered how to build something useful for all Bureaus and Country Offices instead of a small subset.

3. Test Early and Often

• Base your decisions on actual user data and feedback

• Create prototypes before building anything

Figma prototypes are built for each feature before development

• Gather feedback on early ideas from at least 5 users

• Create a list of your assumptions

• Test any existing solutions you plan to use to make sure they can be adapted

• Conduct usability testing to ensure you understand how the solution can be improved before, during, and after going live (launch)

• Start small and leave room to iterate and re-evaluate decisions

4. (Perhaps) Don’t Build It

• Check if other people in UNDP have developed or procured a similar tool. Ask the UNDP Digital Advocates Network if you are unsure.

Similar tools have been developed (e.g., POCOMAS, UNDP-at-a-Glance, STREAM), but they are smaller in scope and not for monitoring overall performance across the organization.

• Check the Digital X Solutions Catalogue, DIAL catalogue and the Digital Public Goods Registry for similar solutions

No solutions were found that could be leveraged for the Performance App.

• Check if ready-made commercial solutions are available

There are no specific commercial solutions available apart from general dashboarding software such as PowerBI. A detailed analysis was made to review the performance epic technology against PowerBI, and it was found that PowerBI was not suitable for the long-term vision.

• Review all three options and report on your findings

A report was drafted and submitted to the team at ITM. This report reviewed the different approaches for the technology, including commercial systems and building something from scratch using open source technology.

• Create a Lean Digital Impact Canvas (see here)

• Consult with technical experts

Consultations with the Chief Digital Office and ITM were undertaken to review the technology options.

5. Do No Harm

• Consider potential human rights impacts and take mitigation measures.

• Consider potential harms for the planet and take mitigation risks.

• Considering long-term consequences of your digital projects.

• Think carefully about how you present choices (especially default choices) in any technology solution and user interface.

• Make sure that you follow the UNDP Data Principles when collecting data from users.

• Map out the potential first, second, third, and fourth-order potential impacts of implementing any solution and look for ways technology could do harm to different populations, environments or ecosystems. Make sure you consider the whole life-cycle of any solution, including energy use and material use and disposal.

• Have a fresh pair of eyes (ideally someone who is not involved in the project) to check for biases and assumptions as well as what could possibly go wrong.

• Be aware of biases in the datasets and technology that you feed into your projects.

• Understand the political environment and cultural context into which your product will be launched and use this understanding to map social risks.

• Define mitigation strategy and protection mechanisms for the identified potential harm.

• Make sure a good understanding of the context your design will be part of and the power dynamics at play within it.

• Consider when it is necessary to incorporate UN Human Rights Due Diligence on Digital Technologies (guide to follow in 2023).

• Read the Ledger of Harms by the Center for Human Technology. This will provide you with an understanding that technology itself has a certain degree of agency, and that the unintended consequences can often outweigh the original intended goals.

6. Form the Right Team

• Understand the expertise you need

The Project Document specifically laid out the required team members for this project.

• Map the skill set you have in your team

• Include people with lived experience

• Interview vendors before contracting them

The technology vendor chosen for the Performance App has previously worked with UNDP on multiple successful intiatives .

• Meet with vendors regularly (a minimum twice per month)

There is a regular technical meeting to plan sprints each week, as well as ad-hoc meetings for specific issues or discussions.

• Ensure vendors continuously share insights and research that support decision-making

• ensure sufficient internal capacity is in place to manage the work

7. Measure What Matters

Interesting here is that this is on two levels, what we measure as indicators in the Performance App and also what we measure in terms of usage.

• Track the number of users, their gender, and level of engagement, whenever possible

Number of page visits and unique users are tracked.

• Install analytics (i.e., Google Analytics)If you have a web or mobile component,

Number of page visits and unique users are tracked.

• Feed data into a PowerBI Dashboard automatically

• Collect data on product crashes/outages (i.e. Crashanalytics)

• Define your success metrics before launch

• Check your metrics and course correct your product/service regularly

• Use data to make informed design and business decisions

• Establish feedback loops using qualitative insights developed through ongoing testing and user feedback, as well as web and programme analytics

• Track metrics from disaggregated data (while respecting privacy) to understand where the digital divide might be showing up (i.e, the product works well for male users but no female users are completing the journey)

8. Follow the UNDP Data Principles

• Have a third-party security audit on your product

• Refer to the data guiderails to verify your solution complies with UNDP Data Principles

• Ensure that the process aligns with the policy and procedures available on data privacy and information classification

• Minimize the collection of private data

The Performance App does not collect any private data on individuals who use the app or within any of the data. For instance, all HR data is fully aggregated to ensure that no personal details can be leaked even accidentally.

• Check the data catalogue available to leverage the existing datasets to avoid collecting information that already exists

Almost all data in the Performance App is pulled from the UNDP Data Warehouse, ensure that there is one source of truth and no data duplication.

• Get specific consent from users, and ensure that they know what data you are collecting and how it will be used

• Plan for reusability and interoperability through using standards

All data in the Performance App is downloadable via CSV, and there is an API planned in the future.

• Allow users to request to delete their data entirely

Not applicable to the Performance App.

9. Default to Open

• Publish a public roadmap

Our roadmap is available here.

• Publish release notes

Our monthly release notes are available here, and a full detailed changelog is available here.

• Check if a similar tool already exists in the DIAL Catalogue, the Digital Public Goods Catalogue, Digital X Catalogue or the country or context you work in.

No solutions were found that could be leveraged for the Performance App.

• Apply an open-source code license to your code and make it freely available.

Not applicable to the Performance App as the platform is specific to UNDP.

• Publish your code to the UNDP Github repository.

The code is stored in UNDP Azure development, not on UNDP GitHub. There has been a choice to not share the code publicly until a security audit has been undertaken.

• Where possible, turn your solution into a Digital Public Good

Not applicable to the Performance App as the platform is specific to UNDP.

• Write about your learnings

This has not been done.

10. Plan for the Long Term

• Store code in the UNDP Github repository

The code is stored in UNDP Azure development, not on UNDP GitHub. There has been a choice to not share the code publicly until a security audit has been undertaken.

• Ensure software code is documented

• Check the skills that exist locally to maintain the tool in the long-term and support digital ecosystems

• Think about interoperability from the start

• Ensure you have budgeted for a minimum six-month maintenance period

The project document for the Performance App provides funding for a three year period.

• Documentation needs to be available online, not as an email attachment

The full methodology and SOP (Standard Operating Procedure) documentation for the Performance App is available online — you're reading it!

• Clarify roles and responsibilities with vendors and implementing partners for ownership

• Clarify roles and responsibilities with institutional partners for ownership

There are no institutional partners for the Performance App.

• Design and implement a user-engagement plan from the start of the product development; and remember to adapt your plan often

• Plan for time and budget for the user engagement plan

• Have clear goals for the number of users who will use your product/service

• Have a clear budget for growing your product/service

• Track your cost per user acquisition for each growth channel

• Consider the long-term business model of the product