On 24 May, Nesta invites you to City Data: From Analytics to AI, a day-long event exploring the latest data innovations helping cities and local governments provide better services. View the details and register your interest here.

In advance of that event, this article explores four steps we have found to be helpful for public sector organisations considering running a data analytics project. I am indebted to my former colleagues Hilary Simpson and Nevena Dragicevic, whose ideas and wording have significantly informed the below.

Public sector organisations face many different challenges. How can you tell which of them might be tackled with data analytics?We believe that successful data analytics projects consist of four elements:

Let’s explore each in turn.

KEY POINT: It’s vital to move from large, macro-level issues to something specific and actionable.

To help narrow down a problem, it’s helpful to understand five problem types where data analytics is particularly well-suited to help.

The analytics team in New Orleans' Office of Performance and Accountability have helpfully outlined some specific problem types, and the opportunities related to them (see their version here).

Your specific problem statement should not be in the form of a question, but phrased as follows:

KEY QUESTION: What would you do differently if you had all the information you needed about your specific problem?

It’s important to focus on actions and interventions that are within your control to change (e.g. no single organisation can ‘solve’ homelessness – but you might help address a specific aspect of it in your area.)

Drill down to precisely who will act, and where and when they will do so.

List all the actions or interventions that you would ideally like to put in place to address your specific problem, as follows:

KEY QUESTION: What would a person need to see on a screen in order to enable the actions defined in the previous step?

It’s unlikely that whoever is doing the action (e.g. a frontline worker or service manager) will want a spreadsheet or raw data. Instead they will want the data conveyed in a more intelligible way that provides a real insight – that’s what we mean by a ‘data product’.

A data product could be a map, a heatmap, a prioritised list, an alert and so on.

Certain data products are suited to certain problem and opportunity types.

You can now see whether an insight from a particular data product could enable one or more of the actions you outlined.

KEY QUESTION: What data do you need to create the data product, does it exist, can you get it, and can you use it?

Data can come from a number of sources, such as:

• Open data
• Public sector
• Businesses & Third Sector
• Citizens

You can use a template like the one below to brainstorm what datasets might be available from these different sources.

If the data you need does not seem to exist, you may wish to consider:

1. Are there other datasets that might contribute a similar type of information, or act as a proxy measure?
   
2. Could you start collecting this data so that analysis becomes more feasible in future?

Once you have determined what data you’d ideally like to use, you need to follow a robust process to determine whether it is legal, ethical and acceptable to do so, and that it can be done securely. Nesta will be providing more templates on how to think about those questions very soon.

It’s best practice to carry out a Privacy Impact Assessment (PIA). A PIA is a standard series of screening questions that guides users through the potential risks and benefits of sharing personal data. The PIA equally prompts users to develop mitigation strategies to minimise potential downsides of information sharing. This editable PIA is provided by the Information Commissioner's Office (ICO).

Wherever possible, it’s better and far simpler to use non-personal data. If you must use personal data, an important step is to identify the legal gatewaysthat grant your organisation the permission or authority to pursue certain objectives, which could be supported by the sharing of personal data.

If the source data is personal, it may be possible to remove personally-identifiable information and aggregate the data to reduce the risks associated with using it. Good guidance on data anonymisation and pseudonymisation is available in the Research Ethics Guidebook.

When sharing data among partners, whether it involves a few teams within an organisation or multiple public sector organisations working together, a common set of rules and conditions should be developed in the form of a Data Sharing Agreement, also known as an Information Sharing Protocol - ISP).

ISPs are necessary whenever personal data is shared, but are equally recommended for the sharing of non-personally identifiable data.

The essential elements to be covered in an ISP are:

• The purpose of the sharing
• The potential recipients and the circumstances in which they will have access
• The exact data to be shared
• Data quality – accuracy, relevance, usability, etc.
• Data security
• Retention of shared data
• Individuals’ rights – procedures for dealing with access requests, queries and complaints
• Review of effectiveness/termination of the sharing agreement
• Sanctions for failure to comply with the agreement or breaches by individual staff

Templates for ISPs are available from the ICO

Based on these steps, the diagram below shows how you may have to adapt your data product based on what data you can use.

Having been through this process, you should have a four-part statement that outlines the problem you wish to solve, and how data can enable a better intervention.

If you're interested in how public sector organisations can use data analytics to improve services, you might like to join Nesta on 24 May for our event City Data: From Analytics to AI. View the details and register your interest here!

Find Eddie on Twitter