Observing the data analytics projects that have been, or are currently being run by the UK's ODAs, we believe they can be classified into three different themes:

• Cashable savings and transforming working practice
• Focus on vulnerability or reducing threat, risk and harm
• Predictive analytics or making better use of AI and machine learning

Although the direction of the project may arise from one of these three areas, it is likely that the benefits will span all three. Or, it may be that as the project grows and the discovery phase is completed, there are other considerations, objectives or benefits that will be met that weren’t on the original agenda.

The pilot for the Essex Centre for Data Analytics (ECDA) sets out to reduce the threat, risk and harm to potential victims of modern slavery through creating a predictive model using business inspections data.

During the discovery phase, it was highlighted that before any predictive algorithm was attempted, a wider problem was the lack of any consistent sharing of business inspections data between agencies.

Therefore, the first outcome of a pilot is now aimed at producing a multi-agency picture of business inspections to identify a broader range of risk factors that can improve business practices. This will also help reduce vulnerability further down the line by laying the foundations for building the predictive product originally envisaged, with much higher chances of success.

Therefore, the potential for the pilot has now become:

• Cashable savings and transforming working practice
• Focus on vulnerability or reducing threat, risk and harm
• Predictive analytics or making better use of AI and machine learning

In addition to these themes, our research has identified a number of recurrent topics currently or already considered by ODAs.

Each ODA approached these in slightly different ways and achieved different outcomes, but it is interesting to note that there are recurring themes in project choices, highlighting the need for shared learnings between ODAs.

Domestic abuse (Essex prior to ECDA through Essex Data Board, WODA, nYODA).

• Primary theme: Focus on vulnerability or reducing threat, risk and harm;
• Other themes: Cashable savings and transforming working practise, and Predictive analytics or making better use of AI and machine learning.

School readiness (Essex, prior to ECDA through Essex Data Board, WMODA, GMODA, nYODA).

• Primary theme: Cashable savings and transforming working practise;
• Other themes: Predictive analytics or making better use of AI and machine learning, and Focus on vulnerability or reducing threat, risk and harm.

Gangs and serious organised crime, early intervention (Essex, prior to ECDA through Essex Data Board, and The Office for Data Analytics - Avon and Somerset).

• Primary theme: Focus on vulnerability or reducing threat, risk and harm;
• Other themes: Predictive analytics or making better use of AI and machine learning and Cashable savings and transforming working practise.

Business and inspection (ECDA pilot on business inspections, SODA business rates forecasting model, and WODA business intelligence register).

• Primary theme: Cashable savings and transforming working practise;
• Other themes: Predictive analytics or making better use of AI and machine learning, and Focus on vulnerability or reducing threat, risk and harm.

We have observed an appetite for sharing project ideas across the ODA network. ODAs would therefore ideally adopt an open approach to the selection and execution of pilots. This means sharing code and being transparent about selection criteria, project execution and lessons learned so that they can be replicated by other regions.

The case studies in section three provide a complete list and description of the projects.

Evaluation

Evaluation is often overlooked or conducted hurriedly at the end of a project. This is a mistake. Good evaluation is vital as it helps identify what works and increases the chance that the best interventions will be refined and scaled. The purpose of testing and evaluation is to trial the data product or intervention in a real-world setting and measure the results.

According to the Magenta Book, the UK Government’s essential guide on evaluation design in the public sector, there are three broad categories of evaluation: process, impact, and economic. The appropriate type will depend on what needs to be learnt, as well as the resources and expertise that needs to be dedicated to the evaluation.

In general, a process evaluation will provide an understanding on how and why an intervention has an impact, while an impact evaluation will measure the change that has occurred, and whether the improvement has been directly caused by the intervention. Following a robust impact evaluation, an economic evaluation is possible to monetise the observed outcomes.

During the pilot for the London Office of Technology and Innovation (LOTI) City Data Analytics Programme (formerly LODA), a randomised control trial was designed to test whether building inspectors could find more unlicensed Houses of Multiple Occupation, using a list created by an algorithm, than they could through their normal business practice.

Detailed guides on conducting good evaluations are available from Nesta (‘Using Research Evidence: A Practice Guide’), from Central Government (‘Magenta Book’, ‘Green Book’), and user-friendly websites, such as BetterEvaluation.

Benefits

In section one, we discussed the types of benefits Offices of Data Analytics can provide: cost savings, increased collaboration, insight and evaluation. However, at the business case design stage, these will need to be made more specific and relevant to each project.

Benefits Mapping

Benefits mapping is a useful method to link ODA projects and programmes to one or more organisations’ strategic aims and could become a useful resource when looking for investment and gaining leadership support. Illustrating the link between project deliverables, new capabilities, potential process changes, and organisational outcomes, benefits mapping should be undertaken at the beginning of the project, in the project conception and initiation phases.

Cost Benefit Analysis

Cost-benefit analysis (CBA) is a useful technique to compare the total costs of the programme/project with its benefits, using a common metric (most commonly monetary units), in order to calculate the net cost or benefit associated with the programme. However, CBA has posed a challenge for ODAs in development across the UK. This is partly due to the complexity of monetising the more intangible benefits such as, safeguarding of individuals or improving public satisfaction.

There are a number of cost benefit tools that can be used - such as those employed in HM Treasury’s ‘The Green Book’.