In the blog below, Laura Bunt, Head of Policy Research and Communications at Citizens Advice, writes about why the charity decided to tap into their wealth of data to achieve greater impact with the help of DataKind and some data scientist volunteers. Here she offers some insights to other organisations considering doing the same. 

Throughout the past year, someone talked to Citizens Advice about a housing problem every two and a half minutes. Between July and September this year, 22,000 of those issues were about problems with the private rented sector, most frequently difficulties with essential maintenance and repairs. We know that of the 2,000 people who came to see us facing eviction despite being up to date with their rent, half had dependent children. We have seen a 20 per cent increase in the number of people coming to see us about this problem since the same period last year, an indication that the private rented market is in some serious need of attention. 

With our network of Citizens Advice Bureau in 3,000 community locations across England and Wales, a national phone service and online Advice Guide, Citizens Advice sees first-hand the challenges people face in their everyday lives. The impacts of poor policy design or delivery; the struggle to manage high energy bills, steep rents or rising food prices; people’s worries about fragile employment, rocketing debts, tough choices at retirement or relationship advice. Last year a third of the UK’s online population sought online advice from Advice Guide, giving us a direct insight into the questions and challenges that are keeping too many people awake at night.  

Image courtesy of Citizens Advice: Real-time searches for online advice from Citizens Advice (Click on image for larger version - for a live version click here)

Combine the breadth and depth of data Citizens Advice generates with the technical skills and tools of data science, and you quickly see the potential for impact. That is why we leapt at the chance to work with DataKind and their network of volunteer data scientists to run two weekend ‘data dives’ over this summer, to explore the scope of Citizens Advice data and to prototype new methods and products that could help us to understand more about the problems facing the country. As a charity already driven by data and insight from people’s experience, we wanted to explore how we could do more with data, and set out to develop new tools to help us respond in real-time.    

The products from the events are still in development (getting ready for launch in early 2015), but here are five important lessons we have learnt so far:

1. Ask the questions that really matter 

As any researcher knows, the insight drawn from data will depend on the quality of the question. Setting the challenge for a data dive is no different. Work hard to define your question properly, and be sure that the answer is something you care about as an organisation. For us, we wanted to learn how Citizens Advice could identify problems early, to act as an ‘early warning system’ and spot emerging trends to help us act to prevent problems escalating and causing greater harm.

It may be tempting to cast the net wide and simply see what comes out of the data, but we found it easier to organise teams and collaborate when working towards a common, clear goal. And by making the question relevant to the organisation it is more likely that the results will be adopted. 

1. Be open to the unexpected

​It is often difficult to make the space for experimentation. Whilst in developing the question you want to be as clear and specific as possible, in beginning to explore avenues with the data be open to different ideas and methodologies. An early visualisation may give you an idea for a new route for analysis, or an approach you thought promising may quickly turn into a dead-end. Keep flexible and responsive to new findings, working together as a team to adjust plans as the project evolves. 

When we started the project, our assumption about what product and analysis would be useful was very different to where we ended up. The application of a dashboard was the result of a final sprint at the last data dive, prompting us to pivot as we quickly realised that this was the way forward.   

1. Method isn’t everything; focus on solving the problem 

The pace of technological development has accelerated our ability to analyse data in increasingly sophisticated ways. Over the course of two weekends working with Citizens Advice data, teams of volunteer coders, data analysts, designers and subject experts ran semantic analysis of free text to make sense of people’s experiences, trend analysis of years of advice code data, geo-spatial analysis, comparisons between different datasets, features analysis on Google analytics data, forecast and prediction models, and many different visualisations and data design tools to aid interpretation. The strength of the approach came from bringing together different skills, methods and tools rather than from any one individual method; the important thing is to keep coming back to the question and working out whether the findings help us to answer it in a robust and helpful way.  

1. Put in the hard work to make data accessible and safe 

We take client confidentiality very seriously. Whilst keen to involve as many people as possible in the data dive, we were clear that this would not breach our commitments to anonymity and privacy and spent considerable time upfront making sure that the data was safe to share. But appropriate and important concern for confidentiality should not preclude openness and collaboration. In October’s data dive, we shared and linked data from Citizens Advice and St Mungo’s Broadway to test if this would help us to learn more about routes into and out of homelessness (and ultimately how to prevent people becoming homeless). Getting the data into shape was a difficult task, but worth the investment to make it safe to explore links without exposing identifiable data. 

1. Your people are your sustainability  

It’s a familiar challenge for any innovation project; integrating new ways of working, new ideas or products back into the wider organisation. After the intensity of the data dive weekend fades away, how do you ensure that work continues and keeps momentum? 

As well as developing genuinely useful products, one of the lasting impacts of the data dive for Citizens Advice was the sense of opportunity it generated within our data analyst teams. Taking part in the events exposed them to new skills and techniques they hadn’t used before, and encouraged them to reflect on how they work as a team and with the rest of the organisation. The team who took part in the event are already planning their own data dive for February, and are trialling online tools that will help to institute new methods into how we explore our data long into the future.   

The work doesn’t stop here. Over the next few weeks, Citizens Advice and DataKind are working together with a team of committed volunteers to develop and refine a flexible, online ‘dashboard’ to visualise trends and common issues, and to help Citizens Advice spot problems when they emerge. 

Huge thanks to DataKind, Peter Passaro, Iago Martinez, Ian Ansell and Pete Watson for their hard work to make this happen, and to Nesta for providing the funding to support the work. 

Photo Credit: BobMical via Compfight cc