We’ve carried out a long-term evaluation of our Destination Local Action Research in Audience Analytics project, to understand whether the training, experiments, resources and other support provided has had a lasting impact on the 10 hyperlocal publishers that took part.

In April 2016 we published the Action Research in Audience Analytics: Project evaluation and learnings report, when the project came to an end. 10 hyperlocal media service providers received grant funding to take part in the project, undertaking monthly experiments, participating in peer learning activities and receiving training, resources and other non-financial support.

The aim of the project was to explore how a more strategic use of website and social media analytics might help participants define and measure their success online; improve their services, audience engagement and reach; and in turn strengthen their sustainability and legacy within their local community.

Participants agreed to take part in a long-term evaluation 12 months after the project began (October 2015), to help us understand how (and if) they’re still using the skills and knowledge learned, whether they’re continuing to experiment with different platforms and tools, and what challenges or opportunities they've come across on the back of taking part.

As part of publishers’ participation in the evaluation, they have:

• Provided Nesta with data about their service from the project start date, in October 2015, to October 2016, including qualitative and quantitative data.
• Allowed us to publish and share the results from the evaluation, to determine the long-term impact of participating in the project, and to share impact and learning with other hyperlocal media services to benefit the wider sector.

This 12-month evaluation, like the initial project evaluation, has helped to answer these five core questions / objectives:

• What barriers are there to hyperlocal media publishers analysing their audience and content?
• How does training, upskilling and trial participation help hyperlocal media publishers overcome these barriers?
• What evidence do the trials provide about local audiences and their consumption of hyperlocal media?
• What evidence does this research provide about the cost-benefit of audience and content analysis, and its impact on the commercial and social value proposition of services, including services’ ability to demonstrate their value to potential partnerships with established media groups and local services?
• Development and sharing of learning resources for the long-term benefit of and use by the wider hyperlocal media sector.

And to answer two other questions about the impact of the project on the individual participants, in light of the above:

• Did participants, overall, achieve their objectives and desired impact (based on what they said in early data capturing)?
• What’s the value of developing communities of practice like this?

Participants were surveyed and their responses, together with all the project data we aggregated and analysed previously, have informed further conclusions about the impact of their participation on their service, as well as the effectiveness of the project as a whole.

Their insights have also informed new, and echoed previous, recommendations – for practitioners, funders, policymakers and industry.

A summary of findings / conclusions:

The full evaluation can be found here. These notes refer to slides from the full presentation: 

1. What barriers are there to hyperlocal media publishers analysing their audience and content?

• The ‘catch 22’ of needing time and resources to research and implement changes. [See slide 14 for details of barriers].

2. How does training, upskilling and trial participation help hyperlocal media publishers overcome these barriers?

• Participation in the action research project has, overall, had a positive impact on participants’ services, and is likely to have had a direct positive impact on online audience engagement in the last 12 months. [See slides 7, 9, 16 & 18 for details of services’ progression and overcoming of barriers].

3. What evidence do the trials provide about local audiences and their consumption of hyperlocal media?

• Audiences are increasingly discovering (and engaging with) hyperlocal content through social media channels, and through search engines.
• There is also growth in audiences subscribing to and engaging with hyperlocal content through email newsletters and mailing lists. [See slides 7, 12, 14 & 16 for details of audience consumption].

4. What evidence does this research provide to the wider sector about the cost-benefit of audience and content analysis and its impact on the commercial and social value proposition of services?

• There is significant evidence that audience and content analysis has a direct positive impact on hyperlocal media services’ social and commercial value proposition. [See slides 7, 12, 16 & 18 for evidence of ‘return of investment’ of audience and content analysis].
• However, there is minimal evidence that audience and content analysis has a direct positive impact on services’ generation of revenue, or their ability to demonstrate their value to potential partnerships with traditional / established media groups.

5. Development and sharing of learning resources for long-term benefit of and use by the wider hyperlocal media sector.

• The resources produced as part of the project have been shared and used widely in the UK and beyond by hyperlocal publishers and supporting organisations.
• One publisher, Star & Crescent, gained international awareness through being invited to contribute at a Google News Lab-supported workshop for Swedish mainstream journalists in September 2016.

Here, we list the recommendations that have been informed by the long-term evaluation. Many of the recommendations were also reflected, either wholly or partially, in our initial evaluation. Read our previous evaluation.

Recommendations for practitioners (see slide 25)

• Undertake audience and content analysis, and experimentation of different tools, platforms and ways of working.
• Dedicate time to learning and upskilling in relation to audience and content analysis and experimentation.
• Go where your audience is, and explore how best to make the most of the platforms where they are.
• Set some short-term and long-term objectives for your service and set out to measure your progress and impact against these objectives.

Recommendations for funders (see slide 26)

• Provide financial and non-financial support for training and technical support, facilitating resource sharing, and to encourage collaboration and experimentation by hyperlocal practitioners.
• Collaborate with established, dominant media and tech organisations to encourage them to support hyperlocal publishers, i.e. Google’s DNI Innovation Fund and through other tailored community news / independent publisher / social innovation initiatives.

Recommendations for policymakers and industry (see slide 26)

• The hyperlocal media sector should prioritise developing communities of practice on a larger scale. These could be developed around geographic clusters of practitioners and / or thematic areas of interest
• Implement policy changes that support hyperlocal and community news services.

We hope the conclusions and recommendations in this evaluation will benefit other hyperlocal publishers in the UK and beyond, and the organisations that support them.
 

This is the final resource produced by Nesta for the Destination Local programme, which concluded in spring 2016. To discuss this evaluation with the author, email Kathryn Geels, or get in touch on Twitter at @girlondon.